cicd
Last updated: 2025-04-02 09:04:17.914437 File source: link on GitLab
NuNet CI/CD Pipeline Documentation
This document provides an overview of the CI/CD pipeline infrastructure for the NuNet project. It is intended for developers, DevOps engineers, and SREs who will maintain and extend this system.
Table of Contents
Project Structure
The CI/CD configuration is organized into modular files that define specific aspects of the pipeline:
Pipeline Stages and Jobs
The pipeline is divided into logical stages, each containing specific jobs:
1. Pre-Build Stages
Stages:
.preJobs:
Prep Semver Config: Prepares versioning configuration for the pipeline.DefectDojo Check Product Exists: Ensures product exists in DefectDojo.
2. Code Quality and Linting
Stages:
code_qualityJobs:
golangci_lint: Runs Go linting with golangci-lint.license_check: Verifies software licenses using go-licence-detector.
3. Unit Tests
Stages:
unit_testsJobs:
Golang Unit Tests: Executes unit tests for Go codebase.JavaScript Unit Tests: Runs JavaScript unit tests with Mocha.Golang Coverage Report: Generates coverage reports for Go code.
4. Integration and E2E Tests
Stages:
integration_tests_1,e2e_tests_1Jobs:
go_integration_tests: Runs integration tests for Go services.go-e2e-tests: Executes end-to-end tests for the system.
5. Security Testing
Stages:
security_tests_1Jobs:
semgrep-sast: Runs Semgrep security analysis.container-scanning-defectdojo: Performs container image scanning with Trivy.secret-detection-defectdojo: Detects secrets in source code.
6. Build Stages
Stages:
buildJobs:
Build: Builds Debian packages for Linux distributions.build:osx: Builds Darwin binaries for macOS.
7. Feature Environment Management
Stages:
test,functional_testsJobs:
run_feature_environment: Spins up feature environments for testing.feature_environment_test_results: Collects and aggregates test results.
8. Post-Build Stages
Stages:
.postJobs:
delete_build_artifacts: Cleans up temporary build artifacts.update_code_coverage: Updates code coverage information in the repository.
Configuration Variables
The pipeline uses a variety of variables to control its behavior. These can be customized in your GitLab project settings under CI/CD > Variables.
Build Configuration
BUILD_DISABLED: Enable/disable builds (default: false).ALLOW_BUILD_FAIL: Allow build failures without blocking the pipeline (default: false).
Testing Configuration
SKIP_UNIT_TESTS: Skip unit tests (default: false).ALLOW_E2E_TESTS_FAIL: Allow E2E test failures (default: false).E2E_TESTS_DISABLE: Disable E2E tests (default: false).
Security Configuration
DEFECTDOJO_DISABLED: Disable DefectDojo integration (default: false).SAST_EXCLUDED_ANALYZERS: List of analyzers to exclude from SAST scans.SECRET_DETECTION_DISABLED: Disable secret detection (default: false).
Feature Environment Configuration
FEATURE_ENVIRONMENT_BRANCH: Branch for feature environment builds (default: release).FEATURE_ENV_CONFIG_VAULT_PATH: Path to feature environment configuration in Vault.
Reporting and Notifications
SLACK_WEBHOOK: Webhook URL for Slack notifications.TESTMO_URL: URL for Testmo integration.CI_REPORTS_SSH_KEY_B64: SSH key for report distribution.
Maintenance Guidelines
Keep It DRY:
Avoid duplicating configuration across files. Use templates and shared variables wherever possible.
Versioned Configuration:
Major changes to the pipeline should be versioned to ensure backward compatibility.
Monitor Performance:
Regularly review job execution times and optimize long-running jobs.
Security Best Practices:
Rotate credentials regularly.
Use scoped tokens with minimal permissions.
Documentation:
Keep this README updated whenever significant changes are made to the pipeline.
Testing Changes:
Test configuration changes in a staging environment before deploying them to production pipelines.
Technical Components
The CI/CD pipeline integrates with several key tools and services:
GitLab CI/CD:
Core orchestration engine for the pipeline.
Manages stages, jobs, and dependencies between tasks.
Allure:
Test reporting and visualization tool.
Provides detailed insights into test results and coverage.
SonarQube:
Code quality analysis tool.
Detects code smells, bugs, and vulnerabilities.
DefectDojo:
Vulnerability management platform.
Integrates with security testing tools to manage findings.
Testmo:
Test result aggregation and reporting tool.
Provides a unified view of test results across the pipeline.
Trivy:
Container image scanner for vulnerabilities and misconfigurations.
Semgrep:
Static application security testing (SAST) tool.
GolangCI-Lint:
Linter for Go codebases.
Enforces coding standards and best practices.
This documentation provides a comprehensive overview of the CI/CD pipeline structure, configuration options, and maintenance guidelines. For more detailed information about specific jobs or stages, refer to the individual .gitlab-ci.yml files in the cicd/Jobs directory.
Last updated