Charon/docs/plans/revert_ci_pipeline.md at 979e464b0cbecb4e1a70a3860727c2c363a4e153

Files

GitHub Actions 3169b05156 fix: skip incomplete system log viewer tests

- Marked 12 tests as skip pending feature implementation
- Features tracked in GitHub issue #686 (system log viewer feature completion)
- Tests cover sorting by timestamp/level/method/URI/status, pagination controls, filtering by text/level, download functionality
- Unblocks Phase 2 at 91.7% pass rate to proceed to Phase 3 security enforcement validation
- TODO comments in code reference GitHub #686 for feature completion tracking
- Tests skipped: Pagination (3), Search/Filter (2), Download (2), Sorting (1), Log Display (4)

2026-02-09 21:55:55 +00:00

9.9 KiB

Raw Blame History

title, status, scope, notes

title	status	scope	notes
Revert CI Pipeline Consolidation	draft	ci/workflows, integration, e2e, security	Restore per-workflow pull_request triggers, retire ci-pipeline.yml, and reestablish self-contained image builds.

1. Introduction

This plan dismantles the consolidated CI pipeline and restores individual pull_request triggers for component workflows. The goal is to return to a simple, independent workflow model where each integration or test workflow runs on PRs without relying on a central pipeline or shared image artifacts.

Objectives:

Identify workflows that had pull_request triggers removed or were merged into ci-pipeline.yml.
Restore per-workflow pull_request triggers for integration, E2E, and build workflows.
Delete ci-pipeline.yml as the required path to retire the consolidated pipeline.
Ensure each workflow is self-contained for image availability.

2. Research Findings

2.1 Current Consolidated Pipeline

.github/workflows/ci-pipeline.yml runs on pull_request and bundles lint, image build, integration tests, E2E, coverage, CodeQL, Trivy, supply-chain scans, and gates.
The pipeline builds and uploads an image artifact for integration and uses e2e-tests-split.yml via workflow_call.

2.2 Integration Workflows (Current State)

.github/workflows/cerberus-integration.yml: workflow_dispatch only.
.github/workflows/crowdsec-integration.yml: workflow_dispatch only.
.github/workflows/waf-integration.yml: workflow_dispatch only.
.github/workflows/rate-limit-integration.yml: workflow_dispatch only.
Each workflow currently pulls a registry image and tags it as charon:local. There is no pull_request trigger and no local build step.

2.3 E2E Workflows (Current State)

.github/workflows/e2e-tests-split.yml: workflow_call + workflow_dispatch only. No pull_request trigger.
The build job can build an image locally when invoked directly, but the file is currently only invoked by ci-pipeline.yml.

2.4 Build Workflow (Current State)

.github/workflows/docker-build.yml: workflow_dispatch only.
This workflow is designed to be the main build pipeline but is not currently triggered by pull_request.

2.5 Security Workflows (Current State)

.github/workflows/security-pr.yml: workflow_dispatch only.
.github/workflows/supply-chain-pr.yml: workflow_dispatch only.
.github/workflows/codeql.yml: schedule + workflow_dispatch only.
These workflows include logic for push and pull_request contexts but their triggers do not include pull_request.

2.6 Historical Reference

.github/workflows/e2e-tests.yml.backup and .github/workflows/e2e-tests-split.yml.backup show prior pull_request trigger patterns and path filters that can be restored.

3. Technical Specifications

3.1 Workflow Inventory and Trigger Restoration

Target workflows to restore pull_request triggers:

Notes:

e2e-tests-split.yml should run directly on pull_request with the internal build job enabled, not only via workflow_call.
security-pr.yml and supply-chain-pr.yml must include pull_request triggers so security coverage is not lost.
codeql.yml needs a decision: re-enable pull_request in codeql.yml or leave CodeQL in a separate PR workflow. The consolidated pipeline is currently the only PR CodeQL path.

3.2 ci-pipeline.yml Decommission Strategy

Decision:

Option A (required): delete ci-pipeline.yml to fully end the consolidated pipeline and avoid duplicate PR checks.

3.3 Image Availability Strategy (Critical Challenge)

Independent PR workflows cannot rely on a shared image from another workflow unless using artifacts or a registry. The user wants to avoid pipeline complexity.

Required behavior for each integration workflow:

Restore the "Build Docker image (Local)" step in each integration workflow, reverting any artifact handover dependency.
Build a local Docker image within the workflow before tests run.
Tag the image as charon:local for consistency with existing scripts.
Avoid external registry dependency for PR builds.

Impacted workflows:

cerberus-integration.yml
crowdsec-integration.yml
waf-integration.yml
rate-limit-integration.yml

E2E workflows:

e2e-tests-split.yml already supports building an image locally when invoked directly. Ensure pull_request triggers route through this path (not workflow_call).

3.4 Pull Request Trigger Scope and Path Filters

Use branch filters consistent with prior backups and docker-build.yml usage: main, development, feature/, hotfix/.
Apply path filters for E2E to avoid unnecessary runs: frontend/, backend/, tests/**, playwright.config.js, .github/workflows/e2e-tests-split.yml.
Integration workflows typically run on any backend/frontend changes. Consider adding path filters if desired, but default to full PR runs for parity with previous behavior.

3.5 Dependency and Concurrency Rules

Remove workflow_run coupling to docker-build.yml for integration and E2E workflows. Each workflow should be independently triggered by pull_request.
Keep job-level concurrency where it prevents duplicate runs on the same PR, but avoid cross-workflow dependencies.

4. Implementation Plan

Phase 1: Baseline Verification (Tests)

Confirm current CI behavior for PRs: identify which checks are now only running via ci-pipeline.yml.
Capture baseline PR check set from GitHub Actions UI for comparison after restoration.

Phase 2: Restore PR Triggers (Core Workflows)

Add pull_request triggers to docker-build.yml with branches including main and development.
Add pull_request triggers to cerberus-integration.yml, crowdsec-integration.yml, waf-integration.yml, and rate-limit-integration.yml.
Add pull_request triggers to e2e-tests-split.yml, using the backup trigger block as the source of truth.

Phase 3: Make Integration Workflows Self-Contained

Restore the "Build Docker image (Local)" step in each integration workflow and remove dependency on ci-pipeline.yml artifacts.
Remove registry pull steps or make them optional for manual runs.
Ensure test scripts continue to reference charon:local.

Phase 4: Security Workflow Triggers

Add pull_request triggers to security-pr.yml and supply-chain-pr.yml as a mandatory requirement to preserve PR security coverage.
Decide on CodeQL: either add pull_request to codeql.yml or create a dedicated PR CodeQL workflow. If the pipeline is deleted, CodeQL must have an alternative PR trigger.

Phase 5: Decommission ci-pipeline.yml

Delete ci-pipeline.yml.

Phase 6: Validation and Audit

Verify that PRs show the restored individual checks instead of a single pipeline job.
Confirm each integration workflow completes without relying on registry or artifact inputs and includes the restored local build step.
Validate E2E workflow runs directly on pull_request with build job executed locally.
Confirm security workflows run on pull_request.

5. Acceptance Criteria (EARS)

WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger docker-build.yml directly on pull_request for main and development.
WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger cerberus-integration.yml, crowdsec-integration.yml, waf-integration.yml, and rate-limit-integration.yml on pull_request.
WHEN an integration workflow runs on pull_request, THE SYSTEM SHALL restore and run the "Build Docker image (Local)" step, build a local Docker image, and tag it as charon:local before tests.
WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger e2e-tests-split.yml directly on pull_request without relying on ci-pipeline.yml.
WHEN the consolidated pipeline is retired, THE SYSTEM SHALL NOT run ci-pipeline.yml on pull_request.
WHEN a pull_request is opened or updated, THE SYSTEM SHALL run security-pr.yml and supply-chain-pr.yml on pull_request.
WHEN CodeQL is required for pull_request, THE SYSTEM SHALL run a CodeQL workflow on pull_request independent of ci-pipeline.yml.

6. Risks and Mitigations

Risk: PR checks increase in parallel count and runtime. Mitigation: use path filters for E2E and consider optional filters for integration workflows.
Risk: Image build duplication increases CI cost. Mitigation: keep builds scoped to workflows that need the image, and avoid registry pushes for PR builds.
Risk: Security scans or CodeQL no longer run on PR if triggers are not restored. Mitigation: explicitly re-enable PR triggers in security workflows or add a dedicated PR security workflow.

7. Confidence Score

Confidence: 82 percent

Rationale: The workflow inventory and trigger gaps are clear. The main uncertainty is selecting the final CodeQL and security trigger model once ci-pipeline.yml is removed.

9.9 KiB Raw Blame History