Charon/docs/plans/revert_ci_pipeline.md

---
title: "Revert CI Pipeline Consolidation"
status: "draft"
scope: "ci/workflows, integration, e2e, security"
notes: Restore per-workflow pull_request triggers, retire ci-pipeline.yml, and reestablish self-contained image builds.
---

## 1. Introduction

This plan dismantles the consolidated CI pipeline and restores individual
pull_request triggers for component workflows. The goal is to return to a
simple, independent workflow model where each integration or test workflow
runs on PRs without relying on a central pipeline or shared image artifacts.

Objectives:

- Identify workflows that had pull_request triggers removed or were merged
  into ci-pipeline.yml.
- Restore per-workflow pull_request triggers for integration, E2E, and
  build workflows.
- Delete ci-pipeline.yml as the required path to retire the consolidated
  pipeline.
- Ensure each workflow is self-contained for image availability.

## 2. Research Findings

### 2.1 Current Consolidated Pipeline

- [.github/workflows/ci-pipeline.yml](.github/workflows/ci-pipeline.yml)
  runs on pull_request and bundles lint, image build, integration tests,
  E2E, coverage, CodeQL, Trivy, supply-chain scans, and gates.
- The pipeline builds and uploads an image artifact for integration and
  uses e2e-tests-split.yml via workflow_call.

### 2.2 Integration Workflows (Current State)

- [.github/workflows/cerberus-integration.yml](.github/workflows/cerberus-integration.yml): workflow_dispatch only.
- [.github/workflows/crowdsec-integration.yml](.github/workflows/crowdsec-integration.yml): workflow_dispatch only.
- [.github/workflows/waf-integration.yml](.github/workflows/waf-integration.yml): workflow_dispatch only.
- [.github/workflows/rate-limit-integration.yml](.github/workflows/rate-limit-integration.yml): workflow_dispatch only.
- Each workflow currently pulls a registry image and tags it as
  charon:local. There is no pull_request trigger and no local build step.

### 2.3 E2E Workflows (Current State)

- [.github/workflows/e2e-tests-split.yml](.github/workflows/e2e-tests-split.yml): workflow_call +
  workflow_dispatch only. No pull_request trigger.
- The build job can build an image locally when invoked directly, but
  the file is currently only invoked by ci-pipeline.yml.

### 2.4 Build Workflow (Current State)

- [.github/workflows/docker-build.yml](.github/workflows/docker-build.yml): workflow_dispatch only.
- This workflow is designed to be the main build pipeline but is not
  currently triggered by pull_request.

### 2.5 Security Workflows (Current State)

- [.github/workflows/security-pr.yml](.github/workflows/security-pr.yml): workflow_dispatch only.
- [.github/workflows/supply-chain-pr.yml](.github/workflows/supply-chain-pr.yml): workflow_dispatch only.
- [.github/workflows/codeql.yml](.github/workflows/codeql.yml): schedule + workflow_dispatch only.
- These workflows include logic for push and pull_request contexts but
  their triggers do not include pull_request.

### 2.6 Historical Reference

- [.github/workflows/e2e-tests.yml.backup](.github/workflows/e2e-tests.yml.backup) and
  [.github/workflows/e2e-tests-split.yml.backup](.github/workflows/e2e-tests-split.yml.backup) show prior pull_request
  trigger patterns and path filters that can be restored.

## 3. Technical Specifications

### 3.1 Workflow Inventory and Trigger Restoration

Target workflows to restore pull_request triggers:

- [.github/workflows/docker-build.yml](.github/workflows/docker-build.yml)
- [.github/workflows/cerberus-integration.yml](.github/workflows/cerberus-integration.yml)
- [.github/workflows/crowdsec-integration.yml](.github/workflows/crowdsec-integration.yml)
- [.github/workflows/waf-integration.yml](.github/workflows/waf-integration.yml)
- [.github/workflows/rate-limit-integration.yml](.github/workflows/rate-limit-integration.yml)
- [.github/workflows/e2e-tests-split.yml](.github/workflows/e2e-tests-split.yml)
- [.github/workflows/security-pr.yml](.github/workflows/security-pr.yml)
- [.github/workflows/supply-chain-pr.yml](.github/workflows/supply-chain-pr.yml)
- [.github/workflows/codeql.yml](.github/workflows/codeql.yml) (decision point)

Notes:

- e2e-tests-split.yml should run directly on pull_request with the
  internal build job enabled, not only via workflow_call.
- security-pr.yml and supply-chain-pr.yml must include pull_request
  triggers so security coverage is not lost.
- codeql.yml needs a decision: re-enable pull_request in codeql.yml or
  leave CodeQL in a separate PR workflow. The consolidated pipeline is
  currently the only PR CodeQL path.

### 3.2 ci-pipeline.yml Decommission Strategy

Decision:

- Option A (required): delete ci-pipeline.yml to fully end the
  consolidated pipeline and avoid duplicate PR checks.

### 3.3 Image Availability Strategy (Critical Challenge)

Independent PR workflows cannot rely on a shared image from another
workflow unless using artifacts or a registry. The user wants to avoid
pipeline complexity.

Required behavior for each integration workflow:

- Restore the "Build Docker image (Local)" step in each integration
  workflow, reverting any artifact handover dependency.
- Build a local Docker image within the workflow before tests run.
- Tag the image as charon:local for consistency with existing scripts.
- Avoid external registry dependency for PR builds.

Impacted workflows:

- cerberus-integration.yml
- crowdsec-integration.yml
- waf-integration.yml
- rate-limit-integration.yml

E2E workflows:

- e2e-tests-split.yml already supports building an image locally when
  invoked directly. Ensure pull_request triggers route through this path
  (not workflow_call).

### 3.4 Pull Request Trigger Scope and Path Filters

- Use branch filters consistent with prior backups and docker-build.yml
  usage: main, development, feature/**, hotfix/**.
- Apply path filters for E2E to avoid unnecessary runs:
  frontend/**, backend/**, tests/**, playwright.config.js,
  .github/workflows/e2e-tests-split.yml.
- Integration workflows typically run on any backend/frontend changes.
  Consider adding path filters if desired, but default to full PR runs
  for parity with previous behavior.

### 3.5 Dependency and Concurrency Rules

- Remove workflow_run coupling to docker-build.yml for integration and
  E2E workflows. Each workflow should be independently triggered by
  pull_request.
- Keep job-level concurrency where it prevents duplicate runs on the
  same PR, but avoid cross-workflow dependencies.

## 4. Implementation Plan

### Phase 1: Baseline Verification (Tests)

- Confirm current CI behavior for PRs: identify which checks are now
  only running via ci-pipeline.yml.
- Capture baseline PR check set from GitHub Actions UI for comparison
  after restoration.

### Phase 2: Restore PR Triggers (Core Workflows)

- Add pull_request triggers to docker-build.yml with branches including
  main and development.
- Add pull_request triggers to cerberus-integration.yml,
  crowdsec-integration.yml, waf-integration.yml, and
  rate-limit-integration.yml.
- Add pull_request triggers to e2e-tests-split.yml, using the backup
  trigger block as the source of truth.

### Phase 3: Make Integration Workflows Self-Contained

- Restore the "Build Docker image (Local)" step in each integration
  workflow and remove dependency on ci-pipeline.yml artifacts.
- Remove registry pull steps or make them optional for manual runs.
- Ensure test scripts continue to reference charon:local.

### Phase 4: Security Workflow Triggers

- Add pull_request triggers to security-pr.yml and supply-chain-pr.yml
  as a mandatory requirement to preserve PR security coverage.
- Decide on CodeQL: either add pull_request to codeql.yml or create a
  dedicated PR CodeQL workflow. If the pipeline is deleted, CodeQL must
  have an alternative PR trigger.

### Phase 5: Decommission ci-pipeline.yml

- Delete ci-pipeline.yml.

### Phase 6: Validation and Audit

- Verify that PRs show the restored individual checks instead of a
  single pipeline job.
- Confirm each integration workflow completes without relying on
  registry or artifact inputs and includes the restored local build step.
- Validate E2E workflow runs directly on pull_request with build job
  executed locally.
- Confirm security workflows run on pull_request.

## 5. Acceptance Criteria (EARS)

- WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger
  docker-build.yml directly on pull_request for main and development.
- WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger
  cerberus-integration.yml, crowdsec-integration.yml, waf-integration.yml,
  and rate-limit-integration.yml on pull_request.
- WHEN an integration workflow runs on pull_request, THE SYSTEM SHALL
  restore and run the "Build Docker image (Local)" step, build a local
  Docker image, and tag it as charon:local before tests.
- WHEN a pull_request is opened or updated, THE SYSTEM SHALL trigger
  e2e-tests-split.yml directly on pull_request without relying on
  ci-pipeline.yml.
- WHEN the consolidated pipeline is retired, THE SYSTEM SHALL NOT run
  ci-pipeline.yml on pull_request.
- WHEN a pull_request is opened or updated, THE SYSTEM SHALL run
  security-pr.yml and supply-chain-pr.yml on pull_request.
- WHEN CodeQL is required for pull_request, THE SYSTEM SHALL run a
  CodeQL workflow on pull_request independent of ci-pipeline.yml.

## 6. Risks and Mitigations

- Risk: PR checks increase in parallel count and runtime.
  Mitigation: use path filters for E2E and consider optional filters
  for integration workflows.
- Risk: Image build duplication increases CI cost.
  Mitigation: keep builds scoped to workflows that need the image, and
  avoid registry pushes for PR builds.
- Risk: Security scans or CodeQL no longer run on PR if triggers are
  not restored.
  Mitigation: explicitly re-enable PR triggers in security workflows
  or add a dedicated PR security workflow.

## 7. Confidence Score

Confidence: 82 percent

Rationale: The workflow inventory and trigger gaps are clear. The main
uncertainty is selecting the final CodeQL and security trigger model
once ci-pipeline.yml is removed.