akanealw/Charon
1
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity

fix(ci): add e2e-tests.yml to push event path filters for workflow triggers

Browse Source
This commit is contained in:
GitHub Actions
2026-01-27 05:23:49 +00:00
parent 436b5f0817
commit cbd9612af5
3 changed files with 570 additions and 80 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/current_spec.md
+115 -80
View File
@@ -1,103 +1,138 @@
# Re-enable Security Playwright Tests and Run Full E2E (feature/beta-release)
# GitHub Actions E2E Trigger Investigation Plan (PR #550)
**Goal**: Turn security Playwright tests back on, run the full E2E suite (including security flows) on Docker base URL, and prepare triage steps for any failures.
**Status**: 🔴 ACTIVE – Planning
**Priority**: 🔴 CRITICAL – CI/CD gating
**Created**: 2026-01-27
**Context**
- Repository: Wikid82/Charon
- Default branch: `main`
- Active PR: #550 chore(docker): migrate from Alpine to Debian Trixie base image
- Working branch: `feature/beta-release`
- Symptom: After pushing an update to re-enable some E2E tests, the expected workflow did not trigger.
---
## Phase 0 – Context Validation (30 min)
- Confirm PR #550 source (fork vs upstream) and actor.
- Identify which E2E workflow should have run (list specific file/job after discovery in Phase 1 Task 1).
- Verify that a push occurred to `feature/beta-release` after re-enabling tests.
- Document expected trigger event vs actual run in Actions history.
## 🎯 Scope and Constraints
- Target branch: `feature/beta-release`.
- Base URL: Docker stack (`http://localhost:8080`) unless security tests require override.
- Keep management-mode rule: no code reading here; instructions only for execution subagents.
- Coverage: run E2E coverage only if already supported via Vite flow; otherwise note as optional follow-up.
Create Decision Record:
- Expected workflow: <file>/<job>
- Expected trigger(s): push/pull_request synchronize
- Observation time window: <timestamps>
---
**Objectives (EARS Requirements)**
- THE SYSTEM SHALL automatically run E2E workflows on eligible events for `feature/**`, `main`, and relevant branches.
- WHEN a commit is pushed to `feature/beta-release`, THE SYSTEM SHALL evaluate workflow `on:` triggers and filters and start corresponding jobs if conditions match.
- WHEN a pull request is updated (synchronize) for PR #550, THE SYSTEM SHALL trigger CI for all workflows configured for `pull_request` to the target branch.
- IF branch/path/actor conditions prevent a run, THEN THE SYSTEM SHALL allow a manual `workflow_dispatch` as a fallback.
## 🗂️ Files to Change (for execution agents)
- [playwright.config.js](playwright.config.js): re-enable security project/shard config, ensure `testDir` includes security specs, and restore any `grep`/`grepInvert` filters previously disabling them.
- Tests security fixtures/utilities: [tests/security/**](tests/security/), [tests/fixtures/security/**](tests/fixtures/security/), and any shared helpers under [tests/utils](tests/utils) that were toggled off (e.g., skip blocks, `test.skip`, env flags).
- Workflows/toggles: [ .github/workflows/*e2e*.yml](.github/workflows) and Docker compose overrides (e.g., [.docker/compose/docker-compose.e2e.yml](.docker/compose/docker-compose.e2e.yml)) to re-enable env vars/secrets for security tests (ACL/emergency/rate-limit toggles, tokens, base URLs).
- Global setup/teardown: [tests/global-setup.ts](tests/global-setup.ts) and related teardown to ensure security setup hooks are active (if previously short-circuited).
- Playwright reports/ignore lists: verify any `.gitignore` or report pruning that might suppress security artifacts.
**Hypotheses to Validate**
1. Path filters exclude the recent changes (e.g., only watching `frontend/**`, `backend/**`, `tests/**`, `playwright.config.js`, or `.github/workflows/**`).
2. Branch filters do not include `feature/**` or the YAML pattern is mis-specified.
3. PR is from a fork; secrets and permissions prevent jobs from running.
4. Skip conditions (`if:` gates) block runs for specific commit messages (e.g., `chore:`) or bots.
5. Concurrency cancellation due to rapid successive pushes suppresses earlier runs (`concurrency` with `cancel-in-progress`).
6. Workflows only run on `workflow_dispatch` or specific events, not `push`/`pull_request`.
---
**Design: Trigger Validation Approach**
- Inspect E2E-related workflows in `.github/workflows/` (e.g., `e2e-tests.yml`, `playwright-e2e.yml`, `docker-build.yml`).
- Enumerate `on:` events: `push`, `pull_request`, `pull_request_target`, `workflow_run`, `workflow_dispatch`.
- Capture `branches`, `branches-ignore`, `paths`, `paths-ignore`, `tags` filters; confirm YAML quoting and glob correctness.
- Review top-level `permissions:` and job-level `if:` conditions; note actor-based skips.
- Confirm matrix/include conditions for E2E jobs (e.g., only run when Playwright-related files change).
- Check Actions history for PR #550 and branch `feature/beta-release` to correlate event delivery vs filter gating.
## 🛠️ Implementation Steps
0) **Prepare environment and secrets**
- Ensure required secrets/vars are present (redact in logs): `CHARON_EMERGENCY_TOKEN`, `CHARON_ADMIN_USERNAME`/`CHARON_ADMIN_PASSWORD`, `PLAYWRIGHT_BASE_URL` (`http://localhost:8080` for Docker runs), feature toggles for security/ACL/rate-limit (e.g., `CHARON_SECURITY_TESTS_ENABLED`).
- Source from GitHub Actions secrets for CI; `.env`/`.env.local` for local. Do not hardcode; validate presence before run. Redact values in logs (print presence only).
## Phase 1 – Diagnosis (Targeted Checks)
1) **Restore security test inclusion**
- Revert skips/filters: remove `test.skip`, `test.describe.skip`, or project-level `grepInvert` that excluded security specs.
- Ensure `projects` in `playwright.config.js` include security shard (or merge back into main matrix) with correct `testDir`/`testMatch`.
- Re-enable security fixture initialization in `global-setup.ts` (e.g., emergency server bootstrap, token wiring) if it was bypassed.
### Task 1: Audit Workflow Triggers (DevOps)
Commands:
- List candidate workflows:
- `find .github/workflows -name '*e2e*' -o -name '*playwright*' -o -name '*test*' | sort`
- Extract triggers and filters:
- `grep -nA10 '^on:' <workflow.yml>`
- `grep -nE 'branches|paths|concurrency|permissions|if:' <workflow.yml>`
Output:
- Table: [Workflow | Triggers | Branches | Paths | if-conditions | Concurrency]
2) **Re-enable env toggles and secrets**
- In E2E workflow and Docker compose for tests, set required env vars (examples: `CHARON_EMERGENCY_SERVER_ENABLED=true`, `CHARON_SECURITY_TESTS_ENABLED=true`, tokens/ports 2019/2020) and confirm mounted secrets for security endpoints.
- Verify base URL resolution matches Docker (avoid Vite unless running coverage skill).
### Task 2: Retrieve Recent Runs (DevOps)
Commands:
- `gh run list --repo Wikid82/Charon --limit 20 --status all`
- `gh run view <run_id> --repo Wikid82/Charon`
- Correlate cancellations and `concurrency` group IDs.
3) **Bring up/refresh test stack**
- Start or rebuild test stack before running Playwright: use task `Docker: Start Local Environment` (or `Docker: Rebuild E2E Environment` if needed).
- Health check: verify ports 8080/2019/2020 respond (`curl http://localhost:8080`, `http://localhost:2019/config`, `http://localhost:2020/health`).
### Task 3: Verify PR Origin & Permissions (DevOps)
Commands:
- `gh pr view 550 --repo Wikid82/Charon --json isCrossRepository,author,headRefName,baseRefName`
Interpretation:
- If `isCrossRepository=true`, factor `pull_request_target` and secret restrictions.
4) **Run full E2E suite (all browsers + security)**
- Preferred tasks (from workspace tasks):
- `Test: E2E Playwright (All Browsers)` for breadth.
- `Test: E2E Playwright (Chromium)` for faster iteration.
- `Test: E2E Playwright (Skill)` if automation wrapper required.
- If security suite has its own task (e.g., `Test: E2E Playwright (Chromium) - Cerberus: Security Dashboard/Rate Limiting`), run those explicitly after re-enable.
### Task 4: Inspect Commit Messages & Actor Filters (DevOps)
Commands:
- `git log --oneline -n 5`
- Check workflow `if:` conditions referencing `github.actor`, commit message patterns.
5) **Optional coverage pass (only if Vite path)**
- Coverage only meaningful via Vite coverage skill (port 5173). Docker/8080 runs will show 0% coverage—do not treat as failure.
- If required: run `.github/skills/scripts/skill-runner.sh test-e2e-playwright-coverage`; target non-zero coverage and patch coverage on changed lines.
**Success Criteria (Phase 1):**
- Root cause identified (±1 hypothesis), reproducible via targeted test.
6) **Report collection and review**
- Generate and open report: `npx playwright show-report` (or task `Test: E2E Playwright - View Report`).
- For failures, gather traces/videos from `playwright-report/` and `test-results/`.
## Phase 1.5 – Hypothesis Elimination (1 hour)
Targeted tests per hypothesis:
1. Path filter: Commit `tests/.keep`; confirm if E2E triggers.
2. Branch filter: Push to `feature/test-trigger` (wildcard); observe triggers.
3. Fork PR: Confirm with `gh pr view`; evaluate secret usage.
4. Commit message: Push with non-`chore:` message; observe.
5. Concurrency: Push two commits quickly; confirm cancellations & group.
7) **Targeted rerun loop for failures**
- For each failing spec: rerun with `npx playwright test --project=chromium --grep "<failing name>"` (and the corresponding security project if separate).
- After fixes, rerun full Chromium suite; then run all-browsers suite.
Deliverable:
- Ranked hypothesis list with evidence and logs.
6) **Triage loop**
- Classify failures: environment/setup vs. locator/data vs. backend errors.
- Log failing specs, error messages, and env snapshot (base URL, env flags) into triage doc or ticket.
## Phase 2 – Remediation (Proper Fix)
---
### Scenario A: Path Filter Mismatch
- Fix: Expand `paths:` to include re-enabled tests and configs.
- Acceptance: Workflow triggers on next push touching those paths.
## ✅ Validation Checklist (execution order)
- [ ] Lint/typecheck: run `Lint: Frontend`, `Lint: TypeScript Check`, `Lint: Frontend (Fix)` if needed.
- [ ] E2E full suite with security (Chromium): task `Test: E2E Playwright (Chromium)` plus security-specific tasks (Rate Limiting/Security Dashboard) once re-enabled.
- [ ] E2E all browsers: `Test: E2E Playwright (All Browsers)`.
- [ ] Coverage (if applicable): run coverage skill; verify non-zero coverage in `coverage/e2e/`.
- [ ] Security scans: `Security: Trivy Scan` and `Security: Go Vulnerability Check` (or CodeQL tasks if required).
- [ ] Reports reviewed: open Playwright HTML report, inspect traces/videos for any failing specs.
- [ ] Triage log captured: record failing spec IDs, errors, env snapshot (base URL, env flags) and artifact links in shared location (e.g., `test-results/triage.md` or ticket).
### Scenario B: Branch Filter Mismatch
- Fix: Add `'feature/**'` (quoted) to `branches:` for relevant events.
- Acceptance: Push to `feature/beta-release` triggers E2E.
---
### Scenario C: Fork PR Gating
- Fix: Use `pull_request_target` with least privileges OR require upstream branch for E2E.
- Acceptance: PR updates trigger E2E without secret leakage.
## 🧪 Triage Strategy for Expected Failures
- **Auth/boot failures**: Check `global-setup` logs, ensure emergency/ACL toggles and tokens present. Validate endpoints 2019/2020 reachable in Docker logs.
- **Locator/strict mode issues**: Use role-based locators and scope to rows/sections; prefer `getByRole` with accessible names. Add short `expect` retries over manual waits.
- **Timing/toast flakiness**: Switch to `await expect(locator).toHaveText(...)` with retries; avoid `waitForTimeout`. Ensure network idle or response awaited on submit.
- **Backend 4xx/5xx**: Capture response bodies via `page.waitForResponse` or Playwright traces; verify env flags not disabling required features.
- **Security endpoint mismatches**: Validate test data/fixtures match current API contract; update fixtures before rerunning.
- **Next steps after failures**: Document failing spec paths, error messages, and suspected root cause; rerun focused spec with `--project` and `--grep` once fixes applied.
### Scenario D: Skip Conditions
- Fix: Adjust `if:` to avoid skipping E2E for `chore:` messages; add `workflow_dispatch` fallback.
- Acceptance: E2E runs for typical commits; manual dispatch available.
---
### Scenario E: Concurrency Conflicts
- Fix: Separate concurrency groups or set `cancel-in-progress: false` for E2E.
- Acceptance: Earlier runs not cancelled improperly; stable execution.
## 📌 Commands for Executors
- Re-enable/verify config: `node -e "console.log(require('./playwright.config'))"` (sanity on projects).
- Run Chromium suite: task `Test: E2E Playwright (Chromium)`.
- Run all browsers: task `Test: E2E Playwright (All Browsers)`.
- Run security-focused tasks: `Test: E2E Playwright (Chromium) - Cerberus: Security Dashboard`, `... - Cerberus: Rate Limiting`.
- Show report: `npx playwright show-report` or task `Test: E2E Playwright - View Report`.
- Coverage (optional): `.github/skills/scripts/skill-runner.sh test-e2e-playwright-coverage`.
Implementation Notes:
- Apply YAML edits in the respective workflow files; validate via `workflow_dispatch` and a watched-path commit.
---
## Phase 3 – Validation & Hardening
- Add/verify `workflow_dispatch` inputs for manual E2E runs.
- Push minimal commit touching guaranteed watched path.
- Document test in `docs/testing/`; update `README.md` CI notes.
- Regression test: Trigger from different branch/actor/event to confirm persistence.
## 📎 Notes
- Keep documentation of any env/secret re-introduction minimal and redacted; avoid hardcoding secrets.
- If security tests require data resets, ensure teardown does not affect subsequent suites.
**Related Config Checks**
- `codecov.yml`: Verify statuses and paths do not block CI.
- `.dockerignore` / `.gitignore`: Ensure test assets are included in context.
- `Dockerfile`: No gating on branch/commit via args.
- `playwright.config.js`: E2E matrix does not restrict by branch erroneously.
**Risks & Fallbacks**
- Increased CI load from wider `paths:` → keep essential paths only.
- Security concerns with `pull_request_target` → restrict permissions, avoid untrusted code execution.
- Fallbacks: Manual `workflow_dispatch`, dedicated E2E workflow with wide triggers, `repository_dispatch` testing.
**Task Owners**
- DevOps: Workflow trigger analysis and fixes
- QA_Security: Validate runs, review permissions and secret usage
- Frontend/Backend: Provide file-change guidance to exercise triggers
**Timeline & Escalation**
- Phase 1: 2 hours; Phase 2: 4 hours; Phase 3: 2 hours.
- If root cause not found by Phase 1.5, escalate with action log to GitHub Support.
**Next Steps**
- Request approval to begin Phase 1 execution per this plan.