diff --git a/docs/reports/qa_report.md b/docs/reports/qa_report.md index 12e8cb41..8ecd2da3 100644 --- a/docs/reports/qa_report.md +++ b/docs/reports/qa_report.md @@ -1,545 +1,132 @@ -## QA/Security Audit — PR-1 Backend Slice (Notify HTTP Wrapper) +# QA/Security Audit Report: `security-pr.yml` Workflow Fix -- Date: 2026-02-23 -- Scope: Current PR-1 backend slice implementation (notification provider handler/service, wrapper path, security gating) -- Verdict: **READY (PASS WITH NON-BLOCKING WARNINGS)** +- Date: 2026-02-27 +- Auditor: QA Security mode +- Scope: `.github/workflows/security-pr.yml` behavior fix only +- Overall verdict: **PASS (scope-specific)** with one **out-of-scope repository security debt** noted -## Commands Run +## Findings (Ordered by Severity) -1. `git rev-parse --abbrev-ref HEAD && git rev-parse --abbrev-ref --symbolic-full-name @{u} && git diff --name-only origin/main...HEAD` -2. `./.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -3. `PLAYWRIGHT_BASE_URL=http://localhost:8080 npx playwright test tests/settings/notifications.spec.ts` -4. `bash scripts/local-patch-report.sh` -5. `bash scripts/go-test-coverage.sh` -6. `pre-commit run --all-files` -7. `./.github/skills/scripts/skill-runner.sh security-scan-trivy` -8. `./.github/skills/scripts/skill-runner.sh security-scan-docker-image` -9. `bash scripts/pre-commit-hooks/codeql-go-scan.sh` -10. `bash scripts/pre-commit-hooks/codeql-js-scan.sh` -11. `bash scripts/pre-commit-hooks/codeql-check-findings.sh` -12. `./scripts/scan-gorm-security.sh --check` +### 🟡 IMPORTANT: Repository secret-scan debt exists (not introduced by scoped workflow change) +- Check: `pre-commit run --hook-stage manual gitleaks-tuned-scan --all-files` +- Result: **FAIL** (`135` findings) +- Scope impact: `touches_security_pr = 0` (no findings in `.github/workflows/security-pr.yml`) +- Evidence source: `test-results/security/gitleaks-tuned-precommit.json` +- Why this matters: Existing credential-like content raises background security risk even if unrelated to this workflow fix. +- Recommended remediation: + 1. Triage findings by rule/file and classify true positives vs allowed test fixtures. + 2. Add justified allowlist entries for confirmed false positives. + 3. Remove or rotate any real secrets immediately. + 4. Re-run `gitleaks-tuned-scan` until clean/accepted baseline is documented. -## Gate Results +### ✅ No blocking defects found in the implemented workflow fix +- Deterministic event handling: validated in workflow logic. +- Artifact/image resolution hardening: validated in workflow logic. +- Security hardening: validated in workflow logic and lint gates. -| Gate | Status | Evidence | -| --- | --- | --- | -| 1) Playwright E2E first | PASS | Notifications feature suite passed: **79/79** on local E2E environment. | -| 2) Local patch coverage preflight | PASS (WARN) | Artifacts generated: `test-results/local-patch-report.md` and `test-results/local-patch-report.json`; mode=`warn` due missing `frontend/coverage/lcov.info`. | -| 3) Backend coverage + threshold | PASS | `scripts/go-test-coverage.sh` reported **87.7% line** / **87.4% statement**; threshold 85% met. | -| 4) `pre-commit --all-files` | PASS | All configured hooks passed. | -| 5a) Trivy filesystem scan | PASS | No CRITICAL/HIGH/MEDIUM findings reported by skill at configured scanners/severities. | -| 5b) Docker image security scan | PASS | No CRITICAL/HIGH; Grype summary from `grype-results.json`: **Medium=10, Low=4**. | -| 5c) CodeQL Go + JS CI-aligned + findings check | PASS | Go and JS scans completed; findings check reported no security issues in both languages. | -| 6) GORM scanner (`--check`) | PASS | 0 CRITICAL/HIGH/MEDIUM; 2 INFO suggestions only. | +## Requested Validations -## Blockers / Notes - -- **No merge-blocking security or QA failures** were found for this PR-1 backend slice. -- Non-blocking operational notes: - - E2E initially failed until stale conflicting container was removed and E2E environment was rebuilt. - - `scripts/local-patch-report.sh` completed artifact generation in warning mode because frontend coverage input was absent. - - `pre-commit run codeql-check-findings --all-files` hook id was not registered in this local setup; direct script execution (`scripts/pre-commit-hooks/codeql-check-findings.sh`) passed. - -## Recommendation - -- **Proceed to PR-2**. -- Carry forward two non-blocking follow-ups: - 1. Ensure frontend coverage artifact generation before local patch preflight to eliminate warning mode. - 2. Optionally align local pre-commit hook IDs with documented CodeQL findings check command. - -## QA Report — PR-2 Security Patch Posture Audit - -- Date: 2026-02-23 -- Scope: PR-2 only (security patch posture, admin API hardening, rollback viability) -- Verdict: **READY (PASS)** - -## Gate Summary - -| Gate | Status | Evidence | -| --- | --- | --- | -| Targeted E2E for PR-2 | PASS | Security settings test for Caddy Admin API URL passed (2/2). | -| Local patch preflight artifacts | PASS | `test-results/local-patch-report.md` and `.json` regenerated. | -| Coverage and type-check | PASS | Backend coverage 87.7% line / 87.4% statement; frontend type-check passed; frontend coverage preflight input passed (88.99% lines). | -| Pre-commit gate | PASS | `pre-commit run --all-files` passed after resolving version and type-check hook issues. | -| Security scans | PASS | CodeQL Go/JS CI-aligned scans passed; findings gate passed with no HIGH/CRITICAL; Trivy passed at configured severities. | -| Runtime posture + rollback | PASS | Default scenario shifted `A -> B` for PR-2 posture; rollback remains explicit via `CADDY_PATCH_SCENARIO=A`; admin API URL now validated and normalized at config load. | - -## Resolved Items - -1. `check-version-match` mismatch fixed by syncing `.version` to `v0.19.1`. -2. `frontend-type-check` hook stabilized to `npx tsc --noEmit` for deterministic pre-commit behavior. - -## PR-2 Closure Statement - -All PR-2 QA/security gates required for merge are passing. No PR-3 scope is included in this report. - ---- - -## QA Report — PR-3 Keepalive Controls Closure - -- Date: 2026-02-23 -- Scope: PR-3 only (keepalive controls, safe fallback/default behavior, non-exposure constraints) -- Verdict: **READY (PASS)** - -## Reviewer Gate Summary (PR-3) - -| Gate | Status | Reviewer evidence | -| --- | --- | --- | -| Targeted E2E rerun | PASS | Security settings targeted rerun completed: **30 passed, 0 failed**. | -| Local patch preflight | PASS | `frontend/coverage/lcov.info` present; `scripts/local-patch-report.sh` artifacts regenerated with `pass` status. | -| Coverage + type-check | PASS | Frontend coverage gate passed (89% lines vs 85% minimum); type-check passed. | -| Pre-commit + security scans | PASS | `pre-commit --all-files`, CodeQL Go/JS CI-aligned scans, findings gate, and Trivy checks passed (no HIGH/CRITICAL blockers). | -| Final readiness | PASS | All PR-3 closure gates are green. | - -## Scope Guardrails Verified (PR-3) - -- Keepalive controls are limited to approved PR-3 scope. -- Safe fallback behavior remains intact when keepalive values are missing or invalid. -- Non-exposure constraints remain intact (`trusted_proxies_unix` and certificate lifecycle internals are not exposed). - -## Manual Verification Reference - -- PR-3 manual test tracking plan: `docs/issues/manual_test_pr3_keepalive_controls_closure.md` - -## PR-3 Closure Statement - -PR-3 is **ready to merge** with no open QA blockers. - ---- - -## QA/Security Audit — PR-2 Frontend Slice (Notifications) - -- Date: 2026-02-24 -- Scope: PR-2 frontend notifications slice only (UI/API contract alignment, tests, QA/security gates) -- Verdict: **READY (PASS WITH NON-BLOCKING WARNINGS)** - -## Commands Run - -1. `.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -2. `/projects/Charon/node_modules/.bin/playwright test /projects/Charon/tests/settings/notifications.spec.ts --config=/projects/Charon/playwright.config.js --project=firefox` -3. `bash /projects/Charon/scripts/local-patch-report.sh` -4. `/projects/Charon/.github/skills/scripts/skill-runner.sh test-frontend-coverage` -5. `cd /projects/Charon/frontend && npm run type-check` -6. `cd /projects/Charon && pre-commit run --all-files` -7. VS Code task: `Security: CodeQL JS Scan (CI-Aligned) [~90s]` -8. VS Code task: `Security: CodeQL Go Scan (CI-Aligned) [~60s]` -9. `cd /projects/Charon && bash scripts/pre-commit-hooks/codeql-check-findings.sh` -10. `/projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-trivy` - -## Gate Results - -| Gate | Status | Evidence | -| --- | --- | --- | -| 1) Playwright E2E first (notifications-focused) | PASS | `tests/settings/notifications.spec.ts`: **27 passed, 0 failed** after PR-2-aligned expectation update. | -| 2) Local patch coverage preflight artifacts | PASS (WARN) | Artifacts generated: `test-results/local-patch-report.md` and `test-results/local-patch-report.json`; report mode=`warn` with `changed_lines=0` for current baseline range. | -| 3) Frontend coverage + threshold | PASS | `test-frontend-coverage` skill completed successfully; coverage gate **PASS** at **89% lines** vs minimum **87%**. | -| 4) TypeScript check | PASS | `npm run type-check` completed with `tsc --noEmit` and no type errors. | -| 5) `pre-commit run --all-files` | PASS | All configured hooks passed, including frontend lint/type checks and fast Go linters. | -| 6a) CodeQL JS (CI-aligned) | PASS | JS scan completed and SARIF generated (`codeql-results-js.sarif`). | -| 6b) CodeQL Go (CI-aligned) | PASS | Go scan completed and SARIF generated (`codeql-results-go.sarif`). | -| 6c) CodeQL findings gate | PASS | `scripts/pre-commit-hooks/codeql-check-findings.sh` reported no security issues in Go/JS. | -| 6d) Trivy filesystem scan | PASS | `security-scan-trivy` completed with **0 vulnerabilities** and **0 secrets** at configured severities. | -| 6e) GORM scanner | SKIPPED (N/A) | Not required for PR-2 frontend-only slice (no `backend/internal/models/**` or GORM persistence scope changes). | - -## Low-Risk Fixes Applied During Audit - -1. Updated Playwright notifications spec to match PR-2 provider UX (`discord/gotify/webhook` selectable, not disabled): - - `tests/settings/notifications.spec.ts` -2. Updated legacy frontend API unit test expectations from Discord-only to supported provider contract: - - `frontend/src/api/__tests__/notifications.test.ts` - -## Blockers / Notes - -- **No merge-blocking QA/security blockers** for PR-2 frontend slice. -- Non-blocking notes: - - Local patch preflight is in `warn` mode with `changed_lines=0` against `origin/development...HEAD`; artifacts are present and valid. - - Local command execution is cwd-sensitive; absolute paths were used for reliable gate execution. - -## Recommendation - -- **Proceed to PR-3**. -- No blocking items remain for the PR-2 frontend slice. - ---- - -## Final QA/Security Audit — Notify Migration (PR-1/PR-2/PR-3) - -- Date: 2026-02-24 -- Scope: Final consolidated verification for completed notify migration slices (PR-1 backend, PR-2 frontend, PR-3 E2E/coverage hardening) -- Verdict: **ALL-PASS** - -## Mandatory Gate Sequence Results - -| Gate | Status | Evidence | -| --- | --- | --- | -| 1) Playwright E2E first (notifications-focused, including new payload suite) | PASS | `npx playwright test tests/settings/notifications.spec.ts tests/settings/notifications-payload.spec.ts --project=firefox --workers=1 --reporter=line` → **37 passed, 0 failed**. | -| 2) Local patch coverage preflight artifacts generation | PASS (WARN mode allowed) | `bash scripts/local-patch-report.sh` generated `test-results/local-patch-report.md` and `test-results/local-patch-report.json` with artifact verification. | -| 3) Backend coverage threshold check | PASS | `bash scripts/go-test-coverage.sh` → **Line coverage 87.4%**, minimum required **85%**. | -| 4) Frontend coverage threshold check | PASS | `bash scripts/frontend-test-coverage.sh` → **Lines 89%**, minimum required **85%** (coverage gate PASS). | -| 5) Frontend TypeScript check | PASS | `cd frontend && npm run type-check` completed with `tsc --noEmit` and no errors. | -| 6) `pre-commit run --all-files` | PASS | First run auto-fixed EOF in `tests/settings/notifications-payload.spec.ts`; rerun passed all hooks. | -| 7a) Trivy filesystem scan | PASS | `./.github/skills/scripts/skill-runner.sh security-scan-trivy` → no CRITICAL/HIGH/MEDIUM issues and no secrets detected. | -| 7b) Docker image scan | PASS | `./.github/skills/scripts/skill-runner.sh security-scan-docker-image` → **Critical 0 / High 0 / Medium 10 / Low 4**; gate policy passed (no critical/high). | -| 7c) CodeQL Go scan (CI-aligned) | PASS | CI-aligned Go scan completed; results written to `codeql-results-go.sarif`. | -| 7d) CodeQL JS scan (CI-aligned) | PASS | CI-aligned JS scan completed; results written to `codeql-results-js.sarif`. | -| 7e) CodeQL findings gate | PASS | `bash scripts/pre-commit-hooks/codeql-check-findings.sh` → no security issues in Go or JS findings gate. | -| 8) GORM security check mode (applicable) | PASS | `./scripts/scan-gorm-security.sh --check` → **0 CRITICAL / 0 HIGH / 0 MEDIUM**, INFO suggestions only. | - -## Final Verdict - -- all-pass / blockers: **ALL-PASS, no unresolved blockers** -- exact failing gates: **None (final reruns all passed)** -- proceed to handoff: **YES** - -## Notes - -- Transient issues were resolved during audit execution: - - Initial Playwright run saw container availability drop (`ECONNREFUSED`); after E2E environment rebuild and deterministic rerun, gate passed. - - Initial pre-commit run required one automatic EOF fix and passed on rerun. - - Shell working-directory drift caused temporary command-not-found noise for root-level security scripts; rerun from repo root passed. - ---- - -## Workflow Fix Validation — GHAS Trivy Compatibility (`docker-build.yml`) - -- Date: 2026-02-24 -- Scope: `.github/workflows/docker-build.yml` only +### 1) `actionlint` on security workflow +- Command: + - `pre-commit run actionlint --files .github/workflows/security-pr.yml` - Result: **PASS** +- Key output: + - `actionlint (GitHub Actions)..............................................Passed` -### Checks Run - -1. Workflow lint/syntax: - - `go run github.com/rhysd/actionlint/cmd/actionlint@latest .github/workflows/docker-build.yml` → `actionlint: OK` - - `python3` YAML parse (`yaml.safe_load`) for `.github/workflows/docker-build.yml` → `YAML parse: OK` -2. Guard/category placement validation: - - Verified Trivy compatibility uploads are gated with `if: always() && steps.trivy-pr-check.outputs.exists == 'true'`. - - Verified compatibility uploads are non-blocking via `continue-on-error: true`. - - Verified category aliases present: - - `.github/workflows/docker-build.yml:build-and-push` - - `.github/workflows/docker-publish.yml:build-and-push` - - `trivy-nightly` - - Verified main Trivy SARIF upload for non-PR path now explicitly sets category `.github/workflows/docker-build.yml:build-and-push`. -3. Security regression review (workflow logic only): - - Patch is additive for SARIF upload routing/compatibility and existence guard. - - No new secret exposure, token scope elevation, or privilege expansion introduced. - - No blocking behavior added to compatibility uploads. - -### Blockers - -- None. - -### Proceed Recommendation - -- **Proceed**. Workflow-only GHAS Trivy compatibility patch is validated and safe to merge. - ---- - -## QA Validation — E2E Auth Helper + Local Docker Socket Diagnostics - -- Date: 2026-02-24 -- Scope: Validation only for: - 1. E2E shard failures previously tied to missing `Authorization` header in test helpers (`createUser` path) - 2. Local Docker socket connection diagnostics/behavior -- Verdict: **PASS for both target tracks** (with unrelated shard test failures outside this scope) - -### Commands Executed - -1. `./.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -2. `pushd /projects/Charon >/dev/null && if [ -f .env ]; then set -a; . ./.env; set +a; fi && : "${CHARON_EMERGENCY_TOKEN:?CHARON_EMERGENCY_TOKEN is required (set it in /projects/Charon/.env)}" && CI=true PLAYWRIGHT_BASE_URL=http://127.0.0.1:8080 CHARON_SECURITY_TESTS_ENABLED=false PLAYWRIGHT_SKIP_SECURITY_DEPS=1 TEST_WORKER_INDEX=1 npx playwright test --project=firefox --shard=1/4 --output=playwright-output/firefox-shard-1 tests/core tests/dns-provider-crud.spec.ts tests/dns-provider-types.spec.ts tests/integration tests/manual-dns-provider.spec.ts tests/monitoring tests/settings tests/tasks` -3. `pushd /projects/Charon >/dev/null && if [ -f .env ]; then set -a; . ./.env; set +a; fi && : "${CHARON_EMERGENCY_TOKEN:?CHARON_EMERGENCY_TOKEN is required (set it in /projects/Charon/.env)}" && CI=true PLAYWRIGHT_BASE_URL=http://127.0.0.1:8080 CHARON_SECURITY_TESTS_ENABLED=false PLAYWRIGHT_SKIP_SECURITY_DEPS=1 npx playwright test --project=firefox tests/fixtures/api-helper-auth.spec.ts` -4. `pushd /projects/Charon/backend >/dev/null && go test -count=1 -v ./internal/services -run 'TestDockerService|TestIsDocker|TestResolveDockerHost|TestBuildLocalDockerUnavailableDetails|TestGetErrorResponseDetails' && go test -count=1 -v ./internal/api/handlers -run 'TestDockerHandler'` - -### Results - -| Check | Status | Output Summary | -| --- | --- | --- | -| E2E environment rebuild | PASS | `charon-e2e` rebuilt and healthy; health endpoint responsive. | -| CI-style non-security shard | PARTIAL (out-of-scope failures) | `124 passed`, `3 failed` in `tests/core/data-consistency.spec.ts` and `tests/core/domain-dns-management.spec.ts`; **no** `Failed to create user: {"error":"Authorization header required"}` observed. | -| Focused `createUser` auth-path spec | PASS | `tests/fixtures/api-helper-auth.spec.ts` → `2 passed (4.5s)`. | -| Backend docker service/handler tests | PASS | Targeted suites passed, including local diagnostics and mapping: `ok .../internal/services`, `ok .../internal/api/handlers`. | - ---- - -## Final QA/Security Gates Delta — Blocker Remediation Validation - -- Date: 2026-02-25 -- Scope: Current branch state after latest blocker remediations -- Verdict: **FAIL (single blocking gate remains)** - -### Exact Commands Run - -1. `.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -2. `.github/skills/scripts/skill-runner.sh test-e2e-playwright --project=firefox --grep="auth-api-enforcement|auth-middleware-cascade|authorization-rbac"` -3. `.github/skills/scripts/skill-runner.sh test-e2e-playwright --project=firefox --grep="Security Enforcement API|Auth Middleware Cascade|Cerberus ACL Role-Based Access Control"` -4. `bash scripts/local-patch-report.sh` (first attempt) -5. `go test ./internal/api/routes -run 'TestRegister_StateChangingRoutesDenyByDefaultWithExplicitAllowlist|TestRegister_StateChangingRoutesRequireAuthentication' -count=1` -6. `go test ./internal/api/handlers -run 'TestUserHandler_Setup_OneWayInvariant_ReentryRejectedAndSingleUser|TestUserHandler_Setup_ConcurrentAttemptInvariant|TestUserHandler_Setup_ResponseSecretEchoContract|TestUserHandler_GetProfile_SecretEchoContract|TestUserHandler_ListUsers_SecretEchoContract' -count=1` -7. `bash /projects/Charon/scripts/go-test-coverage.sh` -8. `bash /projects/Charon/scripts/frontend-test-coverage.sh` -9. `bash /projects/Charon/scripts/local-patch-report.sh` (rerun with coverage inputs present) -10. `bash /projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-codeql go summary` -11. `bash /projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-codeql javascript summary` -12. `pre-commit run --hook-stage manual codeql-check-findings --all-files` -13. `pre-commit run --all-files` (first run) -14. `bash /projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-trivy vuln,secret,misconfig json` -15. `bash /projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-docker-image charon:local` -16. `pre-commit run --all-files` (rerun) - -### Gate Results - -| Gate | Status | Evidence | -| --- | --- | --- | -| 1) E2E first (Playwright skill/task path) | PASS | E2E environment rebuilt and Playwright skill run completed with `7 passed` on Firefox. | -| 2) Local patch coverage preflight | PASS (WARN) | First run failed due missing `frontend/coverage/lcov.info`; after coverage generation, rerun produced required artifacts and warn-mode report. | -| 3) Focused backend regressions | PASS | Routes suite: `ok .../internal/api/routes`; handlers suite: `ok .../internal/api/handlers`. | -| 4) Coverage gates | PASS | Backend: statement `87.0%`, line `87.2%` (min 87%). Frontend: lines `88.97%` (min 87%). | -| 5) CodeQL CI-aligned Go + JS + manual findings hook | PASS | Go: `0 errors`; JS: `0 errors`; manual findings hook passed with no blocking findings. | -| 6) `pre-commit run --all-files` | **FAIL (blocking)** | `actionlint` failed on `.github/workflows/codeql.yml` (ShellCheck `SC2016`). | -| 7) Trivy filesystem + image scan | PASS | Filesystem scan completed with no blocking issues; image scan reported Critical=0, High=0, Medium=10, Low=4 (non-blocking by policy). | - -### Blocker Classification - -- **Real code defect (blocking):** `actionlint` failure in `.github/workflows/codeql.yml` (`SC2016`, single-quoted expression handling in shell block). -- **Environment/tooling-only (non-code) observations:** - - VS Code task runner returned `Task started but no terminal was found` for configured tasks in this session. - - `runTests` tool did not discover Go tests for targeted file inputs. - - Initial local patch preflight required coverage artifacts to be generated before successful rerun. - -### Final Gate Decision - -- **DO NOT APPROVE / DO NOT MERGE YET** -- Reason: one unresolved blocking gate remains (`pre-commit --all-files` -> `actionlint` on `.github/workflows/codeql.yml`). - ---- - -## QA/Security Delta — Post-Hardening E2E Remediation Pass - -- Date: 2026-02-25 -- Scope: Post-hardening E2E remediation for authz restrictions, secret redaction behavior, setup/security guardrails, and settings endpoint protections. -- Final Status: **PASS FOR REMEDIATION SCOPE** (targeted hardening suites green; see non-scope blockers below). - -### Commands Run - -1. `.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -2. `.github/skills/scripts/skill-runner.sh test-e2e-playwright` -3. `PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/security tests/security-enforcement tests/settings --project=firefox` -4. `PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/security tests/security-enforcement tests/settings --project=firefox` (post-fix rerun) -5. `PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/settings/account-settings.spec.ts tests/settings/notifications-payload.spec.ts --project=firefox` -6. `bash scripts/local-patch-report.sh` -7. `.github/skills/scripts/skill-runner.sh test-backend-coverage` -8. `.github/skills/scripts/skill-runner.sh test-frontend-coverage` -9. `.github/skills/scripts/skill-runner.sh qa-precommit-all` -10. VS Code task: `Security: CodeQL Go Scan (CI-Aligned) [~60s]` -11. VS Code task: `Security: CodeQL JS Scan (CI-Aligned) [~90s]` -12. `pre-commit run --hook-stage manual codeql-go-scan --all-files` -13. `pre-commit run --hook-stage manual codeql-js-scan --all-files` -14. `pre-commit run --hook-stage manual codeql-check-findings --all-files` -15. `.github/skills/scripts/skill-runner.sh security-scan-trivy` -16. `.github/skills/scripts/skill-runner.sh security-scan-docker-image` - -### Gate Results - -| Gate | Status | Evidence | -| --- | --- | --- | -| E2E-first hardening verification | PASS (targeted) | Remediated files passed: `tests/settings/account-settings.spec.ts` and `tests/settings/notifications-payload.spec.ts` → **30/30 passed**. | -| Local patch preflight artifacts | PASS (WARN) | `test-results/local-patch-report.md` and `test-results/local-patch-report.json` generated; warning mode due patch coverage below configured threshold. | -| Backend coverage threshold | PASS | Coverage gate met (minimum **87%** required by local gate). | -| Frontend coverage threshold | PASS | Coverage summary: **Lines 88.92%**; gate PASS vs **87%** minimum. | -| Pre-commit all-files | PASS | `.github/skills/scripts/skill-runner.sh qa-precommit-all` passed all hooks. | -| CodeQL Go/JS + findings gate | PASS | Manual-stage scans executed and findings gate reports no security issues in Go/JS. | -| Trivy filesystem | PASS | `security-scan-trivy` completed with no reported issues at configured severities. | -| Docker image vulnerability gate | PASS | No blocking critical/high vulnerabilities; non-blocking medium/low remain tracked in generated artifacts. | -| GORM scanner | N/A | Not triggered: this remediation changed only E2E test files, not backend model/database scope. | - -### Remediation Notes - -1. Updated account settings E2E to reflect hardened API-key redaction behavior: - - Assert masked display and absence of copy action for API key. - - Assert regeneration success without expecting raw key disclosure. -2. Updated notifications payload E2E to reflect hardened endpoint protection and trusted-provider test dispatch model: - - Added authenticated headers where protected endpoints are exercised. - - Updated assertions to expect guardrail contract (`MISSING_PROVIDER_ID`) for untrusted direct dispatch payloads. - -### Non-Scope Blockers (Observed in Broader Rerun) - -- A broad `tests/settings` rerun still showed unrelated failures in: - - `tests/settings/notifications.spec.ts` (event persistence reload timeout) - - `tests/settings/smtp-settings.spec.ts` (reload timeout) - - `tests/settings/user-management.spec.ts` (pending invite/reinvite timing) -- These were not introduced by this remediation and were outside the hardening-failure set addressed here. - -### Recommendation - -- Continue with a separate stability pass for the remaining non-scope settings suite timeouts. -- For this post-hardening remediation objective, proceed with the current changes. - -### Local Docker API Path / Diagnostics Validation - -- Verified via backend tests that local-mode behavior and diagnostics are correct: - - Local host resolution includes unix socket preference path (`unix:///var/run/docker.sock`) in service tests. - - Connectivity classification passes for permission denied, missing socket, daemon connectivity, timeout, and syscall/network error paths. - - Handler mapping passes for docker-unavailable scenarios and returns actionable details with `503` path assertions. - -### Env-only vs Regression Classification - -- Track 1 (`createUser` Authorization helper path): **No regression detected**. - - Focused spec passes and representative shard no longer shows prior auth-header failure signature. -- Track 2 (local Docker socket diagnostics/behavior): **No regression detected**. - - Targeted backend tests pass across local unix socket and failure diagnostic scenarios. -- Remaining shard failures: **Out of scope for requested tracks** (not env bootstrap failures and not related to auth-helper/docker-socket fixes). - ---- - -## Fast Playwright No-HTML Triage (PR #754) - -- Date: 2026-02-25 -- Scope: Focused CI-like local rerun for previously failing no-HTML Playwright specs on Firefox and Chromium +### 2) `pre-commit run --all-files` +- Command: + - `pre-commit run --all-files` - Result: **PASS** +- Key output: + - YAML/shell/actionlint/dockerfile/go vet/golangci-lint/version/LFS/type-check/frontend lint hooks passed. -### Commands Used +### 3) Security scans/tasks relevant to workflow change (feasible locally) +- Executed: + 1. `pre-commit run --hook-stage manual codeql-parity-check --all-files` -> **PASS** + 2. `pre-commit run --hook-stage manual codeql-check-findings --all-files` -> **PASS** (no blocking HIGH/CRITICAL) + 3. `pre-commit run --hook-stage manual gitleaks-tuned-scan --all-files` -> **FAIL** (repo baseline debt; not in scoped file) +- Additional QA evidence: + - `bash scripts/local-patch-report.sh` -> artifacts generated: + - `test-results/local-patch-report.md` + - `test-results/local-patch-report.json` -1. `pushd /projects/Charon >/dev/null && if [ -f .env ]; then set -a; . ./.env; set +a; fi && export CHARON_EMERGENCY_TOKEN="${CHARON_EMERGENCY_TOKEN:-test-emergency-token-for-e2e-32chars}" && CI=true PLAYWRIGHT_BASE_URL=http://127.0.0.1:8080 CHARON_SECURITY_TESTS_ENABLED=false PLAYWRIGHT_SKIP_SECURITY_DEPS=1 npx playwright test --project=firefox tests/settings/no-html.spec.ts tests/settings/notifications-no-html.spec.ts tests/core/no-html-hardening.spec.ts tests/integration/no-html-regression.spec.ts` -2. `pushd /projects/Charon >/dev/null && if [ -f .env ]; then set -a; . ./.env; set +a; fi && export CHARON_EMERGENCY_TOKEN="${CHARON_EMERGENCY_TOKEN:-test-emergency-token-for-e2e-32chars}" && CI=true PLAYWRIGHT_BASE_URL=http://127.0.0.1:8080 CHARON_SECURITY_TESTS_ENABLED=false PLAYWRIGHT_SKIP_SECURITY_DEPS=1 npx playwright test --project=chromium tests/settings/no-html.spec.ts tests/settings/notifications-no-html.spec.ts tests/core/no-html-hardening.spec.ts tests/integration/no-html-regression.spec.ts` +## Workflow Behavior Verification -### Results +## A) Deterministic event handling +Validated in `.github/workflows/security-pr.yml`: +- Manual dispatch input is required and validated as digits-only: + - `.github/workflows/security-pr.yml:10` + - `.github/workflows/security-pr.yml:14` + - `.github/workflows/security-pr.yml:71` + - `.github/workflows/security-pr.yml:78` +- `workflow_run` path constrained to successful upstream PR runs: + - `.github/workflows/security-pr.yml:31` + - `.github/workflows/security-pr.yml:36` + - `.github/workflows/security-pr.yml:38` +- Explicit trust-boundary contract checks for upstream workflow name/event/repository: + - `.github/workflows/security-pr.yml:127` + - `.github/workflows/security-pr.yml:130` + - `.github/workflows/security-pr.yml:136` + - `.github/workflows/security-pr.yml:143` -| Browser | Status | Output Summary | -| --- | --- | --- | -| Firefox | PASS | **43 passed, 0 failed** | -| Chromium | PASS | **43 passed, 0 failed** | +Assessment: **PASS** for deterministic triggering and contract enforcement. -### Conclusion +## B) Artifact and image resolution hardening +Validated in `.github/workflows/security-pr.yml`: +- Artifact is mandatory in `workflow_run`/`workflow_dispatch` artifact path; failures are explicit (`api_error`/`not_found`): + - `.github/workflows/security-pr.yml:159` + - `.github/workflows/security-pr.yml:185` + - `.github/workflows/security-pr.yml:196` + - `.github/workflows/security-pr.yml:214` + - `.github/workflows/security-pr.yml:225` +- Docker image load hardened with: + - tar readability check + - `manifest.json` multi-tag parsing (`RepoTags[]`) + - fallback to `Loaded image ID` + - deterministic alias `charon:artifact` + - `.github/workflows/security-pr.yml:255` + - `.github/workflows/security-pr.yml:261` + - `.github/workflows/security-pr.yml:267` + - `.github/workflows/security-pr.yml:273` + - `.github/workflows/security-pr.yml:282` + - `.github/workflows/security-pr.yml:295` + - `.github/workflows/security-pr.yml:300` +- Extraction consumes resolved alias output rather than reconstructed tag: + - `.github/workflows/security-pr.yml:333` + - `.github/workflows/security-pr.yml:342` -All four previously failing specs are green locally when executed in CI-like environment settings. +Assessment: **PASS** for deterministic artifact/image selection and prior mismatch risk mitigation. ---- +## C) Security hardening +Validated in `.github/workflows/security-pr.yml`: +- Least-privilege job permissions: + - `.github/workflows/security-pr.yml:40` + - `.github/workflows/security-pr.yml:41` + - `.github/workflows/security-pr.yml:42` + - `.github/workflows/security-pr.yml:43` +- Pinned action SHAs maintained for checkout/download/upload/CodeQL SARIF upload/Trivy action usage: + - `.github/workflows/security-pr.yml:48` + - `.github/workflows/security-pr.yml:243` + - `.github/workflows/security-pr.yml:365` + - `.github/workflows/security-pr.yml:388` + - `.github/workflows/security-pr.yml:397` + - `.github/workflows/security-pr.yml:408` -## Deep Security Audit — Huntarr-Style Hardening (Charon) +Assessment: **PASS** for workflow-level security hardening within scope. -- Date: 2026-02-25 -- Scope: Full backend/API/runtime/CI posture against Huntarr-style failure modes and self-hosted hardening requirements -- Constraint honored: `docs/plans/current_spec.md` was not modified -- Verdict: **FAIL (P0 findings present)** +## DoD Mapping for Workflow-Only Change -### Executive Summary +Executed: +- `actionlint` scoped check: **Yes (PASS)** +- Full pre-commit: **Yes (PASS)** +- Workflow-relevant security manual checks (CodeQL parity/findings, gitleaks): **Yes (2 PASS, 1 FAIL out-of-scope debt)** +- Local patch report artifacts: **Yes (generated)** -Charon has strong baseline controls (JWT auth middleware, setup lockout, non-root container runtime, emergency token constant-time verification, and active CI security gates), but this audit found critical gaps in authorization boundaries and secret exposure behavior. The most severe risks are: (1) security-control mutation endpoints accessible to any authenticated user in multiple handlers, (2) import preview/status endpoints exposed without auth middleware and without admin checks, and (3) sensitive values returned in generic settings/profile/invite responses. One container-image vulnerability (HIGH) is also present in `usr/bin/caddy`. +N/A for this scope: +- Playwright E2E feature validation for app behavior: **N/A** (no app/runtime code changes) +- Backend/frontend unit coverage gates: **N/A** (no backend/frontend source modifications in audited fix) +- GORM check-mode gate: **N/A** (no model/database/GORM changes) +- Trivy app binary/image scan execution for changed runtime artifact: **N/A locally for this audit** (workflow logic audited; no image/runtime code delta in this fix) -### Commands Executed - -1. `shell: Security: CodeQL All (CI-Aligned)` -2. `shell: Security: CodeQL Go Scan (CI-Aligned) [~60s]` -3. `shell: Security: CodeQL JS Scan (CI-Aligned) [~90s]` -4. `python3` SARIF summary (`codeql-results-go.sarif`, `codeql-results-js.sarif`, `codeql-results-javascript.sarif`) -5. `pre-commit run codeql-check-findings --all-files` (hook not registered locally; see blockers) -6. `.github/skills/scripts/skill-runner.sh security-scan-trivy vuln,secret,misconfig json > trivy-report.json` (misconfig scanner panic; see blockers) -7. `docker run ... aquasec/trivy:latest fs --scanners vuln,secret ... --format json > vuln-results.json` -8. `docker run ... aquasec/trivy:latest image ... charon:local > trivy-image-report.json` -9. `./scripts/scan-gorm-security.sh --check` -10. `pre-commit run --all-files` - -### Gate Results - -| Gate | Status | Evidence | -| --- | --- | --- | -| CodeQL (Go + JS SARIF artifacts) | PASS | `codeql-results-go.sarif`, `codeql-results-js.sarif`, `codeql-results-javascript.sarif` all contained `0` results. | -| Trivy filesystem (actionable scope: vuln+secret) | PASS | `vuln-results.json` reported `0` CRITICAL/HIGH findings after excluding local caches. | -| Trivy image scan (`charon:local`) | **FAIL** | `trivy-image-report.json`: `1` HIGH vulnerability (`CVE-2026-25793`) in `usr/bin/caddy` (`github.com/slackhq/nebula v1.9.7`). | -| GORM security gate (`--check`) | PASS | `0` CRITICAL/HIGH/MEDIUM; `2` INFO only. | -| Pre-commit full gate | PASS | `pre-commit run --all-files` passed all configured hooks. | - -### Findings - -| ID | Severity | Category | CWE / OWASP | Evidence | Impact | Exploitability | Remediation | -| --- | --- | --- | --- | --- | --- | --- | --- | -| F-001 | **Critical** | Broken authorization on security mutation endpoints | CWE-862 / OWASP A01 | `backend/internal/api/routes/routes.go` exposes `/api/v1/security/config`, `/security/breakglass/generate`, `/security/decisions`, `/security/rulesets*` under authenticated routes; corresponding handlers in `backend/internal/api/handlers/security_handler.go` (`UpdateConfig`, `GenerateBreakGlass`, `CreateDecision`, `UpsertRuleSet`, `DeleteRuleSet`) do not enforce admin role. | Any authenticated non-admin can alter core security controls, generate break-glass token material, and tamper with decision/ruleset state. | High (single authenticated request path). | Enforce admin authorization at route-level or handler-level for all security-mutating endpoints; add deny-by-default middleware tests for all `/security/*` mutators. | -| F-002 | **High** | Unauthenticated import status/preview exposure | CWE-200 + CWE-306 / OWASP A01 + A04 | `backend/internal/api/routes/routes.go` registers import handlers via `RegisterImportHandler`; `backend/internal/api/routes/routes.go` `RegisterImportHandler()` mounts `/api/v1/import/*` without auth middleware. In `backend/internal/api/handlers/import_handler.go`, `GetStatus` and `GetPreview` lack `requireAdmin` checks and can return `caddyfile_content`. | Potential disclosure of infrastructure hostnames/routes/config snippets to unauthenticated users. | Medium-High (network-accessible management endpoint). | Move import routes into protected/admin group; require admin check in `GetStatus` and `GetPreview`; redact/remove raw `caddyfile_content` from API responses. | -| F-003 | **High** | Secret disclosure in API responses | CWE-200 / OWASP A02 + A01 | `backend/internal/api/handlers/settings_handler.go` `GetSettings()` returns full key/value map; `backend/internal/services/mail_service.go` persists `smtp_password` in settings. `backend/internal/api/handlers/user_handler.go` returns `api_key` in profile/regenerate responses and `invite_token` in invite/create/resend flows. | Secrets and account takeover tokens can leak through UI/API, logs, browser storage, and support channels. | Medium (requires authenticated access for some paths; invite token leak is high-risk in admin workflows). | Introduce server-side secret redaction policy: write-only secret fields, one-time reveal tokens, and masked settings API; remove raw invite/API key returns except explicit one-time secure exchange endpoints with re-auth. | -| F-004 | **Medium** | Dangerous operation controls incomplete | CWE-285 / OWASP A01 | High-impact admin operations (security toggles, user role/user deletion pathways) do not consistently require re-auth/step-up confirmation; audit exists in places but not uniformly enforced with confirmation challenge. | Increases blast radius of stolen session or accidental clicks for destructive operations. | Medium. | Add re-auth (password/TOTP) for dangerous operations and explicit confirmation tokens with short TTL; enforce audit record parity for every security mutation endpoint. | -| F-005 | **Medium** | Secure-by-default network exposure posture | CWE-1327 / OWASP A05 | `backend/cmd/api/main.go` starts HTTP server on `:` (all interfaces). Emergency server defaults are safer, but management API default bind remains broad in self-hosted deployments. | Expanded attack surface if deployment network controls are weak/misconfigured. | Medium (environment dependent). | Default management bind to loopback/private interface and require explicit opt-in for public exposure; document hardened reverse-proxy-only deployment mode. | -| F-006 | **Medium** | Container image dependency vulnerability | CWE-1104 / OWASP A06 | `trivy-image-report.json`: `HIGH CVE-2026-25793` in `usr/bin/caddy` (`github.com/slackhq/nebula v1.9.7`) in `charon:local`. | Potential exposure via vulnerable transitive component in runtime image. | Medium (depends on exploit preconditions). | Rebuild with patched Caddy base/version; pin and verify fixed digest; keep image scan as blocking CI gate for CRITICAL/HIGH. | - -### Setup-Mode Re-entry Assessment - -- **Pass**: `backend/internal/api/handlers/user_handler.go` blocks setup when user count is greater than zero (`Setup already completed`). -- Residual risk: concurrent first-run race conditions are still theoretically possible if multiple setup requests arrive before first transaction commits. - -### Charon Safety Contract (Current State) - -| Invariant | Status | Notes | -| --- | --- | --- | -| No state-changing endpoint without strict authz | **FAIL** | Security mutators and import preview/status gaps violate deny-by-default authorization expectations. | -| No raw secrets in API/logs/diagnostics | **FAIL** | Generic settings/profile/invite responses include sensitive values/tokens. | -| Secure-by-default management exposure | **PARTIAL** | Emergency server defaults safer; main API bind remains broad by default. | -| Dangerous operations require re-auth + audit | **PARTIAL** | Audit is present in parts; step-up re-auth/confirmation is inconsistent. | -| Setup mode is one-way lockout after initialization | **PASS** | Setup endpoint rejects execution when users already exist. | - -### Prioritized Remediation Plan - -**P0 (block release / immediate):** - -1. Enforce admin authz on all `/security/*` mutation endpoints (`UpdateConfig`, `GenerateBreakGlass`, `CreateDecision`, `UpsertRuleSet`, `DeleteRuleSet`, and any equivalent mutators). -2. Move all import endpoints behind authenticated admin middleware; add explicit admin checks to `GetStatus`/`GetPreview`. -3. Remove raw secret/token disclosure from settings/profile/invite APIs; implement write-only and masked read semantics. - -**P1 (next sprint):** - -1. Add step-up re-auth for dangerous operations (security toggles, user deletion/role changes, break-glass token generation). -2. Add explicit confirmation challenge for destructive actions with short-lived confirmation tokens. -3. Resolve image CVE by upgrading/pinning patched Caddy dependency and re-scan. - -**P2 (hardening backlog):** - -1. Tighten default bind posture for management API. -2. Add startup race protection for first-run setup path. -3. Expand documentation redaction standards for tokenized URLs and support artifacts. - -### CI Tripwires (Required Enhancements) - -1. **Route-auth crawler test (new):** enumerate all API routes and fail CI when any state-changing route (`POST/PUT/PATCH/DELETE`) is not protected by auth + role policy. -2. **Secret exposure contract tests:** assert sensitive keys (`smtp_password`, API keys, invite tokens, provider tokens) are never returned by generic read APIs. -3. **Security mutator RBAC tests:** negative tests for non-admin callers on all `/security/*` mutators. -4. **Image vulnerability gate:** fail build on CRITICAL/HIGH vulnerabilities unless explicit waiver with expiry exists. -5. **Trivy misconfig stability gate:** pin Trivy version or disable known-crashing parser path until upstream fix; keep scanner reliability monitored. - -### Blockers / Tooling Notes - -- `pre-commit run codeql-check-findings --all-files` failed locally because hook id is not registered in current pre-commit stage. -- Trivy `misconfig` scanner path crashed with a nil-pointer panic in Ansible parser during full filesystem scan; workaround used (`vuln,secret`) for actionable gate execution. - -### Final DoD / Security Gate Decision - -- **Overall Security Gate:** **FAIL** (due to unresolved P0 findings F-001/F-002/F-003 and one HIGH image vulnerability F-006). -- **If this code were Huntarr, would we call it safe now?** **No** — not until P0 authorization and secret-exposure issues are remediated and re-validated. - -### Remediation Update (2026-02-25) - -- Scope: P0 backend remediations from this audit were implemented in a single change set; `docs/plans/current_spec.md` remained untouched. - -**F-001 — Security mutator authorization:** - -- Added explicit admin checks in security mutator handlers (`UpdateConfig`, `GenerateBreakGlass`, `CreateDecision`, `UpsertRuleSet`, `DeleteRuleSet`, `ReloadGeoIP`, `LookupGeoIP`, `AddWAFExclusion`, `DeleteWAFExclusion`). -- Updated security route wiring so mutation endpoints are mounted under admin-protected route groups. -- Added/updated negative RBAC tests to verify non-admin callers receive `403` for security mutators. - -**F-002 — Import endpoint protection:** - -- Updated import route registration to require authenticated admin middleware for `/api/v1/import/*` endpoints. -- Added admin enforcement in `GetStatus` and `GetPreview` handlers. -- Added/updated route tests to verify unauthenticated and non-admin access is blocked. - -**F-003 — Secret/token exposure prevention:** - -- Updated settings read behavior to mask sensitive values and return metadata flags instead of raw secret values. -- Removed raw `api_key` and invite token disclosure from profile/regenerate/invite responses; responses now return masked/redacted values and metadata. -- Updated handler tests to enforce non-disclosure response contracts. - -**Validation executed for this remediation update:** - -- `go test ./internal/api/handlers -run 'SecurityHandler|ImportHandler|SettingsHandler|UserHandler'` ✅ -- `go test ./internal/api/routes` ✅ - -**Residual gate status after this remediation update:** - -- P0 backend findings F-001/F-002/F-003 are addressed in code and covered by updated tests. -- Image vulnerability finding F-006 remains open until runtime image dependency update and re-scan. +## Conclusion +The implemented fix in `.github/workflows/security-pr.yml` meets the requested goals for deterministic event handling, robust artifact/image resolution, and workflow security hardening. Required validation commands were executed and passed (`actionlint`, `pre-commit --all-files`), and additional feasible security checks were run. One repository-wide gitleaks debt remains and should be remediated separately from this workflow fix. diff --git a/tests/core/admin-onboarding.spec.ts b/tests/core/admin-onboarding.spec.ts index 840d536c..c9942c63 100644 --- a/tests/core/admin-onboarding.spec.ts +++ b/tests/core/admin-onboarding.spec.ts @@ -14,6 +14,31 @@ import { waitForAPIResponse, waitForLoadingComplete } from '../utils/wait-helper test.describe('Admin Onboarding & Setup', () => { const baseURL = process.env.PLAYWRIGHT_BASE_URL || 'http://127.0.0.1:8080'; + async function navigateToLoginDeterministic(page: Page): Promise { + const gotoLogin = async (timeout: number): Promise => { + await page.goto('/login', { waitUntil: 'domcontentloaded', timeout }); + await expect(page).toHaveURL(/\/login|\/signin|\/auth/i, { timeout: 5000 }); + }; + + try { + await gotoLogin(15000); + return; + } catch { + // Recover from stale route/session and retry with a short bounded navigation. + await page.goto('/', { waitUntil: 'domcontentloaded', timeout: 10000 }).catch(() => {}); + await page.context().clearCookies(); + try { + await page.evaluate(() => { + localStorage.clear(); + sessionStorage.clear(); + }); + } catch { + // Firefox can block storage access in some transitional states. + } + await gotoLogin(10000); + } + } + async function assertAuthenticatedTransition(page: Page): Promise { const loginEmailField = page.locator('input[type="email"], input[name="email"], input[autocomplete="email"], input[placeholder*="@"]').first(); @@ -58,19 +83,7 @@ test.describe('Admin Onboarding & Setup', () => { const shouldSkipLogin = /Admin logs in with valid credentials|Dashboard displays after login/i.test(testInfo.title); if (shouldSkipLogin) { - // Navigate to home first to avoid Firefox security restrictions on login page - await page.goto('/', { waitUntil: 'domcontentloaded' }); - // Clear auth state for the login test - await page.context().clearCookies(); - try { - await page.evaluate(() => { - localStorage.clear(); - sessionStorage.clear(); - }); - } catch { - // Firefox may block storage access on some pages - continue anyway - } - await page.goto('/login', { waitUntil: 'domcontentloaded' }); + await navigateToLoginDeterministic(page); return; } @@ -86,11 +99,11 @@ test.describe('Admin Onboarding & Setup', () => { const start = Date.now(); await test.step('Navigate to login page', async () => { - await page.goto('/login', { waitUntil: 'domcontentloaded' }); + await navigateToLoginDeterministic(page); if (!/\/login|\/signin|\/auth/i.test(page.url())) { await logoutUser(page).catch(() => {}); - await page.goto('/login', { waitUntil: 'domcontentloaded' }); + await navigateToLoginDeterministic(page); } const emailField = page.locator('input[type="email"], input[name="email"], input[autocomplete="email"], input[placeholder*="@"]'); @@ -124,7 +137,7 @@ test.describe('Admin Onboarding & Setup', () => { // Dashboard displays after login test('Dashboard displays after login', async ({ page, adminUser }) => { await test.step('Perform fresh login and confirm auth transition', async () => { - await page.goto('/login', { waitUntil: 'domcontentloaded' }); + await navigateToLoginDeterministic(page); await submitLoginAndWaitForDashboard(page, adminUser.email); diff --git a/tests/core/authentication.spec.ts b/tests/core/authentication.spec.ts index 3dbddffc..a241de2b 100644 --- a/tests/core/authentication.spec.ts +++ b/tests/core/authentication.spec.ts @@ -411,6 +411,25 @@ test.describe('Authentication Flows', () => { }); test.describe('Authentication Accessibility', () => { + async function pressTabUntilFocused(page: import('@playwright/test').Page, target: import('@playwright/test').Locator, maxTabs: number): Promise { + for (let i = 0; i < maxTabs; i++) { + await page.keyboard.press('Tab'); + const focused = await expect + .poll(async () => target.evaluate((el) => el === document.activeElement), { + timeout: 1500, + intervals: [100, 200, 300], + }) + .toBeTruthy() + .then(() => true) + .catch(() => false); + if (focused) { + return; + } + } + + await expect(target).toBeFocused(); + } + /** * Test: Login form is keyboard accessible */ @@ -427,16 +446,10 @@ test.describe('Authentication Flows', () => { await expect(emailInput).toBeFocused(); // Tab to password field - await page.keyboard.press('Tab'); - await expect(passwordInput).toBeFocused(); + await pressTabUntilFocused(page, passwordInput, 2); // Tab to submit button (may go through "Forgot Password" link first) - await page.keyboard.press('Tab'); - // If there's a "Forgot Password" link, tab again - if (!(await submitButton.evaluate((el) => el === document.activeElement))) { - await page.keyboard.press('Tab'); - } - await expect(submitButton).toBeFocused(); + await pressTabUntilFocused(page, submitButton, 3); }); }); diff --git a/tests/core/caddy-import/caddy-import-firefox.spec.ts b/tests/core/caddy-import/caddy-import-firefox.spec.ts index 47ab81a2..b1df798f 100644 --- a/tests/core/caddy-import/caddy-import-firefox.spec.ts +++ b/tests/core/caddy-import/caddy-import-firefox.spec.ts @@ -20,7 +20,7 @@ import { test, expect } from '../../fixtures/auth-fixtures'; import { Page } from '@playwright/test'; -import { ensureImportUiPreconditions } from './import-page-helpers'; +import { ensureImportUiPreconditions, resetImportSession, waitForSuccessfulImportResponse } from './import-page-helpers'; function firefoxOnly(browserName: string) { test.skip(browserName !== 'firefox', 'This suite only runs on Firefox'); @@ -103,6 +103,7 @@ test.describe('Caddy Import - Firefox-Specific @firefox-only', () => { await test.step('Navigate to import page', async () => { await setupImportMocks(page); + await resetImportSession(page); await ensureImportUiPreconditions(page, adminUser); }); @@ -115,25 +116,18 @@ test.describe('Caddy Import - Firefox-Specific @firefox-only', () => { await textarea.fill('test.example.com { reverse_proxy localhost:3000 }'); await expect(parseButton).toBeEnabled(); - // Verify button is clickable (not obscured by overlays) - const isClickable = await parseButton.evaluate((btn) => { - const rect = btn.getBoundingClientRect(); - const centerX = rect.left + rect.width / 2; - const centerY = rect.top + rect.height / 2; - const topElement = document.elementFromPoint(centerX, centerY); - return topElement === btn || btn.contains(topElement); - }); - expect(isClickable).toBeTruthy(); + // Firefox-safe actionability check without mutating state. + await parseButton.click({ trial: true }); }); await test.step('Verify click event fires in Firefox', async () => { - const requestPromise = page.waitForRequest((req) => req.url().includes('/api/v1/import/upload')); - const parseButton = page.getByRole('button', { name: /parse|review/i }); - await parseButton.click(); - - // Wait for request to be sent - const request = await requestPromise; + const response = await waitForSuccessfulImportResponse( + page, + () => parseButton.click(), + 'firefox-click-handler' + ); + const request = response.request(); expect(request.url()).toContain('/api/v1/import/upload'); expect(request.method()).toBe('POST'); }); @@ -322,13 +316,14 @@ test.describe('Caddy Import - Firefox-Specific @firefox-only', () => { test('should handle large Caddyfile upload (10KB+)', async ({ page, adminUser }) => { await test.step('Navigate to import page', async () => { await setupImportMocks(page); + await resetImportSession(page); await ensureImportUiPreconditions(page, adminUser); }); await test.step('Generate large Caddyfile content', async () => { - // Generate 100 host entries (~10KB+) + // Generate deterministic payload >10KB for all browsers/runtimes. let largeCaddyfile = ''; - for (let i = 0; i < 100; i++) { + for (let i = 0; i < 180; i++) { largeCaddyfile += ` host${i}.example.com { reverse_proxy backend${i}:${3000 + i} @@ -344,7 +339,7 @@ host${i}.example.com { // Verify no UI lag (textarea should update immediately) const value = await textarea.inputValue(); expect(value.length).toBeGreaterThan(10000); - expect(value).toContain('host99.example.com'); + expect(value).toContain('host179.example.com'); }); await test.step('Upload large file to API', async () => {