akanealw/Charon
1
0
Fork 0
Code Issues Pull Requests Actions 15 Packages Projects Releases Wiki Activity

chore: update QA report with detailed gate status and revalidation results

Browse Source
This commit is contained in:
GitHub Actions
2026-04-22 00:13:18 +00:00
parent 07108cfa8d
commit 6546130518
1 changed files with 95 additions and 136 deletions
Download Patch File Download Diff File Expand all files Collapse all files

231
docs/reports/qa_report.md
View File

@@ -3,170 +3,129 @@
Date: 2026-04-21
Repository: /projects/Charon
Branch: feature/beta-release
Scope assessed: backend/internal/services/docker_service.go (SA1019 fix) and mandatory DoD QA/Security gates
Scope assessed: DoD revalidation after recent fixes (E2E-first, frontend coverage, pre-commit/version gate, SA1019, Trivy CVE check)
## Final Recommendation
FAIL
Reason: DoD is blocked by failing mandatory gates (Playwright E2E first, frontend coverage, pre-commit version check).
Reason: Two mandatory gates are still failing in current rerun evidence:
- Playwright E2E-first gate
- Frontend coverage gate
Pre-commit/version-check is now passing.
## Gate Summary
| # | DoD Gate | Status | Notes |
|---|---|---|---|
| 1 | Playwright E2E first | FAIL | Mandatory e2e rebuild repeatedly failed (`docker build ... context canceled`); fallback Firefox run failed auth (`tests/auth.setup.ts` login 401) |
| 2 | GORM security scan (conditional) | PASS (proactive run) | Scope does not touch models/migrations; scanner still executed: 0 Critical, 0 High, 0 Medium, 2 Info |
| 3a | Backend coverage | PASS | `scripts/go-test-coverage.sh`: 91.6% statements, artifact `backend/coverage.txt` |
| 3b | Frontend coverage | FAIL | `scripts/frontend-test-coverage.sh`: tests pass but coverage run exits non-zero due `ENOENT frontend/coverage/.tmp/coverage-151.json` |
| 4 | Local patch coverage report | PASS | `scripts/local-patch-report.sh` exit 0; artifacts generated; patch report shows 0 changed lines / 100% |
| 5 | Frontend type check | PASS | `npm run type-check` (`tsc --noEmit`) passes |
| 6 | Pre-commit hooks | FAIL | `lefthook run pre-commit --all-files` fails at `check-version-match` (.version v0.21.0 vs latest tag v0.27.0) |
| 7a | Trivy filesystem scan | PASS | `security-scan-trivy`: 0 vulnerabilities, 0 secrets findings |
| 7b | Docker image scan | PASS | `security-scan-docker-image`: 0 Critical, 0 High, 4 Medium |
| 7c | CodeQL Go + JS | PASS | Go: 0 errors; JS: 0 errors; no blocking findings |
| 8 | Lint/build checks | PASS | `make lint-fast` 0 issues, `go build ./...` pass, `npm run build` pass, `go vet ./...` pass |
## Explicit SA1019 Verification
Status: RESOLVED
Evidence:
- `lefthook run pre-commit --all-files` output contains no `SA1019` occurrences.
- `golangci-lint-fast` passes with 0 issues.
- Root lint gate `make lint-fast` passes with 0 issues.
Conclusion: The deprecation warning SA1019 for the Issue #929 fix scope is no longer present in current lint/static analysis output.
| 1 | Playwright E2E first | FAIL | Healthy container path confirmed (`charon-e2e Up ... (healthy)`), auth setup passes, but accessibility suite still has 1 failing test (security headers page axe timeout) |
| 2 | Frontend coverage | FAIL | `scripts/frontend-test-coverage.sh` still ends with unhandled `ENOENT` on `frontend/coverage/.tmp/coverage-132.json` |
| 3 | Pre-commit hooks + version check | PASS | `lefthook run pre-commit --all-files` passes; `check-version-match` passes (`.version` matches latest tag `v0.27.0`) |
| 4 | SA1019 reconfirmation | PASS | `golangci-lint run ./... --enable-only staticcheck` reports `0 issues`; no `SA1019` occurrences |
| 5 | Trivy FS status (CVE-2026-34040) | PASS (not detected) | Current FS scan (`trivy fs --scanners vuln .`) exits 0 with no CVE hit; `CVE-2026-34040` not present in available Trivy artifacts |
## Detailed Evidence
### 1) Playwright E2E first
### 1) Playwright E2E-first gate (revalidated)
Execution path:
- Attempted mandatory rebuild: `/projects/Charon/.github/skills/scripts/skill-runner.sh docker-rebuild-e2e --clean --no-cache`
- Result: repeated Docker build cancellation (`failed to solve: Canceled: context canceled`)
- Fallback run against available container:
- `npx playwright test -c /projects/Charon/playwright.config.js --project=firefox`
- Then with explicit base URL: `PLAYWRIGHT_BASE_URL=http://127.0.0.1:8787 ...`
- Result: 1 failed, 722 not run
- Failing spec: `tests/auth.setup.ts` (`authenticate`), error `Login failed: 401 - {"error":"invalid credentials"}`
Gate disposition: FAIL (mandatory first gate not passing).
### 2) GORM scan (conditional)
Execution:
- `./scripts/scan-gorm-security.sh --check`
Result:
- Exit code 0
- Critical 0, High 0, Medium 0, Info 2
- Info items are index recommendations in `backend/internal/models/user.go` (non-blocking).
Gate disposition: PASS (proactively executed for audit completeness).
### 3) Coverage gates
Backend:
- Command: `bash scripts/go-test-coverage.sh`
- Result: PASS
- Coverage: 91.6%
- Artifact: `backend/coverage.txt`
Frontend:
- Command: `bash scripts/frontend-test-coverage.sh`
- Test execution summary: 147 files passed, 5 skipped; 2035 tests passed, 90 skipped
- Final result: FAIL due unhandled rejection / filesystem error:
- `ENOENT: no such file or directory, open '/projects/Charon/frontend/coverage/.tmp/coverage-151.json'`
- Command exit code: 1
- Artifact exists but gate is failing due non-zero exit.
Gate disposition: Backend PASS, Frontend FAIL.
### 4) Local patch coverage report
Execution:
- `bash scripts/local-patch-report.sh`
Result:
- Exit code 0
- Artifacts present:
- `test-results/local-patch-report.md`
- `test-results/local-patch-report.json`
- Patch summary: 0 changed lines, reported 100%
Gate disposition: PASS.
### 5) Frontend type check
Execution:
- `cd frontend && npm run type-check`
Result:
- PASS (`tsc --noEmit` exit 0)
### 6) Pre-commit hooks
Execution:
- `cd backend && lefthook run pre-commit --all-files`
Result:
- FAIL at `check-version-match`
- Failure detail: `.version` reports v0.21.0 while latest tag is v0.27.0
- Important: no SA1019 in output; fast lint/static checks are green
Execution evidence:
- Container health:
- `docker ps --filter name=charon-e2e --format '{{.Names}} {{.Status}}'`
- Output: `charon-e2e Up 35 minutes (healthy)`
- Auth setup:
- `PLAYWRIGHT_HTML_OPEN=never npx playwright test --project=firefox tests/auth.setup.ts -g "authenticate"`
- Result: `1 passed`
- Evidence: `Login successful`
- Accessibility rerun:
- `PLAYWRIGHT_HTML_OPEN=never npx playwright test --project=firefox -g "accessibility"`
- Result: `1 failed, 2 skipped, 64 passed`
- Failing test:
- `tests/a11y/security.a11y.spec.ts:21:5`
- `Accessibility: Security › security headers page has no critical a11y violations`
- Failure detail: `Test timeout of 90000ms exceeded` during axe analyze step.
Gate disposition: FAIL.
### 7) Security scans
Trivy filesystem:
- Command: `/projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-trivy`
- Summary table reports zero vulnerabilities and zero secrets findings for scanned manifests/text targets
- Result: PASS
Docker image scan:
- Command: `/projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-docker-image`
- Result: PASS
- Vulnerability summary: Critical 0, High 0, Medium 4, Low 0
CodeQL Go:
- Command: `/projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-codeql go summary`
- Result: PASS (0 errors, 0 warnings, 0 notes)
CodeQL JavaScript:
- Command: `/projects/Charon/.github/skills/scripts/skill-runner.sh security-scan-codeql javascript summary`
- Result: PASS (0 errors, 0 warnings, 0 notes)
Gotify token exposure review:
- No active tokenized URLs or secret tokens were found in generated scan outputs during this rerun.
### 8) Lint/build checks
### 2) Frontend coverage gate (revalidated)
Execution:
- `make lint-fast` (repo root) -> PASS (0 issues)
- `cd backend && go build ./...` -> PASS
- `cd frontend && npm run build` -> PASS
- `cd backend && go vet ./...` -> PASS
- `bash scripts/frontend-test-coverage.sh`
Result:
- Coverage run still fails with unhandled rejection.
- Blocking error remains present:
- `Error: ENOENT: no such file or directory, open '/projects/Charon/frontend/coverage/.tmp/coverage-132.json'`
- Run summary before abort:
- `Test Files 128 passed | 5 skipped (187)`
- `Tests 1918 passed | 90 skipped (2008)`
Additional state:
- `frontend/coverage/lcov.info` and `frontend/coverage/coverage-summary.json` can exist despite gate failure, but command-level DoD gate remains FAIL due non-zero termination path from unhandled ENOENT.
Gate disposition: FAIL.
### 3) Pre-commit hooks + version-check gate (revalidated)
Execution:
- `lefthook run pre-commit --all-files`
- `bash ./scripts/check-version-match-tag.sh`
Result:
- Pre-commit summary shows all required hooks completed successfully, including:
- `check-version-match`
- `golangci-lint-fast`
- `frontend-type-check`
- `frontend-lint`
- `semgrep`
- Version check output:
- `OK: .version matches latest Git tag v0.27.0`
Gate disposition: PASS.
## Remaining Non-Blocking Issues
### 4) SA1019 reconfirmation
1. GORM scanner reports 2 informational index recommendations in `backend/internal/models/user.go` (performance-oriented, not security-blocking).
2. Docker image scan still contains 4 Medium vulnerabilities (no Critical/High).
3. Frontend test output contains warning noise (React key warning, CrowdSec query undefined warnings) but these are not current blocking gates by themselves.
Execution:
- `cd backend && golangci-lint run ./... --enable-only staticcheck`
Result:
- Output: `0 issues.`
- Additional grep for `SA1019`: no matches.
Conclusion: SA1019 remains resolved.
### 5) Trivy FS reconfirmation for CVE-2026-34040
Execution:
- `trivy fs --scanners vuln .`
Result:
- Exit status: `0`
- Output indicates scan completed with:
- `Number of language-specific files num=0`
- CVE lookup:
- No `CVE-2026-34040` match found in available Trivy JSON artifacts (`vuln-results.json`, `trivy-image-report.json`).
Conclusion: CVE-2026-34040 not detected in current FS scan context.
## Local Patch Report Artifact Check
Execution:
- `bash /projects/Charon/scripts/local-patch-report.sh`
Result:
- Generated successfully in warn mode.
- Artifacts verified:
- `/projects/Charon/test-results/local-patch-report.md`
- `/projects/Charon/test-results/local-patch-report.json`
## Blocking Issues
1. Playwright E2E-first gate failing:
- e2e rebuild repeatedly canceled during Docker build, and fallback Firefox auth setup fails with 401.
2. Frontend coverage gate failing:
- coverage post-processing aborts with missing `.tmp` coverage file (ENOENT).
3. Pre-commit gate failing:
- `check-version-match` mismatch between `.version` and latest git tag.
1. Playwright E2E accessibility suite has one failing security headers test (axe timeout).
2. Frontend coverage command still fails with ENOENT under `frontend/coverage/.tmp`.
## Decision
Overall DoD decision for Issue #929: FAIL
Promotion recommendation: Do not approve/merge as DoD-complete until the three blocking gates above are remediated and rerun successfully.
Promotion recommendation: keep blocked until both failing mandatory gates are green on rerun.