489cd93384
- Updated design documentation to reflect the new Playwright-first approach for frontend testing, including orchestration flow and runbook notes. - Revised requirements to align with the new frontend test iteration strategy, emphasizing E2E environment management and coverage thresholds. - Expanded tasks to outline phased implementation for frontend testing, including Playwright E2E baseline, backend triage, and coverage validation. - Enhanced QA report to capture frontend coverage failures and type errors, with detailed remediation steps for accessibility compliance. - Created new security validation and accessibility remediation reports for CrowdSec configuration, addressing identified issues and implementing fixes. - Adjusted package.json scripts to prioritize Firefox for Playwright tests. - Added canonical links for requirements and tasks documentation.
2.3 KiB
2.3 KiB
Requirements - Frontend Test Iteration
Source: docs/plans/current_spec.md
EARS Requirements
- WHEN the frontend test iteration begins, THE SYSTEM SHALL rebuild the E2E environment only when application or Docker build inputs changed, and SHALL skip rebuild for test-only changes if the container is already healthy.
- WHEN Playwright tests are executed, THE SYSTEM SHALL run the setup project before browser projects and preserve storage state at tests/playwright/.auth/user.json.
- WHEN Playwright failures occur, THE SYSTEM SHALL capture the failing test file, failing step, and related helper or fixture in tests/utils or tests/fixtures.
- WHEN Vitest unit tests are executed, THE SYSTEM SHALL apply frontend/src/test/setup.ts and honor coverage thresholds from CHARON_MIN_COVERAGE or CPM_MIN_COVERAGE.
- WHEN coverage is enforced, THE SYSTEM SHALL meet 100 percent patch coverage and at least the configured frontend minimum coverage threshold.
- WHEN a failure indicates backend behavior (HTTP 4xx/5xx or missing API contract), THE SYSTEM SHALL open a backend triage path before modifying frontend tests.
- WHEN Phase 3, 4, or 5 test runs are executed, THE SYSTEM SHALL use the VS Code task labels defined in the plan and avoid ad hoc commands.
- WHEN the targeted Playwright rerun task is required, THE SYSTEM SHALL create the task in Phase 0 if it does not already exist.
- WHEN Phase 5 validation runs are executed, THE SYSTEM SHALL run Lint: TypeScript Check and record a zero-error result before completion.
- WHEN PoC/MVP success criteria are evaluated, THE SYSTEM SHALL require the top 3 failing suites to pass twice (baseline plus one rerun), allow one rerun per suite, and record that no new failures were introduced.
- WHEN a developer runs the Navigation Shard task, THE SYSTEM SHALL execute only tests/core/navigation.spec.ts using the Playwright Firefox project (firefox).
- WHEN the Navigation Shard task executes, THE SYSTEM SHALL apply --shard=1/1 to preserve CI-style shard semantics.
- WHEN the Navigation Shard task runs, THE SYSTEM SHALL keep Cerberus dependencies disabled by setting PLAYWRIGHT_SKIP_SECURITY_DEPS=1.
- WHEN the Navigation Shard task completes, THE SYSTEM SHALL produce standard Playwright outputs in playwright-report/ and test-results/.