diff --git a/docs/issues/manual-test-e2e-feature-flags.md b/docs/issues/manual-test-e2e-feature-flags.md deleted file mode 100644 index 13a3612d..00000000 --- a/docs/issues/manual-test-e2e-feature-flags.md +++ /dev/null @@ -1,165 +0,0 @@ -# Manual Test Plan: E2E Feature Flags Timeout Fix - -**Created:** 2026-02-02 -**Priority:** P1 - High -**Type:** Manual Testing -**Component:** E2E Tests, Feature Flags API -**Related PR:** #583 - ---- - -## Objective - -Manually verify the E2E test timeout fix implementation works correctly in a real CI environment after resolving the Playwright infrastructure issue. - -## Prerequisites - -- [ ] Playwright deduplication issue resolved: `rm -rf node_modules && npm install && npm dedupe` -- [ ] E2E container rebuilt: `.github/skills/scripts/skill-runner.sh docker-rebuild-e2e` -- [ ] Container health check passing: `docker ps` shows `charon-e2e` as healthy - -## Test Scenarios - -### 1. Feature Flag Toggle Tests (Chromium) - -**File:** `tests/settings/system-settings.spec.ts` - -**Execute:** -```bash -npx playwright test tests/settings/system-settings.spec.ts --project=chromium --workers=1 --retries=0 -``` - -**Expected Results:** -- [ ] All 7 tests pass (4 refactored + 3 new) -- [ ] Zero timeout errors -- [ ] Test execution time: ≤5s per test -- [ ] Console shows retry attempts (if transient failures occur) - -**Tests to Validate:** -1. [ ] `should toggle Cerberus security feature` -2. [ ] `should toggle CrowdSec console enrollment` -3. [ ] `should toggle uptime monitoring` -4. [ ] `should persist feature toggle changes` -5. [ ] `should handle concurrent toggle operations` -6. [ ] `should retry on 500 Internal Server Error` -7. [ ] `should fail gracefully after max retries exceeded` - -### 2. Cross-Browser Validation - -**Execute:** -```bash -npx playwright test tests/settings/system-settings.spec.ts --project=chromium --project=firefox --project=webkit -``` - -**Expected Results:** -- [ ] All browsers pass: Chromium, Firefox, WebKit -- [ ] No browser-specific timeout issues -- [ ] Consistent behavior across browsers - -### 3. Performance Metrics Extraction - -**Execute:** -```bash -docker logs charon-e2e 2>&1 | grep "\[METRICS\]" -``` - -**Expected Results:** -- [ ] Metrics logged for GET operations: `[METRICS] GET /feature-flags: {latency}ms` -- [ ] Metrics logged for PUT operations: `[METRICS] PUT /feature-flags: {latency}ms` -- [ ] Latency values: <200ms P99 (CI environment) - -### 4. Reliability Test (10 Consecutive Runs) - -**Execute:** -```bash -for i in {1..10}; do - echo "Run $i of 10" - npx playwright test tests/settings/system-settings.spec.ts --project=chromium --workers=1 --retries=0 - if [ $? -ne 0 ]; then - echo "FAILED on run $i" - break - fi -done -``` - -**Expected Results:** -- [ ] 10/10 runs pass (100% pass rate) -- [ ] Zero timeout errors across all runs -- [ ] Retry attempts: <5% of operations - -### 5. UI Verification - -**Manual Steps:** -1. [ ] Navigate to `/settings/system` in browser -2. [ ] Toggle Cerberus security feature switch -3. [ ] Verify toggle animation completes -4. [ ] Verify "Saved" notification appears -5. [ ] Refresh page -6. [ ] Verify toggle state persists - -**Expected Results:** -- [ ] UI responsive (<1s toggle feedback) -- [ ] State changes reflect immediately -- [ ] No console errors - -## Bug Discovery Focus - -**Look for potential issues in:** - -### Backend Performance -- [ ] Feature flags endpoint latency spikes (>500ms) -- [ ] Database lock timeouts -- [ ] Transaction rollback failures -- [ ] Memory leaks after repeated toggles - -### Test Resilience -- [ ] Retry logic not triggering on transient failures -- [ ] Polling timeouts on slow CI runners -- [ ] Race conditions in concurrent toggle test -- [ ] Hard-coded wait remnants causing flakiness - -### Edge Cases -- [ ] Concurrent toggles causing data corruption -- [ ] Network failures not handled gracefully -- [ ] Max retries not throwing expected error -- [ ] Initial state mismatch in `beforeEach` - -## Success Criteria - -- [ ] All 35 checks above pass without issues -- [ ] Zero timeout errors in 10 consecutive runs -- [ ] Performance metrics confirm <200ms P99 latency -- [ ] Cross-browser compatibility verified -- [ ] No new bugs discovered during manual testing - -## Failure Handling - -**If any test fails:** - -1. **Capture Evidence:** - - Screenshot of failure - - Full test output (no truncation) - - `docker logs charon-e2e` output - - Network/console logs from browser DevTools - -2. **Analyze Root Cause:** - - Is it a code defect or infrastructure issue? - - Is it reproducible locally? - - Does it happen in all browsers? - -3. **Take Action:** - - **Code Defect:** Reopen issue, describe failure, assign to developer - - **Infrastructure:** Document in known issues, create follow-up ticket - - **Flaky Test:** Investigate retry logic, increase timeouts if justified - -## Notes - -- Run tests during low CI load times for accurate performance measurement -- Use `--headed` flag for UI verification: `npx playwright test --headed` -- Check Playwright trace if tests fail: `npx playwright show-report` - ---- - -**Assigned To:** QA Team -**Estimated Time:** 2-3 hours -**Due Date:** Within 24 hours of Playwright infrastructure fix