Charon/docs/issues/created/20260204-manual-test-phase2-e2e-optimizations.md

Manual Test Plan: Phase 2 E2E Test Optimizations

Status: Pending Manual Testing Created: 2026-02-02 Priority: P1 (Performance Validation) Estimated Time: 30-45 minutes

Overview

Validate Phase 2 E2E test optimizations in real-world scenarios to ensure performance improvements don't introduce regressions or unexpected behavior.

Objective

Confirm that feature flag polling optimizations, cross-browser label helpers, and conditional verification logic work correctly across different browsers and test execution patterns.

Prerequisites

• E2E environment running (docker-rebuild-e2e completed)
• All browsers installed (Chromium, Firefox, WebKit)
• Clean test environment (no orphaned test data)
• Baseline metrics captured (pre-Phase 2)

---

Test Cases

TC-1: Feature Flag Polling Optimization

Goal: Verify feature flag changes propagate correctly without beforeEach polling

Steps:

1. Run system settings tests in isolation:

   npx playwright test tests/settings/system-settings.spec.ts --project=chromium
   

2. Monitor console output for feature flag API calls
3. Compare API call count to baseline (should be ~90% fewer)

Expected Results:

• ✅ All tests pass
• ✅ Feature flag toggles work correctly
• ✅ API calls reduced from ~31 to 3-5 per test file
• ✅ No inter-test dependencies (tests pass in any order)

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-2: Test Isolation with afterEach Cleanup

Goal: Verify test cleanup restores default state without side effects

Steps:

1. Run tests with random execution order:

   npx playwright test tests/settings/system-settings.spec.ts \
     --repeat-each=3 \
     --workers=1 \
     --project=chromium
   

2. Check for flakiness or state leakage between tests
3. Verify cleanup logs in console output

Expected Results:

• ✅ Tests pass consistently across all 3 runs
• ✅ No test failures due to unexpected initial state
• ✅ Cleanup logs show state restoration

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-3: Cross-Browser Label Locator (Chromium)

Goal: Verify label helper works in Chromium

Steps:

1. Run DNS provider tests in Chromium:

   npx playwright test tests/dns-provider-types.spec.ts --project=chromium --headed
   

2. Watch for "Script Path" field locator behavior
3. Verify no locator timeout errors

Expected Results:

• ✅ All DNS provider form tests pass
• ✅ Script path field located successfully
• ✅ No "strict mode violation" errors

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-4: Cross-Browser Label Locator (Firefox)

Goal: Verify label helper works in Firefox (previously failing)

Steps:

1. Run DNS provider tests in Firefox:

   npx playwright test tests/dns-provider-types.spec.ts --project=firefox --headed
   

2. Watch for "Script Path" field locator behavior
3. Verify fallback chain activates if primary locator fails

Expected Results:

• ✅ All DNS provider form tests pass
• ✅ Script path field located successfully (primary or fallback)
• ✅ No browser-specific workarounds needed

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-5: Cross-Browser Label Locator (WebKit)

Goal: Verify label helper works in WebKit (previously failing)

Steps:

1. Run DNS provider tests in WebKit:

   npx playwright test tests/dns-provider-types.spec.ts --project=webkit --headed
   

2. Watch for "Script Path" field locator behavior
3. Verify fallback chain activates if primary locator fails

Expected Results:

• ✅ All DNS provider form tests pass
• ✅ Script path field located successfully (primary or fallback)
• ✅ No browser-specific workarounds needed

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-6: Conditional Feature Flag Verification

Goal: Verify conditional skip optimization reduces polling iterations

Steps:

1. Enable debug logging in wait-helpers.ts (if available)
2. Run a test that verifies flags but doesn't toggle them:

   npx playwright test tests/security/security-dashboard.spec.ts --project=chromium
   

3. Check console logs for "[POLL] Feature flags already in expected state" messages

Expected Results:

• ✅ Tests pass
• ✅ Conditional skip activates when flags already match
• ✅ ~50% fewer polling iterations observed

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

TC-7: Full Suite Performance (All Browsers)

Goal: Verify overall test suite performance improved

Steps:

1. Run full E2E suite across all browsers:

   npx playwright test --project=chromium --project=firefox --project=webkit
   

2. Record total execution time
3. Compare to baseline metrics (pre-Phase 2)

Expected Results:

• ✅ All tests pass (except known skips)
• ✅ Execution time reduced by 20-30%
• ✅ No new flaky tests introduced
• ✅ No timeout errors observed

Actual Results:

• Pass / [ ] Fail
• Total time: _______ (Baseline: _______)
• Notes: _______________________

---

TC-8: Parallel Execution Stress Test

Goal: Verify optimizations handle parallel execution gracefully

Steps:

1. Run tests with maximum workers:

   npx playwright test tests/settings/system-settings.spec.ts --workers=4
   

2. Monitor for race conditions or resource contention
3. Check for worker-isolated cache behavior

Expected Results:

• ✅ Tests pass consistently
• ✅ No race conditions observed
• ✅ Worker isolation functions correctly
• ✅ Request coalescing reduces duplicate API calls

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

Regression Checks

RC-1: Existing Test Behavior

Goal: Verify Phase 2 changes don't break existing tests

Steps:

1. Run tests that don't use new helpers:

   npx playwright test tests/proxy-hosts/ --project=chromium
   

2. Verify backward compatibility

Expected Results:

• ✅ All tests pass
• ✅ No unexpected failures in unrelated tests

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

RC-2: CI/CD Pipeline Simulation

Goal: Verify changes work in CI environment

Steps:

1. Run tests with CI environment variables:

   CI=true npx playwright test --workers=1 --retries=2
   

2. Verify CI-specific behavior (retries, reporting)

Expected Results:

• ✅ Tests pass in CI mode
• ✅ Retry logic works correctly
• ✅ Reports generated successfully

Actual Results:

• Pass / [ ] Fail
• Notes: _______________________

---

Known Issues

Issue 1: E2E Test Interruptions (Non-Blocking)

• Location: tests/core/access-lists-crud.spec.ts:766, 794
• Impact: 2 tests interrupted during login
• Action: Tracked separately, not caused by Phase 2 changes

Issue 2: Frontend Security Page Test Failures (Non-Blocking)

• Location: src/pages/__tests__/Security.loading.test.tsx
• Impact: 15 test failures, WebSocket mock issues
• Action: Testing infrastructure issue, not E2E changes

---

Success Criteria

PASS Conditions:

• All manual test cases pass (TC-1 through TC-8)
• No new regressions introduced (RC-1, RC-2)
• Performance improvements validated (20-30% faster)
• Cross-browser compatibility confirmed

FAIL Conditions:

• Any CRITICAL test failures in Phase 2 changes
• New flaky tests introduced by optimizations
• Performance degradation observed
• Cross-browser compatibility broken

---

Sign-Off

Role	Name	Date	Status
QA Engineer	__________	_______	[ ] Pass / [ ] Fail
Tech Lead	__________	_______	[ ] Approved / [ ] Rejected

Notes: _____________________________________________

---

Next Actions

If PASS:

• Mark issue as complete
• Merge PR #609
• Monitor production metrics

If FAIL:

• Document failures in detail
• Create remediation tickets
• Re-run tests after fixes

Follow-Up Items (Regardless):

• Fix login flow timeouts (Issue tracked separately)
• Restore frontend coverage measurement
• Update baseline metrics documentation