Charon/docs/plans/archive/phase1-skipped-tests-remediation.md

Phase 1: Skipped Playwright Tests Remediation - Implementation Plan

Date: January 21, 2026 Status: Ready for Implementation Priority: P0 - Quick Wins Target: Enable 40+ skipped tests with minimal effort Estimated Effort: 2-4 hours

---

Executive Summary

This plan addresses the first phase of the skipped Playwright tests remediation. Phase 1 focuses on "quick wins" that can enable 40+ tests with minimal code changes. The primary fix involves enabling Cerberus in the E2E test environment, which alone will restore 35+ tests.

Phase 1 Targets

Fix	Tests Enabled	Effort	Files Modified
Enable Cerberus in E2E environment	35	5 min	2
Fix checkbox toggle wait in account-settings	1	10 min	1
Fix language selector test in system-settings	1	10 min	1
Stabilize keyboard navigation tests	3	30 min	2
Total	40	~1 hour	6

---

1. Enable Cerberus in E2E Environment (+35 tests)

1.1 Root Cause Analysis

Problem: The Cerberus security module is disabled in E2E test environments via FEATURE_CERBERUS_ENABLED=false. Tests check this flag at runtime and skip when false.

Evidence:

1. docker-compose.playwright.yml (line 54):

   - FEATURE_CERBERUS_ENABLED=false
   

2. docker-compose.e2e.yml (line 33):

   - FEATURE_CERBERUS_ENABLED=false
   

3. Test Skip Pattern in tests/monitoring/real-time-logs.spec.ts:

   let cerberusEnabled = false;
   // ...
   const connectionStatus = page.locator('[data-testid="connection-status"]');
   cerberusEnabled = await connectionStatus.isVisible({ timeout: 3000 }).catch(() => false);
   // ...
   test.skip(!cerberusEnabled, 'LiveLogViewer not available - Cerberus security module is disabled');
   

Affected Test Files:

• tests/monitoring/real-time-logs.spec.ts - 25 tests
• tests/security/security-dashboard.spec.ts - 7 tests
• tests/security/rate-limiting.spec.ts - 2+ tests

1.2 Implementation

Step 1: Update docker-compose.playwright.yml

File: .docker/compose/docker-compose.playwright.yml Line: 54 Change:

      # Security features - disabled by default for faster tests
      # Enable via profile: --profile security-tests
-     - FEATURE_CERBERUS_ENABLED=false
+     - FEATURE_CERBERUS_ENABLED=true
      - CHARON_SECURITY_CROWDSEC_MODE=disabled

Rationale: The Playwright compose file is used for E2E testing. Enabling Cerberus allows all security-related tests to run. CrowdSec mode can remain disabled (it has separate tests with the security-tests profile).

Step 2: Update docker-compose.e2e.yml

File: .docker/compose/docker-compose.e2e.yml Line: 33 Change:

      - CHARON_ACME_STAGING=true
-     - FEATURE_CERBERUS_ENABLED=false
+     - FEATURE_CERBERUS_ENABLED=true
    volumes:

Rationale: This is the legacy E2E compose file. Both files should have consistent configuration.

1.3 Verification

After making the changes:

# Rebuild E2E environment
docker compose -f .docker/compose/docker-compose.playwright.yml down
docker compose -f .docker/compose/docker-compose.playwright.yml up -d --build

# Wait for healthy
sleep 10

# Verify Cerberus is enabled
curl -s http://localhost:8080/api/v1/feature-flags | jq '."feature.cerberus.enabled"'
# Expected output: true

# Run affected tests
npx playwright test tests/monitoring/real-time-logs.spec.ts --project=chromium
npx playwright test tests/security/security-dashboard.spec.ts --project=chromium

Expected Result: 35+ tests that were previously skipped should now execute.

---

2. Fix Checkbox Toggle Wait in Account Settings (+1 test)

2.1 Root Cause Analysis

Problem: The checkbox toggle behavior in the account settings page is inconsistent. The test clicks the checkbox but the state change is not reliably detected.

Location: tests/settings/account-settings.spec.ts

Current Skip Reason (line 258):

/**
 * Test: Enter custom certificate email
 * Note: Skip - checkbox toggle behavior inconsistent; may need double-click or wait
 */
test.skip('should enter custom certificate email', async ({ page }) => {

Root Issue: The checkbox is a Radix UI component that uses custom rendering. Direct .click() may not reliably toggle the underlying input state. The working test at lines 200-250 uses:

const checkbox = page.getByRole('checkbox', { name: /use.*account.*email|same.*email/i });
await checkbox.click({ force: true });
await page.waitForTimeout(100);

2.2 Implementation

File: tests/settings/account-settings.spec.ts Lines: 259-275 Change:

    /**
     * Test: Enter custom certificate email
-    * Note: Skip - checkbox toggle behavior inconsistent; may need double-click or wait
     */
-   test.skip('should enter custom certificate email', async ({ page }) => {
+   test('should enter custom certificate email', async ({ page }) => {
      const customEmail = `cert-${Date.now()}@custom.local`;

      await test.step('Uncheck use account email', async () => {
-       const checkbox = page.locator('#useUserEmail');
-       await checkbox.click();
-       await expect(checkbox).not.toBeChecked();
+       // Use getByRole for Radix UI checkbox with force click
+       const checkbox = page.getByRole('checkbox', { name: /use.*account.*email|same.*email/i });
+
+       // First check if already unchecked
+       const isChecked = await checkbox.isChecked();
+       if (isChecked) {
+         await checkbox.click({ force: true });
+         // Wait for state transition
+         await page.waitForTimeout(100);
+       }
+       await expect(checkbox).not.toBeChecked({ timeout: 5000 });
      });

      await test.step('Enter custom email', async () => {
        const certEmailInput = page.locator('#cert-email');
-       await expect(certEmailInput).toBeVisible();
+       await expect(certEmailInput).toBeVisible({ timeout: 5000 });
        await certEmailInput.clear();
        await certEmailInput.fill(customEmail);
        await expect(certEmailInput).toHaveValue(customEmail);
      });
    });

2.3 Verification

npx playwright test tests/settings/account-settings.spec.ts \
  --grep "should enter custom certificate email" \
  --project=chromium

Expected Result: Test passes consistently.

---

3. Fix Language Selector Test in System Settings (+1 test)

3.1 Root Cause Analysis

Problem: The language selector test conditionally skips when it can't find the selector. The selector pattern is too broad and may not match the actual component.

Location: tests/settings/system-settings.spec.ts

Current Code:

await test.step('Find language selector', async () => {
  // Language selector may be a custom component
  const languageSelector = page
    .getByRole('combobox', { name: /language/i })
    .or(page.locator('[id*="language"]'))
    .or(page.getByText(/language/i).locator('..').locator('select, [role="combobox"]'));

  const hasLanguageSelector = await languageSelector.first().isVisible({ timeout: 3000 }).catch(() => false);

  if (hasLanguageSelector) {
    await expect(languageSelector.first()).toBeVisible();
  } else {
    // Skip if no language selector found
    test.skip();
  }
});

Actual Component: Based on frontend/src/components/LanguageSelector.tsx:

<select
  value={language}
  onChange={handleChange}
  className="bg-surface-elevated border border-border rounded-md..."
>

The component is a native <select> element without an ID or data-testid attribute.

3.2 Implementation

Step 1: Add data-testid to LanguageSelector component

File: frontend/src/components/LanguageSelector.tsx Change:

  return (
    <div className="flex items-center gap-3">
      <Globe className="h-5 w-5 text-content-secondary" />
      <select
+       data-testid="language-selector"
        value={language}
        onChange={handleChange}
        className="bg-surface-elevated border border-border rounded-md px-3 py-2 text-content-primary focus:outline-none focus:ring-2 focus:ring-primary focus:border-transparent transition-all"
      >

Step 2: Update the test to use the data-testid

File: tests/settings/system-settings.spec.ts Lines: 373-388 Change:

    await test.step('Find language selector', async () => {
-     // Language selector may be a custom component
-     const languageSelector = page
-       .getByRole('combobox', { name: /language/i })
-       .or(page.locator('[id*="language"]'))
-       .or(page.getByText(/language/i).locator('..').locator('select, [role="combobox"]'));
-
-     const hasLanguageSelector = await languageSelector.first().isVisible({ timeout: 3000 }).catch(() => false);
-
-     if (hasLanguageSelector) {
-       await expect(languageSelector.first()).toBeVisible();
-     } else {
-       // Skip if no language selector found
-       test.skip();
-     }
+     // Language selector is a native select element
+     const languageSelector = page.getByTestId('language-selector');
+     await expect(languageSelector).toBeVisible({ timeout: 5000 });
    });

3.3 Verification

# Rebuild frontend after component change
cd frontend && npm run build && cd ..

# Rebuild Docker image
docker compose -f .docker/compose/docker-compose.playwright.yml up -d --build

# Run the test
npx playwright test tests/settings/system-settings.spec.ts \
  --grep "language" \
  --project=chromium

Expected Result: Test finds the language selector and passes.

---

4. Stabilize Keyboard Navigation Tests (+3 tests)

4.1 Root Cause Analysis

Problem: Keyboard navigation tests are flaky due to timing issues with tab counts and focus detection.

Affected Tests:

1. tests/settings/account-settings.spec.ts#L675

   // Skip: Tab navigation order is browser/layout dependent
   test.skip('should be keyboard navigable', async ({ page }) => {
   

2. tests/settings/user-management.spec.ts#L1000

   // Skip: Keyboard navigation test is flaky due to timing issues with tab count
   test.skip('should be keyboard navigable', async ({ page }) => {
   

3. tests/core/navigation.spec.ts#L597

   // TODO: Implement skip-to-content link in the application
   test.skip('should have skip to main content link', async ({ page }) => {
   

Root Issues:

• Tests loop through tab presses looking for specific elements
• Focus order is layout-dependent and may vary
• No explicit waits between key presses
• The skip-to-content test requires an actual skip link implementation (intentional skip)

4.2 Implementation

Fix 1: Account Settings Keyboard Navigation

File: tests/settings/account-settings.spec.ts Lines: 670-720 Change:

  test.describe('Accessibility', () => {
    /**
     * Test: Keyboard navigation through account settings
-    * Note: Skip - Tab navigation order is browser/layout dependent
     */
-   test.skip('should be keyboard navigable', async ({ page }) => {
+   test('should be keyboard navigable', async ({ page }) => {
      await test.step('Tab through profile section', async () => {
        // Start from first focusable element
        await page.keyboard.press('Tab');
+       await page.waitForTimeout(50); // Brief pause for focus to settle

        // Tab to profile name
        const nameInput = page.locator('#profile-name');
        let foundName = false;

-       for (let i = 0; i < 15; i++) {
+       for (let i = 0; i < 20; i++) {
          if (await nameInput.evaluate((el) => el === document.activeElement)) {
            foundName = true;
            break;
          }
          await page.keyboard.press('Tab');
+         await page.waitForTimeout(50); // Allow focus to update
        }

        expect(foundName).toBeTruthy();
      });

      await test.step('Tab through password section', async () => {
        const currentPasswordInput = page.locator('#current-password');
        let foundPassword = false;

-       for (let i = 0; i < 20; i++) {
+       for (let i = 0; i < 25; i++) {
          if (await currentPasswordInput.evaluate((el) => el === document.activeElement)) {
            foundPassword = true;
            break;
          }
          await page.keyboard.press('Tab');
+         await page.waitForTimeout(50);
        }

        expect(foundPassword).toBeTruthy();
      });

Fix 2: User Management Keyboard Navigation

File: tests/settings/user-management.spec.ts Lines: 995-1060 Change:

    /**
     * Test: Keyboard navigation
     * Priority: P1
     */
-   // Skip: Keyboard navigation test is flaky due to timing issues with tab count
-   test.skip('should be keyboard navigable', async ({ page }) => {
+   test('should be keyboard navigable', async ({ page }) => {
      await test.step('Tab to invite button', async () => {
        await page.keyboard.press('Tab');
+       await page.waitForTimeout(50);

        let foundInviteButton = false;
-       for (let i = 0; i < 10; i++) {
+       for (let i = 0; i < 15; i++) {
          const focused = page.locator(':focus');
          const text = await focused.textContent().catch(() => '');

          if (text?.toLowerCase().includes('invite')) {
            foundInviteButton = true;
            break;
          }
          await page.keyboard.press('Tab');
+         await page.waitForTimeout(50);
        }

        expect(foundInviteButton).toBeTruthy();
      });

      await test.step('Activate with Enter key', async () => {
        await page.keyboard.press('Enter');
+       await page.waitForTimeout(200); // Wait for modal animation

        // Modal should open
        const modal = page.locator('[class*="fixed"]').filter({
          has: page.getByRole('heading', { name: /invite/i }),
        });
-       await expect(modal).toBeVisible();
+       await expect(modal).toBeVisible({ timeout: 5000 });
      });

      await test.step('Close modal with Escape', async () => {
        await page.keyboard.press('Escape');
+       await page.waitForTimeout(200); // Wait for modal close animation

        // Modal should close (if escape is wired up)
        const closeButton = page.getByRole('button', { name: /close|×|cancel/i });
        if (await closeButton.isVisible()) {
          await closeButton.click();
        }
      });

      await test.step('Tab through table rows', async () => {
        // Focus should be able to reach action buttons in table
        let foundActionButton = false;

-       for (let i = 0; i < 20; i++) {
+       for (let i = 0; i < 30; i++) {
          await page.keyboard.press('Tab');
+         await page.waitForTimeout(50);
          const focused = page.locator(':focus');
          const tagName = await focused.evaluate((el) => el.tagName.toLowerCase()).catch(() => '');

          if (tagName === 'button') {
            const isInTable = await focused.evaluate((el) => {
              return !!el.closest('table');
            }).catch(() => false);

            if (isInTable) {
              foundActionButton = true;
              break;

Note on Skip-to-Content Test

The test at tests/core/navigation.spec.ts#L597 requires implementing an actual skip-to-content link in the application. This is an intentional skip and should remain skipped until the application feature is implemented.

This is a Phase 2+ task - requires frontend development to add the skip link component.

4.3 Verification

# Run account settings keyboard test
npx playwright test tests/settings/account-settings.spec.ts \
  --grep "keyboard navigable" \
  --project=chromium

# Run user management keyboard test
npx playwright test tests/settings/user-management.spec.ts \
  --grep "keyboard navigable" \
  --project=chromium

Expected Result: Both keyboard navigation tests pass consistently.

---

Implementation Order

Execute changes in this order to avoid build failures:

Step 1: Frontend Component Change

1. Add data-testid="language-selector" to frontend/src/components/LanguageSelector.tsx
2. Rebuild frontend: cd frontend && npm run build

Step 2: Docker Configuration Changes

1. Update .docker/compose/docker-compose.playwright.yml - set FEATURE_CERBERUS_ENABLED=true
2. Update .docker/compose/docker-compose.e2e.yml - set FEATURE_CERBERUS_ENABLED=true
3. Rebuild: docker compose -f .docker/compose/docker-compose.playwright.yml up -d --build

Step 3: Test File Updates

1. Update tests/settings/account-settings.spec.ts:
   • Fix checkbox toggle test (lines 259-275)
   • Fix keyboard navigation test (lines 670-720)
2. Update tests/settings/system-settings.spec.ts:
   • Fix language selector test (lines 373-388)
3. Update tests/settings/user-management.spec.ts:
   • Fix keyboard navigation test (lines 995-1060)

Step 4: Verification

# Run full E2E test suite to verify
npx playwright test --project=chromium

# Or run specific affected files
npx playwright test \
  tests/monitoring/real-time-logs.spec.ts \
  tests/security/security-dashboard.spec.ts \
  tests/security/rate-limiting.spec.ts \
  tests/settings/account-settings.spec.ts \
  tests/settings/system-settings.spec.ts \
  tests/settings/user-management.spec.ts \
  --project=chromium

---

Files to Modify Summary

File	Type	Changes
.docker/compose/docker-compose.playwright.yml	Config	Line 54: FEATURE_CERBERUS_ENABLED=true
.docker/compose/docker-compose.e2e.yml	Config	Line 33: FEATURE_CERBERUS_ENABLED=true
frontend/src/components/LanguageSelector.tsx	React	Add data-testid="language-selector"
tests/settings/account-settings.spec.ts	Test	Lines 259-275, 670-720: Fix skipped tests
tests/settings/system-settings.spec.ts	Test	Lines 373-388: Fix selector pattern
tests/settings/user-management.spec.ts	Test	Lines 995-1060: Fix keyboard navigation

---

Success Metrics

Metric	Before	After	Target
Skipped Tests (Total)	98	~58	<60
Cerberus Tests Running	0	35	35
Account Settings Skips	3	1*	1*
System Settings Skips	4	3	3
User Management Skips	22	21	21

*Note: Some skips are intentional (e.g., skip-to-content link not implemented)

---

Rollback Plan

If issues occur, revert these changes:

# Revert Docker configs
git checkout .docker/compose/docker-compose.playwright.yml
git checkout .docker/compose/docker-compose.e2e.yml

# Revert frontend component
git checkout frontend/src/components/LanguageSelector.tsx

# Revert test files
git checkout tests/settings/account-settings.spec.ts
git checkout tests/settings/system-settings.spec.ts
git checkout tests/settings/user-management.spec.ts

# Rebuild
docker compose -f .docker/compose/docker-compose.playwright.yml up -d --build

---

Next Phase Preview

After Phase 1 completion, Phase 2 will address:

1. TestDataManager Authentication Fix (+8 tests)

   • Refactor to use authenticated API context
   • Update auth-fixtures.ts

2. SMTP Persistence Backend Fix (+3 tests)

   • Investigate /api/v1/settings/smtp endpoint

3. Import Route Implementation (+6 tests)

   • Implement NPM/JSON import handlers

---

Change Log

Date	Author	Change
2026-01-21	Planning Agent	Initial Phase 1 implementation plan