# Charon E2E Testing Plan: Comprehensive Playwright Coverage **Date:** January 18, 2026 **Status:** Phase 3 Complete - 346+ tests passing **Priority:** Critical - Blocking new feature development **Objective:** Establish comprehensive E2E test coverage for all existing Charon features **Timeline:** 10 weeks (with proper infrastructure setup and comprehensive feature coverage) > **Revision Note:** This document has been completely revised to address critical infrastructure gaps, expand underspecified sections, and provide implementation-ready specifications. Major additions include test data management, authentication strategy, CI/CD integration, flaky test prevention, and detailed security feature testing. --- ## Table of Contents 1. [Current State & Coverage Gaps](#1-current-state--coverage-gaps) 2. [Testing Infrastructure](#2-testing-infrastructure) - 2.1 [Test Environment Setup](#21-test-environment-setup) - 2.2 [Test Data Management Strategy](#22-test-data-management-strategy) - 2.3 [Authentication Strategy](#23-authentication-strategy) - 2.4 [Flaky Test Prevention](#24-flaky-test-prevention) - 2.5 [CI/CD Integration](#25-cicd-integration) 3. [Test Organization](#3-test-organization) 4. [Implementation Plan](#4-implementation-plan) - Phase 0: Infrastructure Setup (Week 1-2) - Phase 1: Foundation (Week 3) - Phase 2: Critical Path (Week 4-5) - Phase 3: Security Features (Week 6-7) - Phase 4: Settings (Week 8) - Phase 5: Tasks (Week 9) - Phase 6: Integration (Week 10) 5. [Security Feature Testing Strategy](#5-security-feature-testing-strategy) 6. [Risk Mitigation](#6-risk-mitigation) 7. [Success Metrics](#7-success-metrics) 8. [Next Steps](#8-next-steps) --- ## 1. Current State & Coverage Gaps ### Existing Test Files **Current E2E Test Coverage:** - ✅ `tests/auth.setup.ts` - Authentication setup (shared fixture) - ✅ `tests/manual-dns-provider.spec.ts` - Manual DNS provider E2E tests (comprehensive) - ✅ `tests/dns-provider-crud.spec.ts` - DNS provider CRUD operations - ✅ `tests/dns-provider-types.spec.ts` - DNS provider type validation - ✅ `tests/example.spec.js` - Legacy example (can be removed) - ✅ `tests/fixtures/dns-providers.ts` - Shared DNS test fixtures **Critical Infrastructure Gaps Identified:** - ❌ No test data management system (causes data conflicts, FK violations) - ❌ No per-test user creation (shared auth state breaks parallel execution) - ❌ No CI/CD integration strategy (no automated testing on PRs) - ❌ No flaky test prevention utilities (arbitrary timeouts everywhere) - ❌ No environment setup documentation (manual setup, no verification) - ❌ No mock external service strategy (tests depend on real services) ### Coverage Gaps **All major features lack E2E test coverage except DNS providers:** - ❌ Proxy Hosts management - ❌ Access Lists (ACL) - ❌ SSL Certificates - ❌ CrowdSec integration - ❌ Coraza WAF - ❌ Rate Limiting - ❌ Security Headers - ❌ Backups & Restore - ❌ User Management - ❌ System Settings - ❌ Audit Logs - ❌ Remote Servers - ❌ Uptime Monitoring - ❌ Notifications - ❌ Import/Export features - ❌ Encryption Management --- ## 2. Testing Infrastructure ### 2.1 Test Environment Setup **Objective:** Ensure consistent, reproducible test environments for local development and CI. #### 2.1.1 Local Development Setup **Prerequisites:** - Docker and Docker Compose installed - Node.js 18+ and npm - Go 1.21+ (for backend development) - Playwright browsers installed (`npx playwright install`) **Environment Configuration:** ```bash # .env.test (create in project root) NODE_ENV=test DATABASE_URL=sqlite:./data/charon_test.db BASE_URL=http://localhost:8080 PLAYWRIGHT_BASE_URL=http://localhost:8080 TEST_USER_EMAIL=test-admin@charon.local TEST_USER_PASSWORD=TestPassword123! DOCKER_HOST=unix:///var/run/docker.sock ENABLE_CROWDSEC=false # Disabled for unit tests, enabled for integration ENABLE_WAF=false LOG_LEVEL=warn ``` **Required Docker Services:** > **Note:** Use the committed `docker-compose.playwright.yml` for E2E testing. > The `docker-compose.test.yml` is gitignored and reserved for personal/local configurations. ```yaml # .docker/compose/docker-compose.playwright.yml # See the actual file for the full configuration with: # - Charon app service with test environment # - Optional CrowdSec profile: --profile security-tests # - Optional MailHog profile: --profile notification-tests # # Usage: # docker compose -f .docker/compose/docker-compose.playwright.yml up -d # docker compose -f .docker/compose/docker-compose.playwright.yml --profile security-tests up -d **Setup Script:** ```bash #!/bin/bash # scripts/setup-e2e-env.sh set -euo pipefail echo "🚀 Setting up E2E test environment..." # 1. Check prerequisites command -v docker >/dev/null 2>&1 || { echo "❌ Docker not found"; exit 1; } command -v node >/dev/null 2>&1 || { echo "❌ Node.js not found"; exit 1; } # 2. Install dependencies echo "📦 Installing dependencies..." npm ci # 3. Install Playwright browsers echo "🎭 Installing Playwright browsers..." npx playwright install chromium # 4. Create test environment file if [ ! -f .env.test ]; then echo "📝 Creating .env.test..." cp .env.example .env.test # Set test-specific values sed -i 's/NODE_ENV=.*/NODE_ENV=test/' .env.test sed -i 's/DATABASE_URL=.*/DATABASE_URL=sqlite:\.\/data\/charon_test.db/' .env.test fi # 5. Start test environment echo "🐳 Starting Docker services..." docker compose -f .docker/compose/docker-compose.playwright.yml up -d # 6. Wait for service health echo "⏳ Waiting for service to be healthy..." timeout 60 bash -c 'until docker compose -f .docker/compose/docker-compose.playwright.yml exec -T charon-app curl -f http://localhost:8080/api/v1/health; do sleep 2; done' # 7. Run database migrations echo "🗄️ Running database migrations..." docker compose -f .docker/compose/docker-compose.playwright.yml exec -T charon-app /app/backend/charon migrate echo "✅ E2E environment ready!" echo "📍 Application: http://localhost:8080" echo "🧪 Run tests: npm run test:e2e" ``` **Environment Health Check:** ```typescript // tests/utils/health-check.ts export async function waitForHealthyEnvironment(baseURL: string, timeout = 60000): Promise { const startTime = Date.now(); while (Date.now() - startTime < timeout) { try { const response = await fetch(`${baseURL}/api/v1/health`); if (response.ok) { const health = await response.json(); if (health.status === 'healthy' && health.database === 'connected') { console.log('✅ Environment is healthy'); return; } } } catch (error) { // Service not ready yet } await new Promise(resolve => setTimeout(resolve, 2000)); } throw new Error(`Environment not healthy after ${timeout}ms`); } export async function verifyTestPrerequisites(): Promise { const checks = { 'Database writable': async () => { // Attempt to create a test record const response = await fetch(`${process.env.PLAYWRIGHT_BASE_URL}/api/v1/test/db-check`, { method: 'POST' }); return response.ok; }, 'Docker socket accessible': async () => { // Check if Docker is available (for proxy host tests) const response = await fetch(`${process.env.PLAYWRIGHT_BASE_URL}/api/v1/test/docker-check`); return response.ok; } }; for (const [name, check] of Object.entries(checks)) { try { const result = await check(); if (!result) throw new Error(`Check failed: ${name}`); console.log(`✅ ${name}`); } catch (error) { console.error(`❌ ${name}: ${error}`); throw new Error(`Prerequisite check failed: ${name}`); } } } ``` #### 2.1.2 CI Environment Configuration **GitHub Actions Environment:** - Use `localhost:8080` instead of Tailscale IP - Run services in Docker containers - Use GitHub Actions cache for dependencies and browsers - Upload test artifacts on failure **Network Configuration:** ```yaml # In CI, all services communicate via Docker network services: charon: networks: - test-network networks: test-network: driver: bridge ``` #### 2.1.3 Mock External Service Strategy **DNS Provider API Mocks:** ```typescript // tests/mocks/dns-provider-api.ts import { rest } from 'msw'; import { setupServer } from 'msw/node'; export const dnsProviderMocks = [ // Mock Cloudflare API rest.post('https://api.cloudflare.com/client/v4/zones/:zoneId/dns_records', (req, res, ctx) => { return res( ctx.status(200), ctx.json({ success: true, result: { id: 'mock-record-id', name: req.body.name } }) ); }), // Mock Route53 API rest.post('https://route53.amazonaws.com/*', (req, res, ctx) => { return res(ctx.status(200), ctx.xml('mock-change-id')); }) ]; export const mockServer = setupServer(...dnsProviderMocks); ``` **ACME Server Mock (for certificate tests):** ```typescript // tests/mocks/acme-server.ts export const acmeMocks = [ // Mock Let's Encrypt directory rest.get('https://acme-v02.api.letsencrypt.org/directory', (req, res, ctx) => { return res(ctx.json({ newNonce: 'https://mock-acme/new-nonce', newAccount: 'https://mock-acme/new-account', newOrder: 'https://mock-acme/new-order' })); }) ]; ``` --- ### 2.2 Test Data Management Strategy **Critical Problem:** Current approach uses shared test data, causing conflicts in parallel execution and leaving orphaned records. **Solution:** Implement `TestDataManager` utility with namespaced isolation and guaranteed cleanup. #### 2.2.1 TestDataManager Design ```typescript // tests/utils/TestDataManager.ts import { APIRequestContext } from '@playwright/test'; import crypto from 'crypto'; export interface ManagedResource { id: string; type: 'proxy-host' | 'certificate' | 'access-list' | 'dns-provider' | 'user'; namespace: string; createdAt: Date; } export class TestDataManager { private resources: ManagedResource[] = []; private namespace: string; private request: APIRequestContext; constructor(request: APIRequestContext, testName?: string) { this.request = request; // Create unique namespace per test to avoid conflicts this.namespace = testName ? `test-${this.sanitize(testName)}-${Date.now()}` : `test-${crypto.randomUUID()}`; } private sanitize(name: string): string { return name.toLowerCase().replace(/[^a-z0-9]/g, '-').substring(0, 30); } /** * Create a proxy host with automatic cleanup tracking */ async createProxyHost(data: { domain: string; forwardHost: string; forwardPort: number; scheme?: 'http' | 'https'; }): Promise<{ id: string; domain: string }> { const namespaced = { ...data, domain: `${this.namespace}.${data.domain}` // Ensure unique domain }; const response = await this.request.post('/api/v1/proxy-hosts', { data: namespaced }); if (!response.ok()) { throw new Error(`Failed to create proxy host: ${await response.text()}`); } const result = await response.json(); this.resources.push({ id: result.uuid, type: 'proxy-host', namespace: this.namespace, createdAt: new Date() }); return { id: result.uuid, domain: namespaced.domain }; } /** * Create an access list with automatic cleanup */ async createAccessList(data: { name: string; rules: Array<{ type: 'allow' | 'deny'; value: string }>; }): Promise<{ id: string }> { const namespaced = { ...data, name: `${this.namespace}-${data.name}` }; const response = await this.request.post('/api/v1/access-lists', { data: namespaced }); if (!response.ok()) { throw new Error(`Failed to create access list: ${await response.text()}`); } const result = await response.json(); this.resources.push({ id: result.id, type: 'access-list', namespace: this.namespace, createdAt: new Date() }); return { id: result.id }; } /** * Create a certificate with automatic cleanup */ async createCertificate(data: { domains: string[]; type: 'letsencrypt' | 'custom'; privateKey?: string; certificate?: string; }): Promise<{ id: string }> { const namespaced = { ...data, domains: data.domains.map(d => `${this.namespace}.${d}`) }; const response = await this.request.post('/api/v1/certificates', { data: namespaced }); if (!response.ok()) { throw new Error(`Failed to create certificate: ${await response.text()}`); } const result = await response.json(); this.resources.push({ id: result.id, type: 'certificate', namespace: this.namespace, createdAt: new Date() }); return { id: result.id }; } /** * Create a DNS provider with automatic cleanup */ async createDNSProvider(data: { type: 'manual' | 'cloudflare' | 'route53'; name: string; credentials?: Record; }): Promise<{ id: string }> { const namespaced = { ...data, name: `${this.namespace}-${data.name}` }; const response = await this.request.post('/api/v1/dns-providers', { data: namespaced }); if (!response.ok()) { throw new Error(`Failed to create DNS provider: ${await response.text()}`); } const result = await response.json(); this.resources.push({ id: result.id, type: 'dns-provider', namespace: this.namespace, createdAt: new Date() }); return { id: result.id }; } /** * Create a test user with automatic cleanup */ async createUser(data: { email: string; password: string; role: 'admin' | 'user' | 'guest'; }): Promise<{ id: string; email: string; token: string }> { const namespaced = { ...data, email: `${this.namespace}+${data.email}` }; const response = await this.request.post('/api/v1/users', { data: namespaced }); if (!response.ok()) { throw new Error(`Failed to create user: ${await response.text()}`); } const result = await response.json(); this.resources.push({ id: result.id, type: 'user', namespace: this.namespace, createdAt: new Date() }); // Automatically log in the user and return token const loginResponse = await this.request.post('/api/v1/auth/login', { data: { email: namespaced.email, password: data.password } }); const { token } = await loginResponse.json(); return { id: result.id, email: namespaced.email, token }; } /** * Clean up all resources in reverse order (respects FK constraints) */ async cleanup(): Promise { // Sort by creation time (newest first) to respect dependencies const sortedResources = [...this.resources].sort( (a, b) => b.createdAt.getTime() - a.createdAt.getTime() ); const errors: Error[] = []; for (const resource of sortedResources) { try { await this.deleteResource(resource); } catch (error) { errors.push(error as Error); console.error(`Failed to cleanup ${resource.type}:${resource.id}:`, error); } } this.resources = []; if (errors.length > 0) { throw new Error(`Cleanup completed with ${errors.length} errors`); } } private async deleteResource(resource: ManagedResource): Promise { const endpoints = { 'proxy-host': `/api/v1/proxy-hosts/${resource.id}`, 'certificate': `/api/v1/certificates/${resource.id}`, 'access-list': `/api/v1/access-lists/${resource.id}`, 'dns-provider': `/api/v1/dns-providers/${resource.id}`, 'user': `/api/v1/users/${resource.id}` }; const endpoint = endpoints[resource.type]; const response = await this.request.delete(endpoint); if (!response.ok() && response.status() !== 404) { throw new Error(`Failed to delete ${resource.type}: ${await response.text()}`); } } /** * Get all resources created in this namespace */ getResources(): ManagedResource[] { return [...this.resources]; } /** * Get namespace identifier */ getNamespace(): string { return this.namespace; } } ``` #### 2.2.2 Usage Pattern ```typescript // Example test using TestDataManager import { test, expect } from '@playwright/test'; import { TestDataManager } from './utils/TestDataManager'; test.describe('Proxy Host Management', () => { let testData: TestDataManager; test.beforeEach(async ({ request }, testInfo) => { testData = new TestDataManager(request, testInfo.title); }); test.afterEach(async () => { await testData.cleanup(); }); test('should create and delete proxy host', async ({ page, request }) => { await test.step('Create proxy host', async () => { const { id, domain } = await testData.createProxyHost({ domain: 'app.example.com', forwardHost: '192.168.1.100', forwardPort: 3000, scheme: 'http' }); await page.goto('/proxy-hosts'); await expect(page.getByText(domain)).toBeVisible(); }); // Cleanup happens automatically in afterEach }); }); ``` #### 2.2.3 Database Seeding Strategy **Seed Data for Reference Tests:** ```typescript // tests/fixtures/seed-data.ts export async function seedReferenceData(request: APIRequestContext): Promise { // Create stable reference data that doesn't change const referenceData = { accessLists: [ { name: 'Global Allowlist', rules: [{ type: 'allow', value: '0.0.0.0/0' }] } ], dnsProviders: [ { type: 'manual', name: 'Manual DNS (Default)' } ] }; // Idempotent seeding - only create if not exists for (const list of referenceData.accessLists) { const response = await request.get('/api/v1/access-lists', { params: { name: list.name } }); const existing = await response.json(); if (existing.length === 0) { await request.post('/api/v1/access-lists', { data: list }); } } } ``` #### 2.2.4 Parallel Execution Handling **Test Isolation Strategy:** - Each test worker gets its own namespace via `TestDataManager` - Database transactions are NOT used (not supported in E2E context) - Unique identifiers prevent collisions (domain names, usernames) - Cleanup runs independently per test **Worker ID Integration:** ```typescript // playwright.config.ts adjustment export default defineConfig({ workers: process.env.CI ? 4 : undefined, use: { storageState: ({ workerIndex }) => `auth/state-worker-${workerIndex}.json` } }); ``` --- ### 2.3 Authentication Strategy **Critical Problem:** Current `auth.setup.ts` uses a single shared user, causing race conditions in parallel execution. **Solution:** Per-test user creation with role-based fixtures. #### 2.3.1 Per-Test User Creation ```typescript // tests/fixtures/auth-fixtures.ts import { test as base, expect, APIRequestContext } from '@playwright/test'; import { TestDataManager } from '../utils/TestDataManager'; export interface TestUser { id: string; email: string; token: string; role: 'admin' | 'user' | 'guest'; } interface AuthFixtures { authenticatedUser: TestUser; adminUser: TestUser; regularUser: TestUser; guestUser: TestUser; testData: TestDataManager; } export const test = base.extend({ testData: async ({ request }, use, testInfo) => { const manager = new TestDataManager(request, testInfo.title); await use(manager); await manager.cleanup(); }, // Default authenticated user (admin role) authenticatedUser: async ({ testData }, use, testInfo) => { const user = await testData.createUser({ email: `admin-${Date.now()}@test.local`, password: 'TestPass123!', role: 'admin' }); await use(user); }, // Explicit admin user fixture adminUser: async ({ testData }, use) => { const user = await testData.createUser({ email: `admin-${Date.now()}@test.local`, password: 'TestPass123!', role: 'admin' }); await use(user); }, // Regular user (non-admin) regularUser: async ({ testData }, use) => { const user = await testData.createUser({ email: `user-${Date.now()}@test.local`, password: 'TestPass123!', role: 'user' }); await use(user); }, // Guest user (read-only) guestUser: async ({ testData }, use) => { const user = await testData.createUser({ email: `guest-${Date.now()}@test.local`, password: 'TestPass123!', role: 'guest' }); await use(user); } }); export { expect } from '@playwright/test'; ``` #### 2.3.2 Usage Pattern ```typescript // Example test with per-test authentication import { test, expect } from './fixtures/auth-fixtures'; test.describe('User Management', () => { test('admin can create users', async ({ page, adminUser }) => { await test.step('Login as admin', async () => { await page.goto('/login'); await page.getByLabel('Email').fill(adminUser.email); await page.getByLabel('Password').fill('TestPass123!'); await page.getByRole('button', { name: 'Login' }).click(); await page.waitForURL('/'); }); await test.step('Create new user', async () => { await page.goto('/users'); await page.getByRole('button', { name: 'Add User' }).click(); // ... rest of test }); }); test('regular user cannot create users', async ({ page, regularUser }) => { await test.step('Login as regular user', async () => { await page.goto('/login'); await page.getByLabel('Email').fill(regularUser.email); await page.getByLabel('Password').fill('TestPass123!'); await page.getByRole('button', { name: 'Login' }).click(); await page.waitForURL('/'); }); await test.step('Verify no access to user management', async () => { await page.goto('/users'); await expect(page.getByText('Access Denied')).toBeVisible(); }); }); }); ``` #### 2.3.3 Storage State Management **Per-Worker Storage:** ```typescript // tests/auth.setup.ts (revised) import { test as setup, expect } from '@playwright/test'; import { TestDataManager } from './utils/TestDataManager'; // Generate storage state per worker const authFile = process.env.CI ? `auth/state-worker-${process.env.TEST_WORKER_INDEX || 0}.json` : 'auth/state.json'; setup('authenticate', async ({ request, page }) => { const testData = new TestDataManager(request, 'setup'); try { // Create a dedicated setup user for this worker const user = await testData.createUser({ email: `setup-worker-${process.env.TEST_WORKER_INDEX || 0}@test.local`, password: 'SetupPass123!', role: 'admin' }); await page.goto('/login'); await page.getByLabel('Email').fill(user.email); await page.getByLabel('Password').fill('SetupPass123!'); await page.getByRole('button', { name: 'Login' }).click(); await page.waitForURL('/'); // Save authenticated state await page.context().storageState({ path: authFile }); console.log(`✅ Auth state saved for worker ${process.env.TEST_WORKER_INDEX || 0}`); } finally { // Cleanup happens automatically via TestDataManager await testData.cleanup(); } }); ``` --- ### 2.4 Flaky Test Prevention **Critical Problem:** Arbitrary timeouts (`page.waitForTimeout(1000)`) cause flaky tests and slow execution. **Solution:** Deterministic wait utilities that poll for specific conditions. #### 2.4.1 Wait Utilities ```typescript // tests/utils/wait-helpers.ts import { Page, Locator, expect } from '@playwright/test'; /** * Wait for a toast notification with specific text */ export async function waitForToast( page: Page, text: string | RegExp, options: { timeout?: number; type?: 'success' | 'error' | 'info' } = {} ): Promise { const { timeout = 10000, type } = options; const toastSelector = type ? `[role="alert"][data-type="${type}"]` : '[role="alert"]'; const toast = page.locator(toastSelector); await expect(toast).toContainText(text, { timeout }); } /** * Wait for a specific API response */ export async function waitForAPIResponse( page: Page, urlPattern: string | RegExp, options: { status?: number; timeout?: number } = {} ): Promise { const { status, timeout = 30000 } = options; const responsePromise = page.waitForResponse( (response) => { const matchesURL = typeof urlPattern === 'string' ? response.url().includes(urlPattern) : urlPattern.test(response.url()); const matchesStatus = status ? response.status() === status : true; return matchesURL && matchesStatus; }, { timeout } ); return await responsePromise; } /** * Wait for loading spinner to disappear */ export async function waitForLoadingComplete( page: Page, options: { timeout?: number } = {} ): Promise { const { timeout = 10000 } = options; // Wait for any loading indicator to disappear const loader = page.locator('[role="progressbar"], [aria-busy="true"], .loading-spinner'); await expect(loader).toHaveCount(0, { timeout }); } /** * Wait for specific element count (e.g., table rows) */ export async function waitForElementCount( locator: Locator, count: number, options: { timeout?: number } = {} ): Promise { const { timeout = 10000 } = options; await expect(locator).toHaveCount(count, { timeout }); } /** * Wait for WebSocket connection to be established */ export async function waitForWebSocketConnection( page: Page, urlPattern: string | RegExp, options: { timeout?: number } = {} ): Promise { const { timeout = 10000 } = options; await page.waitForEvent('websocket', { predicate: (ws) => { const matchesURL = typeof urlPattern === 'string' ? ws.url().includes(urlPattern) : urlPattern.test(ws.url()); return matchesURL; }, timeout }); } /** * Wait for WebSocket message with specific content */ export async function waitForWebSocketMessage( page: Page, matcher: (data: string | Buffer) => boolean, options: { timeout?: number } = {} ): Promise { const { timeout = 10000 } = options; return new Promise((resolve, reject) => { const timer = setTimeout(() => { reject(new Error(`WebSocket message not received within ${timeout}ms`)); }, timeout); page.on('websocket', (ws) => { ws.on('framereceived', (event) => { const data = event.payload; if (matcher(data)) { clearTimeout(timer); resolve(data); } }); }); }); } /** * Wait for progress bar to complete */ export async function waitForProgressComplete( page: Page, options: { timeout?: number } = {} ): Promise { const { timeout = 30000 } = options; const progressBar = page.locator('[role="progressbar"]'); // Wait for progress to reach 100% or disappear await page.waitForFunction( () => { const bar = document.querySelector('[role="progressbar"]'); if (!bar) return true; // Progress bar gone = complete const value = bar.getAttribute('aria-valuenow'); return value === '100'; }, { timeout } ); // Wait for progress bar to disappear await expect(progressBar).toHaveCount(0, { timeout: 5000 }); } /** * Wait for modal to open */ export async function waitForModal( page: Page, titleText: string | RegExp, options: { timeout?: number } = {} ): Promise { const { timeout = 10000 } = options; const modal = page.locator('[role="dialog"]'); await expect(modal).toBeVisible({ timeout }); if (titleText) { await expect(modal.getByRole('heading')).toContainText(titleText); } return modal; } /** * Wait for dropdown/listbox to open */ export async function waitForDropdown( page: Page, triggerId: string, options: { timeout?: number } = {} ): Promise { const { timeout = 5000 } = options; const trigger = page.locator(`#${triggerId}`); const expanded = await trigger.getAttribute('aria-expanded'); if (expanded !== 'true') { throw new Error(`Dropdown ${triggerId} is not expanded`); } const listboxId = await trigger.getAttribute('aria-controls'); if (!listboxId) { throw new Error(`Dropdown ${triggerId} has no aria-controls`); } const listbox = page.locator(`#${listboxId}`); await expect(listbox).toBeVisible({ timeout }); return listbox; } /** * Wait for table to finish loading and render rows */ export async function waitForTableLoad( page: Page, tableRole: string = 'table', options: { minRows?: number; timeout?: number } = {} ): Promise { const { minRows = 0, timeout = 10000 } = options; const table = page.getByRole(tableRole); await expect(table).toBeVisible({ timeout }); // Wait for loading state to clear await waitForLoadingComplete(page); // If minimum rows specified, wait for them if (minRows > 0) { const rows = table.locator('tbody tr'); await expect(rows).toHaveCount(minRows, { timeout }); } } /** * Retry an action until it succeeds or timeout */ export async function retryAction( action: () => Promise, options: { maxAttempts?: number; interval?: number; timeout?: number; } = {} ): Promise { const { maxAttempts = 5, interval = 1000, timeout = 30000 } = options; const startTime = Date.now(); let lastError: Error | undefined; for (let attempt = 1; attempt <= maxAttempts; attempt++) { if (Date.now() - startTime > timeout) { throw new Error(`Retry timeout after ${timeout}ms`); } try { return await action(); } catch (error) { lastError = error as Error; if (attempt < maxAttempts) { await new Promise(resolve => setTimeout(resolve, interval)); } } } throw lastError || new Error('Retry failed'); } ``` #### 2.4.2 Usage Examples ```typescript import { test, expect } from '@playwright/test'; import { waitForToast, waitForAPIResponse, waitForLoadingComplete, waitForModal } from './utils/wait-helpers'; test('create proxy host with deterministic waits', async ({ page }) => { await test.step('Navigate and open form', async () => { await page.goto('/proxy-hosts'); await page.getByRole('button', { name: 'Add Proxy Host' }).click(); await waitForModal(page, 'Create Proxy Host'); }); await test.step('Fill form and submit', async () => { await page.getByLabel('Domain Name').fill('test.example.com'); await page.getByLabel('Forward Host').fill('192.168.1.100'); await page.getByLabel('Forward Port').fill('3000'); // Wait for API call to complete const responsePromise = waitForAPIResponse(page, '/api/v1/proxy-hosts', { status: 201 }); await page.getByRole('button', { name: 'Save' }).click(); await responsePromise; }); await test.step('Verify success', async () => { await waitForToast(page, 'Proxy host created successfully', { type: 'success' }); await waitForLoadingComplete(page); await expect(page.getByRole('row', { name: /test.example.com/ })).toBeVisible(); }); }); ``` --- ### 2.5 CI/CD Integration **Objective:** Automate E2E test execution on every PR with parallel execution, comprehensive reporting, and failure artifacts. #### 2.5.1 GitHub Actions Workflow ```yaml # .github/workflows/e2e-tests.yml name: E2E Tests (Playwright) on: pull_request: branches: [main, develop] paths: - 'frontend/**' - 'backend/**' - 'tests/**' - 'playwright.config.js' - '.github/workflows/e2e-tests.yml' push: branches: [main] workflow_dispatch: inputs: browser: description: 'Browser to test' required: false default: 'chromium' type: choice options: - chromium - firefox - webkit - all env: NODE_VERSION: '18' GO_VERSION: '1.21' jobs: # Build application once, share across test shards build: name: Build Application runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Go uses: actions/setup-go@v5 with: go-version: ${{ env.GO_VERSION }} cache: true - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: ${{ env.NODE_VERSION }} cache: 'npm' - name: Install dependencies run: npm ci - name: Build frontend run: npm run build working-directory: frontend - name: Build backend run: make build working-directory: backend - name: Build Docker image run: | docker build -t charon:test . docker save charon:test -o charon-test-image.tar - name: Upload Docker image uses: actions/upload-artifact@v4 with: name: docker-image path: charon-test-image.tar retention-days: 1 # Run tests in parallel shards e2e-tests: name: E2E Tests (Shard ${{ matrix.shard }}) runs-on: ubuntu-latest needs: build timeout-minutes: 30 strategy: fail-fast: false matrix: shard: [1, 2, 3, 4] browser: [chromium] # Can be extended to [chromium, firefox, webkit] steps: - uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: ${{ env.NODE_VERSION }} cache: 'npm' - name: Download Docker image uses: actions/download-artifact@v4 with: name: docker-image - name: Load Docker image run: docker load -i charon-test-image.tar - name: Start test environment run: | docker compose -f .docker/compose/docker-compose.playwright.yml up -d - name: Wait for service healthy run: | timeout 60 bash -c 'until curl -f http://localhost:8080/api/v1/health; do sleep 2; done' - name: Install dependencies run: npm ci - name: Install Playwright browsers run: npx playwright install --with-deps ${{ matrix.browser }} - name: Run E2E tests (Shard ${{ matrix.shard }}) run: | npx playwright test \ --project=${{ matrix.browser }} \ --shard=${{ matrix.shard }}/4 \ --reporter=html,json,junit env: PLAYWRIGHT_BASE_URL: http://localhost:8080 CI: true TEST_WORKER_INDEX: ${{ matrix.shard }} - name: Upload test results if: always() uses: actions/upload-artifact@v4 with: name: test-results-${{ matrix.browser }}-shard-${{ matrix.shard }} path: | playwright-report/ test-results/ retention-days: 7 - name: Upload test traces if: failure() uses: actions/upload-artifact@v4 with: name: traces-${{ matrix.browser }}-shard-${{ matrix.shard }} path: test-results/**/*.zip retention-days: 7 - name: Collect Docker logs if: failure() run: | docker compose -f .docker/compose/docker-compose.playwright.yml logs > docker-logs.txt - name: Upload Docker logs if: failure() uses: actions/upload-artifact@v4 with: name: docker-logs-shard-${{ matrix.shard }} path: docker-logs.txt retention-days: 7 # Merge reports from all shards merge-reports: name: Merge Test Reports runs-on: ubuntu-latest needs: e2e-tests if: always() steps: - uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: ${{ env.NODE_VERSION }} - name: Download all reports uses: actions/download-artifact@v4 with: pattern: test-results-* path: all-results - name: Merge Playwright HTML reports run: npx playwright merge-reports --reporter html all-results - name: Upload merged report uses: actions/upload-artifact@v4 with: name: merged-playwright-report path: playwright-report/ retention-days: 30 - name: Generate summary run: | echo "## E2E Test Results" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY cat all-results/*/report.json | jq -s ' { total: (map(.stats.expected + .stats.unexpected + .stats.flaky + .stats.skipped) | add), passed: (map(.stats.expected) | add), failed: (map(.stats.unexpected) | add), flaky: (map(.stats.flaky) | add), skipped: (map(.stats.skipped) | add) } ' | jq -r '"- **Total**: \(.total)\n- **Passed**: \(.passed)\n- **Failed**: \(.failed)\n- **Flaky**: \(.flaky)\n- **Skipped**: \(.skipped)"' >> $GITHUB_STEP_SUMMARY # Comment on PR with results comment-results: name: Comment Test Results on PR runs-on: ubuntu-latest needs: merge-reports if: github.event_name == 'pull_request' permissions: pull-requests: write steps: - name: Download merged report uses: actions/download-artifact@v4 with: name: merged-playwright-report path: playwright-report - name: Extract test stats id: stats run: | STATS=$(cat playwright-report/report.json | jq -c '.stats') echo "stats=$STATS" >> $GITHUB_OUTPUT - name: Comment on PR uses: actions/github-script@v7 with: script: | const stats = JSON.parse('${{ steps.stats.outputs.stats }}'); const passed = stats.expected; const failed = stats.unexpected; const flaky = stats.flaky; const total = passed + failed + flaky + stats.skipped; const emoji = failed > 0 ? '❌' : flaky > 0 ? '⚠️' : '✅'; const status = failed > 0 ? 'FAILED' : flaky > 0 ? 'FLAKY' : 'PASSED'; const body = `## ${emoji} E2E Test Results: ${status} | Metric | Count | |--------|-------| | Total | ${total} | | Passed | ✅ ${passed} | | Failed | ❌ ${failed} | | Flaky | ⚠️ ${flaky} | | Skipped | ⏭️ ${stats.skipped} | [View full Playwright report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}) ${failed > 0 ? '⚠️ **Tests failed!** Please review the failures and fix before merging.' : ''} ${flaky > 0 ? '⚠️ **Flaky tests detected!** Please investigate and stabilize before merging.' : ''} `; github.rest.issues.createComment({ issue_number: context.issue.number, owner: context.repo.owner, repo: context.repo.repo, body: body }); # Block merge if tests fail e2e-results: name: E2E Test Results runs-on: ubuntu-latest needs: e2e-tests if: always() steps: - name: Check test results run: | if [ "${{ needs.e2e-tests.result }}" != "success" ]; then echo "E2E tests failed or were cancelled" exit 1 fi ``` #### 2.5.2 Test Sharding Strategy **Why Shard:** Reduces CI run time from ~40 minutes to ~10 minutes with 4 parallel shards. **Sharding Configuration:** ```typescript // playwright.config.ts export default defineConfig({ testDir: './tests', fullyParallel: true, workers: process.env.CI ? 4 : undefined, retries: process.env.CI ? 2 : 0, projects: [ { name: 'chromium', use: { ...devices['Desktop Chrome'] } } ], // CI-specific optimizations ...(process.env.CI && { reporter: [ ['html', { outputFolder: 'playwright-report' }], ['json', { outputFile: 'test-results/results.json' }], ['junit', { outputFile: 'test-results/junit.xml' }] ], maxFailures: 10 // Stop after 10 failures to save CI time }) }); ``` **Shard Distribution:** - Shard 1: `tests/core/**`, `tests/proxy/**` (~10 min) - Shard 2: `tests/dns/**`, `tests/certificates/**` (~10 min) - Shard 3: `tests/security/**` (~10 min) - Shard 4: `tests/settings/**`, `tests/tasks/**`, `tests/monitoring/**`, `tests/integration/**` (~10 min) #### 2.5.3 Cache Strategy ```yaml # Cache Playwright browsers - name: Cache Playwright browsers uses: actions/cache@v4 with: path: ~/.cache/ms-playwright key: playwright-browsers-${{ hashFiles('package-lock.json') }} # Cache npm dependencies - name: Cache npm dependencies uses: actions/cache@v4 with: path: ~/.npm key: npm-${{ hashFiles('package-lock.json') }} # Cache Docker layers - name: Cache Docker layers uses: actions/cache@v4 with: path: /tmp/.buildx-cache key: docker-${{ github.sha }} restore-keys: docker- ``` #### 2.5.4 Failure Notification Strategy **Slack Notification (Optional):** ```yaml - name: Notify Slack on failure if: failure() && github.event_name == 'push' uses: slackapi/slack-github-action@v1 with: payload: | { "text": "❌ E2E tests failed on ${{ github.ref }}", "blocks": [ { "type": "section", "text": { "type": "mrkdwn", "text": "*E2E Tests Failed*\n\nBranch: `${{ github.ref }}`\nCommit: `${{ github.sha }}`\n" } } ] } env: SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} ``` --- ## 3. Test Organization **Directory Structure:** ``` tests/ ├── auth.setup.ts # ✅ Exists - Authentication setup ├── fixtures/ # Shared test data and utilities │ ├── dns-providers.ts # ✅ Exists - DNS fixtures │ ├── proxy-hosts.ts # 📝 To create │ ├── access-lists.ts # 📝 To create │ ├── certificates.ts # 📝 To create │ └── test-data.ts # 📝 To create - Common test data ├── core/ # Core application features │ ├── authentication.spec.ts # 📝 Auth flows │ ├── dashboard.spec.ts # 📝 Dashboard UI │ └── navigation.spec.ts # 📝 App navigation ├── proxy/ # Proxy management │ ├── proxy-hosts-crud.spec.ts # 📝 CRUD operations │ ├── proxy-hosts-validation.spec.ts # 📝 Form validation │ ├── proxy-hosts-docker.spec.ts # 📝 Docker discovery │ └── remote-servers.spec.ts # 📝 Remote Docker servers ├── dns/ # DNS management │ ├── manual-dns-provider.spec.ts # ✅ Exists - Manual provider │ ├── dns-provider-crud.spec.ts # ✅ Exists - CRUD operations │ ├── dns-provider-types.spec.ts # ✅ Exists - Provider types │ ├── dns-provider-credentials.spec.ts # 📝 Credential management │ └── dns-plugins.spec.ts # 📝 Plugin management ├── certificates/ # SSL certificate management │ ├── certificates-list.spec.ts # 📝 List and view │ ├── certificates-upload.spec.ts # 📝 Upload custom certs │ └── certificates-acme.spec.ts # 📝 ACME integration ├── security/ # Cerberus security suite │ ├── access-lists-crud.spec.ts # 📝 ACL CRUD │ ├── access-lists-rules.spec.ts # 📝 ACL rule engine │ ├── crowdsec-config.spec.ts # 📝 CrowdSec setup │ ├── crowdsec-decisions.spec.ts # 📝 Ban management │ ├── crowdsec-presets.spec.ts # 📝 Preset management │ ├── waf-config.spec.ts # 📝 WAF configuration │ ├── rate-limiting.spec.ts # 📝 Rate limit rules │ ├── security-headers.spec.ts # 📝 Security headers │ └── audit-logs.spec.ts # 📝 Audit trail ├── settings/ # System configuration │ ├── system-settings.spec.ts # 📝 System config │ ├── smtp-settings.spec.ts # 📝 Email config │ ├── notifications.spec.ts # 📝 Notification config │ ├── user-management.spec.ts # 📝 User CRUD │ ├── encryption-management.spec.ts # 📝 Key rotation │ └── account-settings.spec.ts # 📝 User profile ├── tasks/ # Background tasks & maintenance │ ├── backups-create.spec.ts # 📝 Backup creation │ ├── backups-restore.spec.ts # 📝 Backup restoration │ ├── logs-viewing.spec.ts # 📝 Log viewer │ ├── import-caddyfile.spec.ts # 📝 Caddy import │ └── import-crowdsec.spec.ts # 📝 CrowdSec import ├── monitoring/ # Monitoring features │ ├── uptime-monitoring.spec.ts # 📝 Uptime checks │ └── real-time-logs.spec.ts # 📝 WebSocket logs └── integration/ # Cross-feature integration ├── proxy-acl-integration.spec.ts # 📝 Proxy + ACL ├── proxy-certificate.spec.ts # 📝 Proxy + SSL ├── security-suite-integration.spec.ts # 📝 Full security stack └── backup-restore-e2e.spec.ts # 📝 Full backup cycle ``` ### Test Execution Strategy **Playwright Configuration:** - ✅ Base URL: `http://100.98.12.109:8080` (Tailscale IP) or `http://localhost:8080` (CI) - ✅ Browser support: Chromium (primary), Firefox, WebKit - ✅ Parallel execution: Enabled for faster runs - ✅ Authentication: Shared state via `auth.setup.ts` - ✅ Timeouts: 30s test, 5s expect - ✅ Retries: 2 on CI, 0 on local **Test Data Management:** - Use fixtures for reusable test data - Clean up created resources after tests - Use unique identifiers for test resources (timestamps, UUIDs) - Avoid hardcoded IDs or names that could conflict **Accessibility Testing:** - All tests must verify keyboard navigation - Use `toMatchAriaSnapshot` for component structure validation - Verify ARIA labels, roles, and live regions - Test screen reader announcements for state changes --- ## 4. Implementation Plan ### Phase 0: Infrastructure Setup (Week 1-2) **Goal:** Build robust test infrastructure before writing feature tests #### Week 1: Core Infrastructure **Priority:** Critical - Blocking all test development **Estimated Effort:** 5 days **Tasks:** - [ ] Set up `TestDataManager` utility with namespace isolation - [ ] Implement per-test user creation in `auth-fixtures.ts` - [ ] Create all wait helper utilities (`waitForToast`, `waitForAPIResponse`, etc.) - [ ] Configure test environment Docker Compose files (`.test.yml`) - [ ] Write `setup-e2e-env.sh` script with health checks - [ ] Implement mock external services (DNS providers, ACME servers) - [ ] Configure test environment variables in `.env.test` **Acceptance Criteria:** - `TestDataManager` can create and cleanup all resource types - Per-test users can be created with different roles - Wait utilities replace all `page.waitForTimeout()` calls - Test environment starts reliably with `npm run test:env:start` - Mock services respond to API calls correctly #### Week 2: CI/CD Integration **Priority:** Critical - Required for PR automation **Estimated Effort:** 5 days **Tasks:** - [ ] Create `.github/workflows/e2e-tests.yml` - [ ] Implement test sharding strategy (4 shards) - [ ] Configure artifact upload (reports, traces, logs) - [ ] Set up PR comment reporting - [ ] Configure caching for npm, Playwright browsers, Docker layers - [ ] Test workflow end-to-end on a feature branch - [ ] Document CI/CD troubleshooting guide **Acceptance Criteria:** - E2E tests run automatically on every PR - Test results appear as PR comments - Failed tests upload traces and logs - CI run completes in <15 minutes with sharding - Flaky test retries work correctly ### Phase 1: Foundation (Week 3) **Status:** ✅ COMPLETE **Completion Date:** January 17, 2026 **Test Results:** 112/119 passing (94%) **Goal:** Establish core application testing patterns #### 1.1 Test Fixtures & Helpers **Priority:** Critical **Status:** ✅ Complete **Delivered Files:** - [x] `tests/fixtures/test-data.ts` - Common test data generators - [x] `tests/fixtures/proxy-hosts.ts` - Mock proxy host data - [x] `tests/fixtures/access-lists.ts` - Mock ACL data - [x] `tests/fixtures/certificates.ts` - Mock certificate data - [x] `tests/fixtures/auth-fixtures.ts` - Per-test authentication - [x] `tests/fixtures/navigation.ts` - Navigation helpers - [x] `tests/utils/api-helpers.ts` - Common API operations - [x] `tests/utils/wait-helpers.ts` - Deterministic wait utilities - [x] `tests/utils/test-data-manager.ts` - Test data isolation - [x] `tests/utils/accessibility-helpers.ts` - A11y testing utilities **Acceptance Criteria:** ✅ Met - Fixtures provide consistent, reusable test data - API helpers reduce code duplication - All utilities have JSDoc comments and usage examples **Test File Template:** ```typescript import { test, expect } from './fixtures/auth-fixtures'; // Use custom fixtures import { TestDataManager } from './utils/TestDataManager'; test.describe('Feature Name', () => { let testData: TestDataManager; test.beforeEach(async ({ request, page }, testInfo) => { testData = new TestDataManager(request, testInfo.title); await page.goto('/feature-path'); }); test.afterEach(async () => { await testData.cleanup(); // Guaranteed cleanup }); test('should perform specific action', async ({ page, authenticatedUser }) => { await test.step('User action', async () => { // Use authenticatedUser fixture for API calls await page.goto('/feature'); await page.getByRole('button', { name: 'Action' }).click(); }); await test.step('Verify result', async () => { await expect(page.getByText('Success')).toBeVisible(); }); }); }); ``` #### 1.2 Core Authentication & Navigation Tests **Priority:** Critical **Status:** ✅ Complete (with known issues tracked) **Delivered Test Files:** **`tests/core/authentication.spec.ts`** - 16 tests (13 passing, 3 failing - tracked) - ✅ Login with valid credentials (covered by auth.setup.ts) - ✅ Login with invalid credentials - ✅ Logout functionality - ✅ Session persistence - ⚠️ Session expiration handling (3 tests failing - see [Issue: e2e-session-expiration-tests](../issues/e2e-session-expiration-tests.md)) - ✅ Password reset flow (if implemented) **`tests/core/dashboard.spec.ts`** - All tests passing - ✅ Dashboard loads successfully - ✅ Summary cards display correct data - ✅ Quick action buttons are functional - ✅ Recent activity shows latest changes - ✅ System status indicators work **`tests/core/navigation.spec.ts`** - All tests passing - ✅ All main menu items are clickable - ✅ Sidebar navigation works - ✅ Breadcrumbs display correctly - ✅ Deep links resolve properly - ✅ Back button navigation works **Known Issues:** - 3 session expiration tests require route mocking - tracked in [docs/issues/e2e-session-expiration-tests.md](../issues/e2e-session-expiration-tests.md) **Acceptance Criteria:** ✅ Met (with known exceptions) - All authentication flows covered (session expiration deferred) - Dashboard displays without errors - Navigation between all pages works - No console errors during navigation - Keyboard navigation fully functional ### Phase 2: Critical Path (Week 4-5) **Goal:** Cover the most critical user journeys #### 2.1 Proxy Hosts Management **Priority:** Critical **Estimated Effort:** 4 days **Test Files:** **`tests/proxy/proxy-hosts-crud.spec.ts`** Test Scenarios: - ✅ List all proxy hosts (empty state) - ✅ Create new proxy host with basic configuration - Enter domain name (e.g., `test-app.example.com`) - Enter forward hostname (e.g., `192.168.1.100`) - Enter forward port (e.g., `3000`) - Select scheme (HTTP/HTTPS) - Enable/disable WebSocket support - Save and verify host appears in list - ✅ View proxy host details - ✅ Edit existing proxy host - Update domain name - Update forward hostname/port - Toggle WebSocket support - Save and verify changes - ✅ Delete proxy host - Delete single host - Verify deletion confirmation dialog - Verify host removed from list - ✅ Bulk operations (if supported) **Key User Flows:** 1. **Create Basic Proxy Host:** ``` Navigate → Click "Add Proxy Host" → Fill form → Save → Verify in list ``` 2. **Edit Existing Host:** ``` Navigate → Select host → Click edit → Modify → Save → Verify changes ``` 3. **Delete Host:** ``` Navigate → Select host → Click delete → Confirm → Verify removal ``` **Critical Assertions:** - Host appears in list after creation - Edit changes are persisted - Deletion removes host from database - Validation prevents invalid data - Success/error messages display correctly #### 2.2 SSL Certificates Management **Priority:** Critical **Estimated Effort:** 4 days **Test Files:** **`tests/certificates/certificates-list.spec.ts`** Test Scenarios: - ✅ List all certificates (empty state) - ✅ Display certificate details (domain, expiry, issuer) - ✅ Filter certificates by status (valid, expiring, expired) - ✅ Sort certificates by expiry date - ✅ Search certificates by domain name - ✅ Show certificate chain details **`tests/certificates/certificates-upload.spec.ts`** Test Scenarios: - ✅ Upload custom certificate with private key - ✅ Validate PEM format - ✅ Reject invalid certificate formats - ✅ Reject mismatched certificate and key - ✅ Support intermediate certificate chains - ✅ Update existing certificate - ✅ Delete custom certificate **`tests/certificates/certificates-acme.spec.ts`** Test Scenarios: **ACME HTTP-01 Challenge:** - ✅ Request certificate via HTTP-01 challenge - Select domain from proxy hosts - Choose HTTP-01 validation method - Verify challenge file is served at `/.well-known/acme-challenge/` - Mock ACME server validates challenge - Certificate issued and stored - ✅ HTTP-01 challenge fails if proxy host not accessible - ✅ HTTP-01 challenge fails with invalid domain **ACME DNS-01 Challenge:** - ✅ Request certificate via DNS-01 challenge - Select DNS provider (Cloudflare, Route53, Manual) - Mock DNS provider API for TXT record creation - Verify TXT record `_acme-challenge.domain.com` created - Mock ACME server validates DNS record - Certificate issued and stored - ✅ DNS-01 challenge supports wildcard certificates - Request `*.example.com` certificate - Verify TXT record for `_acme-challenge.example.com` - Certificate covers all subdomains - ✅ DNS-01 challenge fails with invalid DNS credentials - ✅ DNS-01 challenge retries on DNS propagation delay **Certificate Renewal:** - ✅ Automatic renewal triggered 30 days before expiry - Mock certificate with expiry in 29 days - Verify renewal task scheduled - Renewal completes successfully - Old certificate archived - ✅ Manual certificate renewal - Click "Renew Now" button - Renewal process uses same validation method - New certificate replaces old - ✅ Renewal fails gracefully - Old certificate remains active - Error notification displayed - Retry mechanism available **Wildcard Certificates:** - ✅ Request wildcard certificate (`*.example.com`) - DNS-01 challenge required (HTTP-01 not supported) - Verify TXT record created - Certificate issued with wildcard SAN - ✅ Wildcard certificate applies to all subdomains - Create proxy host `app.example.com` - Wildcard certificate auto-selected - HTTPS works for any subdomain **Certificate Revocation:** - ✅ Revoke Let's Encrypt certificate - Click "Revoke" button - Confirm revocation reason - Certificate marked as revoked - ACME server notified - ✅ Revoked certificate cannot be used - Proxy hosts using certificate show warning - HTTPS connections fail **Validation Error Handling:** - ✅ ACME account registration fails - Invalid email address - Rate limit exceeded - Network error during registration - ✅ Challenge validation fails - HTTP-01: Challenge file not accessible - DNS-01: TXT record not found - DNS-01: DNS propagation timeout - ✅ Certificate issuance fails - ACME server error - Domain validation failed - Rate limit exceeded **Mixed Certificate Sources:** - ✅ Use Let's Encrypt and custom certificates together - Some domains use Let's Encrypt - Some domains use custom certificates - Certificates don't conflict - ✅ Migrate from custom to Let's Encrypt - Replace custom certificate with Let's Encrypt - No downtime during migration - Old certificate archived **Certificate Metadata:** - ✅ Display certificate information - Issuer, subject, validity period - SAN (Subject Alternative Names) - Signature algorithm - Certificate chain - ✅ Export certificate in various formats - PEM, DER, PFX - With or without private key - Include full chain **Key User Flows:** 1. **HTTP-01 Challenge Flow:** ``` Navigate → Click "Request Certificate" → Select domain → Choose HTTP-01 → Monitor challenge → Certificate issued → Verify in list ``` 2. **DNS-01 Wildcard Flow:** ``` Navigate → Click "Request Certificate" → Enter *.example.com → Choose DNS-01 → Select DNS provider → Monitor DNS propagation → Certificate issued → Verify wildcard works ``` 3. **Certificate Renewal Flow:** ``` Navigate → Select expiring certificate → Click "Renew" → Automatic challenge re-validation → New certificate issued → Old certificate archived ``` **Critical Assertions:** - Challenge files/records created correctly - ACME server validates challenges - Certificates issued with correct domains - Renewal happens before expiry - Validation errors display helpful messages - Certificate chain is complete and valid #### 2.3 Access Lists (ACL) **Priority:** Critical **Estimated Effort:** 3 days **Test Files:** **`tests/access-lists/access-lists-crud.spec.ts`** Test Scenarios: - ✅ List all access lists (empty state) - Verify empty state message displayed - "Create Access List" CTA visible - ✅ Create IP whitelist (Allow Only) - Enter name (e.g., "Office IPs") - Add description - Select type: IP Whitelist - Add IP rules (single IP, CIDR ranges) - Save and verify appears in list - ✅ Create IP blacklist (Block Only) - Select type: IP Blacklist - Add blocked IPs/ranges - Verify badge shows "Deny" - ✅ Create geo-whitelist - Select type: Geo Whitelist - Select allowed countries (US, CA, GB) - Verify country badges displayed - ✅ Create geo-blacklist - Select type: Geo Blacklist - Block high-risk countries - Apply security presets - ✅ Enable/disable access list - Toggle enabled state - Verify badge shows correct status - ✅ Edit access list - Update name, description, rules - Add/remove IP ranges - Change type (whitelist ↔ blacklist) - ✅ Delete access list - Confirm backup creation before delete - Verify removed from list - Verify proxy hosts unaffected **`tests/access-lists/access-lists-rules.spec.ts`** Test Scenarios: - ✅ Add single IP address - Enter `192.168.1.100` - Add optional description - Verify appears in rules list - ✅ Add CIDR range - Enter `10.0.0.0/24` - Verify covers 256 IPs - Display IP count badge - ✅ Add multiple rules - Add 5+ IP rules - Verify all rules displayed - Support pagination/scrolling - ✅ Remove individual rule - Click delete on specific rule - Verify removed from list - Other rules unaffected - ✅ RFC1918 Local Network Only - Toggle "Local Network Only" switch - Verify IP rules section hidden - Description shows "RFC1918 ranges only" - ✅ Invalid IP validation - Enter invalid IP (e.g., `999.999.999.999`) - Verify error message displayed - Form not submitted - ✅ Invalid CIDR validation - Enter invalid CIDR (e.g., `192.168.1.0/99`) - Verify error message displayed - ✅ Get My IP feature - Click "Get My IP" button - Verify current IP populated in field - Toast shows IP source **`tests/access-lists/access-lists-geo.spec.ts`** Test Scenarios: - ✅ Select single country - Click country in dropdown - Country badge appears - ✅ Select multiple countries - Add US, CA, GB - All badges displayed - Deselect removes badge - ✅ Country search/filter - Type "united" in search - Shows United States, United Kingdom, UAE - Select from filtered list - ✅ Geo-whitelist vs geo-blacklist behavior - Whitelist: only selected countries allowed - Blacklist: selected countries blocked **`tests/access-lists/access-lists-presets.spec.ts`** Test Scenarios: - ✅ Show security presets (blacklist only) - Presets section hidden for whitelists - Presets section visible for blacklists - ✅ Apply "Known Malicious Actors" preset - Click "Apply" on preset - IP rules populated - Toast shows rules added count - ✅ Apply "High-Risk Countries" preset - Apply geo-blacklist preset - Countries auto-selected - Can add additional countries - ✅ Preset warning displayed - Shows data source and update frequency - Warning for aggressive presets **`tests/access-lists/access-lists-test-ip.spec.ts`** Test Scenarios: - ✅ Open Test IP dialog - Click test tube icon on ACL row - Dialog opens with IP input - ✅ Test allowed IP - Enter IP that should be allowed - Click "Test" - Success toast: "✅ IP Allowed: [reason]" - ✅ Test blocked IP - Enter IP that should be blocked - Click "Test" - Error toast: "🚫 IP Blocked: [reason]" - ✅ Invalid IP test - Enter invalid IP - Error toast displayed - ✅ Test RFC1918 detection - Test with private IP (192.168.x.x) - Verify local network detection - ✅ Test IPv6 address - Enter IPv6 address - Verify correct allow/block decision **`tests/access-lists/access-lists-integration.spec.ts`** Test Scenarios: - ✅ Assign ACL to proxy host - Edit proxy host - Select ACL from dropdown - Save and verify assignment - ✅ ACL selector shows only enabled lists - Disabled ACLs hidden from dropdown - Enabled ACLs visible with type badge - ✅ Bulk update ACL on multiple hosts - Select multiple hosts - Click "Update ACL" bulk action - Select ACL from modal - Verify all hosts updated - ✅ Remove ACL from proxy host - Select "No Access Control (Public)" - Verify ACL unassigned - ✅ Delete ACL in use - Attempt delete of assigned ACL - Warning shows affected hosts - Confirm or cancel **Key UI Selectors:** ```typescript // AccessLists.tsx page selectors 'button >> text=Create Access List' // Create button '[role="table"]' // ACL list table '[role="row"]' // Individual ACL rows 'button >> text=Edit' // Edit action (row) 'button >> text=Delete' // Delete action (row) 'button[title*="Test IP"]' // Test IP button (TestTube2 icon) // AccessListForm.tsx selectors 'input#name' // Name input 'textarea#description' // Description input 'select#type' // Type dropdown (whitelist/blacklist/geo) '[data-state="checked"]' // Enabled toggle (checked) 'button >> text=Get My IP' // Get current IP 'button >> text=Add' // Add IP rule 'input[placeholder*="192.168"]' // IP input field // AccessListSelector.tsx selectors 'select >> text=Access Control List' // ACL selector in ProxyHostForm 'option >> text=No Access Control' // Public option ``` **API Endpoints:** ```typescript // Access Lists CRUD GET /api/v1/access-lists // List all GET /api/v1/access-lists/:id // Get single POST /api/v1/access-lists // Create PUT /api/v1/access-lists/:id // Update DELETE /api/v1/access-lists/:id // Delete POST /api/v1/access-lists/:id/test // Test IP against ACL GET /api/v1/access-lists/templates // Get presets // Proxy Host ACL Integration PUT /api/v1/proxy-hosts/bulk-update-acl // Bulk ACL update ``` **Critical Assertions:** - ACL appears in list after creation - IP rules correctly parsed and displayed - Type badges match ACL configuration - Test IP returns accurate allow/block decisions - ACL assignment persists on proxy hosts - Validation prevents invalid CIDR/IP input - Security presets apply correctly --- ## Phase 2 Implementation Plan (Detailed) **Timeline:** Week 4-5 (2 weeks) **Total Tests Estimated:** 95-105 tests **Based on Phase 1 Velocity:** 112 tests in ~1 week = ~16 tests/day ### Week 4: Proxy Hosts & Access Lists (Days 1-5) #### Day 1-2: Proxy Hosts CRUD (30-35 tests) **File: `tests/proxy/proxy-hosts-crud.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | displays empty state when no hosts exist | `[data-testid="empty-state"]`, `text=Add Proxy Host` | `GET /proxy-hosts` | P0 | | 2 | shows skeleton loading while fetching | `[data-testid="skeleton-table"]` | `GET /proxy-hosts` | P1 | | 3 | lists all proxy hosts in table | `role=table`, `role=row` | `GET /proxy-hosts` | P0 | | 4 | displays host details (domain, forward, ssl) | `role=cell` | - | P0 | | 5 | opens create form when Add clicked | `button >> text=Add Proxy Host`, `role=dialog` | - | P0 | | 6 | creates basic HTTP proxy host | `#proxy-name`, `#domain-names`, `#forward-host`, `#forward-port` | `POST /proxy-hosts` | P0 | | 7 | creates HTTPS proxy host with SSL | `[name="ssl_forced"]` | `POST /proxy-hosts` | P0 | | 8 | creates proxy with WebSocket support | `[name="allow_websocket_upgrade"]` | `POST /proxy-hosts` | P1 | | 9 | creates proxy with HTTP/2 support | `[name="http2_support"]` | `POST /proxy-hosts` | P1 | | 10 | shows Docker containers in dropdown | `button >> text=Docker Discovery` | `GET /docker/containers` | P1 | | 11 | auto-fills from Docker container | Docker container option | - | P1 | | 12 | validates empty domain name | `#domain-names:invalid` | - | P0 | | 13 | validates invalid domain format | Error toast | - | P0 | | 14 | validates empty forward host | `#forward-host:invalid` | - | P0 | | 15 | validates invalid forward port | `#forward-port:invalid` | - | P0 | | 16 | validates port out of range (0, 65536) | Error message | - | P1 | | 17 | rejects XSS in domain name | 422 response | `POST /proxy-hosts` | P0 | | 18 | rejects SQL injection in fields | 422 response | `POST /proxy-hosts` | P0 | | 19 | opens edit form for existing host | `button[aria-label="Edit"]` | `GET /proxy-hosts/:uuid` | P0 | | 20 | updates domain name | Form submission | `PUT /proxy-hosts/:uuid` | P0 | | 21 | updates forward host and port | Form submission | `PUT /proxy-hosts/:uuid` | P0 | | 22 | toggles host enabled/disabled | `role=switch` | `PUT /proxy-hosts/:uuid` | P0 | | 23 | assigns SSL certificate | Certificate selector | `PUT /proxy-hosts/:uuid` | P1 | | 24 | assigns access list | ACL selector | `PUT /proxy-hosts/:uuid` | P1 | | 25 | shows delete confirmation dialog | `role=alertdialog` | - | P0 | | 26 | deletes single host | Confirm button | `DELETE /proxy-hosts/:uuid` | P0 | | 27 | cancels delete operation | Cancel button | - | P1 | | 28 | shows success toast after CRUD | `role=alert` | - | P0 | | 29 | shows error toast on failure | `role=alert[data-type="error"]` | - | P0 | | 30 | navigates back to list after save | URL check | - | P1 | **File: `tests/proxy/proxy-hosts-bulk.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 31 | selects single host via checkbox | `role=checkbox` | - | P0 | | 32 | selects all hosts via header checkbox | Header checkbox | - | P0 | | 33 | shows bulk actions when selected | Bulk action buttons | - | P0 | | 34 | bulk updates ACL on multiple hosts | `button >> text=Update ACL` | `PUT /proxy-hosts/bulk-update-acl` | P0 | | 35 | bulk deletes multiple hosts | `button >> text=Delete` | Multiple `DELETE` | P1 | | 36 | bulk updates security headers | Security headers modal | `PUT /proxy-hosts/bulk-update-security-headers` | P1 | | 37 | clears selection after bulk action | Checkbox states | - | P1 | **File: `tests/proxy/proxy-hosts-search-filter.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 38 | filters hosts by domain search | Search input | - | P1 | | 39 | filters by enabled/disabled status | Status filter | - | P1 | | 40 | filters by SSL status | SSL filter | - | P2 | | 41 | sorts by domain name | Column header click | - | P2 | | 42 | sorts by creation date | Column header click | - | P2 | | 43 | paginates large host lists | Pagination controls | `GET /proxy-hosts?page=2` | P2 | #### Day 3: Access Lists CRUD (20-25 tests) **File: `tests/access-lists/access-lists-crud.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | displays empty state when no ACLs | `[data-testid="empty-state"]` | `GET /access-lists` | P0 | | 2 | lists all access lists in table | `role=table` | `GET /access-lists` | P0 | | 3 | shows ACL type badge (Allow/Deny) | `Badge[variant="success"]` | - | P0 | | 4 | creates IP whitelist | `select#type`, `option[value="whitelist"]` | `POST /access-lists` | P0 | | 5 | creates IP blacklist | `option[value="blacklist"]` | `POST /access-lists` | P0 | | 6 | creates geo-whitelist | `option[value="geo_whitelist"]` | `POST /access-lists` | P0 | | 7 | creates geo-blacklist | `option[value="geo_blacklist"]` | `POST /access-lists` | P0 | | 8 | validates empty name | `input#name:invalid` | - | P0 | | 9 | adds single IP rule | IP input, Add button | - | P0 | | 10 | adds CIDR range rule | `10.0.0.0/24` input | - | P0 | | 11 | shows IP count for CIDR | IP count badge | - | P1 | | 12 | removes IP rule | Delete button on rule | - | P0 | | 13 | validates invalid CIDR | Error message | - | P0 | | 14 | enables RFC1918 local network only | Toggle switch | - | P1 | | 15 | Get My IP populates field | `button >> text=Get My IP` | `GET /system/my-ip` | P1 | | 16 | edits existing ACL | Edit button, form | `PUT /access-lists/:id` | P0 | | 17 | deletes ACL with backup | Delete, confirm | `DELETE /access-lists/:id` | P0 | | 18 | toggles ACL enabled/disabled | Enable switch | `PUT /access-lists/:id` | P0 | | 19 | shows CGNAT warning on first load | Alert component | - | P2 | | 20 | dismisses CGNAT warning | Dismiss button | - | P2 | **File: `tests/access-lists/access-lists-geo.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 21 | selects country from list | Country dropdown | - | P0 | | 22 | adds multiple countries | Country badges | - | P0 | | 23 | removes country | Badge X button | - | P0 | | 24 | shows all 40+ countries | Country list | - | P1 | **File: `tests/access-lists/access-lists-test.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 25 | opens Test IP dialog | TestTube2 icon button | - | P0 | | 26 | tests allowed IP shows success | Success toast | `POST /access-lists/:id/test` | P0 | | 27 | tests blocked IP shows error | Error toast | `POST /access-lists/:id/test` | P0 | | 28 | validates invalid IP input | Error message | - | P1 | #### Day 4-5: Access Lists Integration & Presets (10-15 tests) **File: `tests/access-lists/access-lists-presets.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | shows presets section for blacklist | Presets toggle | - | P1 | | 2 | hides presets for whitelist | - | - | P1 | | 3 | applies security preset | Apply button | - | P1 | | 4 | shows preset warning | Warning icon | - | P2 | | 5 | shows data source link | External link | - | P2 | **File: `tests/access-lists/access-lists-integration.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 6 | assigns ACL to proxy host | ACL selector | `PUT /proxy-hosts/:uuid` | P0 | | 7 | shows only enabled ACLs in selector | Dropdown options | `GET /access-lists` | P0 | | 8 | bulk assigns ACL to hosts | Bulk ACL modal | `PUT /proxy-hosts/bulk-update-acl` | P0 | | 9 | removes ACL from proxy host | "No Access Control" | `PUT /proxy-hosts/:uuid` | P0 | | 10 | warns when deleting ACL in use | Warning dialog | - | P1 | ### Week 5: SSL Certificates (Days 6-10) #### Day 6-7: Certificate List & Upload (25-30 tests) **File: `tests/certificates/certificates-list.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | displays empty state when no certs | Empty state | `GET /certificates` | P0 | | 2 | lists all certificates | Table rows | `GET /certificates` | P0 | | 3 | shows certificate details | Name, domain, expiry | - | P0 | | 4 | shows status badge (valid) | `Badge[variant="success"]` | - | P0 | | 5 | shows status badge (expiring) | `Badge[variant="warning"]` | - | P0 | | 6 | shows status badge (expired) | `Badge[variant="error"]` | - | P0 | | 7 | sorts by name column | Header click | - | P1 | | 8 | sorts by expiry date | Header click | - | P1 | | 9 | shows associated proxy hosts | Host count/badges | - | P2 | **File: `tests/certificates/certificates-upload.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 10 | opens upload modal | `button >> text=Add Certificate` | - | P0 | | 11 | uploads valid cert and key | File inputs | `POST /certificates` (multipart) | P0 | | 12 | validates PEM format | Error on invalid | - | P0 | | 13 | rejects mismatched cert/key | Error toast | - | P0 | | 14 | rejects expired certificate | Error toast | - | P1 | | 15 | shows upload progress | Progress indicator | - | P2 | | 16 | closes modal after success | Modal hidden | - | P1 | | 17 | shows success toast | `role=alert` | - | P0 | | 18 | deletes certificate | Delete button | `DELETE /certificates/:id` | P0 | | 19 | shows delete confirmation | Confirm dialog | - | P0 | | 20 | creates backup before delete | Backup API | `POST /backups` | P1 | **File: `tests/certificates/certificates-validation.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 21 | rejects empty name | Validation error | - | P0 | | 22 | rejects missing cert file | Required error | - | P0 | | 23 | rejects missing key file | Required error | - | P0 | | 24 | rejects self-signed (if configured) | Warning/Error | - | P2 | | 25 | handles network error gracefully | Error toast | - | P1 | #### Day 8-9: ACME Certificates (15-20 tests) **File: `tests/certificates/certificates-acme.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | shows ACME certificate info | Let's Encrypt badge | - | P0 | | 2 | displays HTTP-01 challenge type | Challenge type indicator | - | P1 | | 3 | displays DNS-01 challenge type | Challenge type indicator | - | P1 | | 4 | shows certificate renewal date | Expiry countdown | - | P0 | | 5 | shows "Renew Now" for expiring | Renew button visible | - | P1 | | 6 | hides "Renew Now" for valid | Renew button hidden | - | P1 | | 7 | displays wildcard indicator | Wildcard badge | - | P1 | | 8 | shows SAN (multiple domains) | Domain list | - | P2 | **Note:** Full ACME flow testing requires mocked ACME server (staging.letsencrypt.org) - these tests verify UI behavior with pre-existing ACME certificates. **File: `tests/certificates/certificates-status.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 9 | dashboard shows certificate stats | CertificateStatusCard | - | P1 | | 10 | shows valid certificate count | Valid count badge | - | P1 | | 11 | shows expiring certificate count | Warning count | - | P1 | | 12 | shows pending certificate count | Pending count | - | P2 | | 13 | links to certificates page | Card link | - | P2 | | 14 | progress bar shows coverage | Progress component | - | P2 | #### Day 10: Certificate Integration & Cleanup (10-15 tests) **File: `tests/certificates/certificates-integration.spec.ts`** | # | Test Name | UI Selectors | API Endpoint | Priority | |---|-----------|-------------|--------------|----------| | 1 | assigns certificate to proxy host | Certificate selector | `PUT /proxy-hosts/:uuid` | P0 | | 2 | shows only valid certs in selector | Dropdown filtered | `GET /certificates` | P0 | | 3 | certificate cleanup dialog on host delete | CertificateCleanupDialog | - | P0 | | 4 | deletes orphan certs option | Checkbox in dialog | - | P1 | | 5 | keeps certs option | Default unchecked | - | P1 | | 6 | shows affected hosts on cert delete | Host list | - | P1 | | 7 | warns about hosts using certificate | Warning message | - | P1 | --- ### Fixtures Reference **Proxy Hosts:** `tests/fixtures/proxy-hosts.ts` - `basicProxyHost` - HTTP proxy to internal service - `proxyHostWithSSL` - HTTPS with forced SSL - `proxyHostWithWebSocket` - WebSocket enabled - `proxyHostFullSecurity` - All security features - `wildcardProxyHost` - Wildcard domain - `dockerProxyHost` - From Docker discovery - `invalidProxyHosts` - Validation test cases (XSS, SQL injection) **Access Lists:** `tests/fixtures/access-lists.ts` - `emptyAccessList` - No rules - `allowOnlyAccessList` - IP whitelist - `denyOnlyAccessList` - IP blacklist - `mixedRulesAccessList` - Multiple IP ranges - `authEnabledAccessList` - With HTTP basic auth - `ipv6AccessList` - IPv6 ranges - `invalidACLConfigs` - Validation test cases **Certificates:** `tests/fixtures/certificates.ts` - `letsEncryptCertificate` - HTTP-01 ACME - `multiDomainLetsEncrypt` - SAN certificate - `wildcardCertificate` - DNS-01 wildcard - `customCertificateMock` - Uploaded PEM - `expiredCertificate` - For error testing - `expiringCertificate` - 25 days to expiry - `invalidCertificates` - Validation test cases --- ### Acceptance Criteria for Phase 2 **Proxy Hosts (40 tests minimum):** - [ ] All CRUD operations covered - [ ] Bulk operations functional - [ ] Docker discovery integration works - [ ] Validation prevents all invalid input - [ ] XSS/SQL injection rejected **SSL Certificates (30 tests minimum):** - [ ] List/upload/delete operations covered - [ ] PEM validation enforced - [ ] Certificate status displayed correctly - [ ] Dashboard stats accurate - [ ] Cleanup dialog handles orphan certs **Access Lists (25 tests minimum):** - [ ] All 4 ACL types covered (IP/Geo × Allow/Block) - [ ] IP/CIDR rule management works - [ ] Country selection works - [ ] Test IP feature functional - [ ] Integration with proxy hosts works **Overall:** - [ ] 95+ tests passing - [ ] <5% flaky test rate - [ ] All P0 tests complete - [ ] 90%+ P1 tests complete - [ ] No hardcoded waits - [ ] All tests use TestDataManager for cleanup --- ### Phase 3: Security Features (Week 6-7) **Goal:** Cover all Cerberus security features #### 3.1 CrowdSec Integration **Priority:** High **Estimated Effort:** 4 days **Test Files:** **`tests/security/crowdsec-startup.spec.ts`** **Objective:** Verify CrowdSec container lifecycle and connectivity Test Scenarios: - ✅ CrowdSec container starts successfully - Verify container health check passes - Verify LAPI (Local API) is accessible - Verify logs show successful initialization - ✅ CrowdSec LAPI connection from Charon - Backend connects to CrowdSec LAPI - Authentication succeeds - API health endpoint returns 200 - ✅ CrowdSec bouncer registration - Charon registers as a bouncer - Bouncer API key generated - Bouncer appears in CrowdSec bouncer list - ✅ CrowdSec graceful shutdown - Stop CrowdSec container - Charon handles disconnection gracefully - No errors in Charon logs - Restart CrowdSec, Charon reconnects - ✅ CrowdSec container restart recovery - Kill CrowdSec container abruptly - Charon detects connection loss - Auto-reconnect after CrowdSec restarts - ✅ CrowdSec version compatibility - Verify minimum version check - Warn if CrowdSec version too old - Block connection if incompatible **Key Assertions:** - Container health: `docker inspect crowdsec --format '{{.State.Health.Status}}' == 'healthy'` - LAPI reachable: `curl http://crowdsec:8080/health` returns 200 - Bouncer registered: API call to `/v1/bouncers` shows Charon - Logs clean: No error/warning logs after startup **`tests/security/crowdsec-decisions.spec.ts`** **Objective:** Test IP ban management and decision enforcement Test Scenarios: **Manual IP Ban:** - ✅ Add IP ban via Charon UI - Navigate to CrowdSec → Decisions - Click "Add Decision" - Enter IP address (e.g., `192.168.1.100`) - Select ban duration (1h, 4h, 24h, permanent) - Select scope (IP, range, country) - Add ban reason (e.g., "Suspicious activity") - Save decision - ✅ Verify decision appears in CrowdSec - Query CrowdSec LAPI `/v1/decisions` - Verify decision contains correct IP, duration, reason - ✅ Banned IP cannot access proxy hosts - Make HTTP request from banned IP - Verify 403 Forbidden response - Verify block logged in audit log **Automatic IP Ban (Scenario-based):** - ✅ Trigger ban via brute force scenario - Simulate 10 failed login attempts from IP - CrowdSec scenario detects pattern - Decision created automatically - IP banned for configured duration - ✅ Trigger ban via HTTP flood scenario - Send 100 requests/second from IP - CrowdSec detects flood pattern - IP banned automatically **Ban Duration & Expiration:** - ✅ Temporary ban expires automatically - Create 5-second ban - Verify IP blocked immediately - Wait 6 seconds - Verify IP can access again - ✅ Permanent ban persists - Create permanent ban - Restart Charon and CrowdSec - Verify ban still active - ✅ Manual ban removal - Select active ban - Click "Remove Decision" - Confirm removal - Verify IP can access immediately **Ban Scope Testing:** - ✅ Single IP ban: `192.168.1.100` - Only that IP blocked - `192.168.1.101` can access - ✅ IP range ban: `192.168.1.0/24` - All IPs in range blocked - `192.168.2.1` can access - ✅ Country-level ban: `CN` (China) - All Chinese IPs blocked - Uses GeoIP database - Other countries can access **Decision Priority & Conflicts:** - ✅ Allow decision overrides ban decision - Ban IP range `10.0.0.0/8` - Allow specific IP `10.0.0.5` - Verify `10.0.0.5` can access - Verify `10.0.0.6` is blocked - ✅ Multiple decisions for same IP - Add 1-hour ban - Add 24-hour ban - Longer duration takes precedence **Decision Metadata:** - ✅ Display decision details - IP/Range/Country - Ban duration and expiry time - Scenario that triggered ban - Reason/origin - Creation timestamp - ✅ Decision history - View past decisions (expired/deleted) - Filter by IP, scenario, date range - Export decision history **Key User Flows:** 1. **Manual Ban Flow:** ``` CrowdSec page → Decisions tab → Add Decision → Enter IP → Select duration → Save → Verify block ``` 2. **Automatic Ban Flow:** ``` Trigger scenario (e.g., brute force) → CrowdSec detects → Decision created → View in Decisions tab → Verify block ``` 3. **Ban Removal Flow:** ``` CrowdSec page → Decisions tab → Select decision → Remove → Confirm → Verify access restored ``` **Critical Assertions:** - Banned IPs receive 403 status code - Decision sync between Charon and CrowdSec - Expiration timing accurate (±5 seconds) - Allow decisions override ban decisions - Decision changes appear in audit log **`tests/security/crowdsec-presets.spec.ts`** **Objective:** Test CrowdSec scenario preset management Test Scenarios: **Preset Listing:** - ✅ View available presets - Navigate to CrowdSec → Presets - Display preset categories (Web, SSH, System) - Show preset descriptions - Indicate enabled/disabled status **Enable/Disable Presets:** - ✅ Enable web attack preset - Select "Web Attacks" preset - Click "Enable" - Verify scenarios installed in CrowdSec - Verify collection appears in CrowdSec collections list - ✅ Disable web attack preset - Select enabled preset - Click "Disable" - Verify scenarios removed - Existing decisions preserved - ✅ Bulk enable multiple presets - Select multiple presets - Click "Enable Selected" - All scenarios installed **Custom Scenarios:** - ✅ Create custom scenario - Click "Add Custom Scenario" - Enter scenario name (e.g., "api-abuse") - Define pattern (e.g., 50 requests to /api in 10s) - Set ban duration (e.g., 1h) - Save scenario - Verify scenario YAML created - ✅ Test custom scenario - Trigger scenario conditions - Verify decision created - Verify ban enforced - ✅ Edit custom scenario - Modify pattern thresholds - Save changes - Reload CrowdSec scenarios - ✅ Delete custom scenario - Select scenario - Confirm deletion - Scenario removed from CrowdSec **Preset Configuration:** - ✅ Configure scenario thresholds - Select scenario (e.g., "http-bf" brute force) - Modify threshold (e.g., 5 → 10 failed attempts) - Modify time window (e.g., 30s → 60s) - Save configuration - Verify new thresholds apply - ✅ Configure ban duration per scenario - Different scenarios have different ban times - Brute force: 4h ban - Port scan: 24h ban - Verify durations respected **Scenario Testing & Validation:** - ✅ Test scenario before enabling - View scenario details - Simulate trigger conditions in test mode - Verify pattern matching works - No actual bans created (dry-run) - ✅ Scenario validation on save - Invalid regex pattern rejected - Impossible thresholds rejected (e.g., 0 requests) - Missing required fields flagged **Key User Flows:** 1. **Enable Preset Flow:** ``` CrowdSec page → Presets tab → Select preset → Enable → Verify scenarios active ``` 2. **Custom Scenario Flow:** ``` CrowdSec page → Presets tab → Add Custom → Define pattern → Set duration → Save → Test trigger ``` 3. **Configure Scenario Flow:** ``` CrowdSec page → Presets tab → Select scenario → Edit thresholds → Save → Reload CrowdSec ``` **Critical Assertions:** - Enabled scenarios appear in CrowdSec collections - Scenario triggers create correct decisions - Custom scenarios persist after restart - Threshold changes take effect immediately - Invalid scenarios rejected with clear error messages #### 3.2 Coraza WAF (Web Application Firewall) **Priority:** High **Estimated Effort:** 3 days **Test Files:** **`tests/security/waf-config.spec.ts`** Test Scenarios: - ✅ Enable/disable WAF globally - ✅ Configure WAF for specific proxy host - ✅ Set WAF rule sets (OWASP Core Rule Set) - ✅ Configure anomaly scoring thresholds - ✅ Set blocking/logging mode - ✅ Custom rule creation - ✅ Rule exclusions (false positive handling) **`tests/security/waf-blocking.spec.ts`** Test Scenarios: - ✅ Block SQL injection attempts - Send request with `' OR 1=1--` in query - Verify 403 response - Verify attack logged - ✅ Block XSS attempts - Send request with `` - Verify 403 response - ✅ Block path traversal attempts - Send request with `../../etc/passwd` - Verify 403 response - ✅ Block command injection attempts - ✅ Block file upload attacks - ✅ Allow legitimate requests - Normal user traffic passes through - No false positives **Key Assertions:** - Malicious requests blocked (403 status) - Legitimate requests allowed (200/3xx status) - Attacks logged in audit log with details - WAF performance overhead <50ms #### 3.3 Rate Limiting **Priority:** High **Estimated Effort:** 2 days ### Phase 4: Settings (Week 8) **Goal:** Cover system configuration and user management **Estimated Effort:** 5 days **Test Files:** - `tests/settings/system-settings.spec.ts` - System configuration - `tests/settings/smtp-settings.spec.ts` - Email configuration - `tests/settings/notifications.spec.ts` - Notification rules - `tests/settings/user-management.spec.ts` - User CRUD and roles - `tests/settings/encryption-management.spec.ts` - Encryption key rotation - `tests/settings/account-settings.spec.ts` - User profile management **Key Features:** - System configuration (timezone, language, theme) - Email settings (SMTP, templates) - Notification rules (email, webhook) - User management (CRUD, roles, permissions) - Encryption management (key rotation, backup) - Account settings (profile, password, 2FA) ### Phase 5: Tasks (Week 9) **Goal:** Cover backup, logs, and monitoring features **Estimated Effort:** 5 days **Test Files:** - `tests/tasks/backups-create.spec.ts` - Backup creation - `tests/tasks/backups-restore.spec.ts` - Backup restoration - `tests/tasks/logs-viewing.spec.ts` - Log viewer functionality - `tests/tasks/import-caddyfile.spec.ts` - Caddyfile import - `tests/tasks/import-crowdsec.spec.ts` - CrowdSec config import - `tests/monitoring/uptime-monitoring.spec.ts` - Uptime checks - `tests/monitoring/real-time-logs.spec.ts` - WebSocket log streaming **Key Features:** - Backup creation (manual, scheduled) - Backup restoration (full, selective) - Log viewing (filtering, search, export) - Caddyfile import (validation, migration) - CrowdSec import (scenarios, decisions) - Uptime monitoring (HTTP checks, alerts) - Real-time logs (WebSocket, filtering) ### Phase 6: Integration & Buffer (Week 10) **Goal:** Test cross-feature interactions, edge cases, and provide buffer for overruns **Estimated Effort:** 5 days (3 days testing + 2 days buffer) **Test Files:** - `tests/integration/proxy-acl-integration.spec.ts` - Proxy + ACL - `tests/integration/proxy-certificate.spec.ts` - Proxy + SSL - `tests/integration/security-suite-integration.spec.ts` - Full security stack - `tests/integration/backup-restore-e2e.spec.ts` - Full backup cycle **Key Scenarios:** - Create proxy host with ACL and SSL certificate - Test security stack: WAF + CrowdSec + Rate Limiting - Full backup → Restore → Verify all data intact - Multi-feature workflows (e.g., import Caddyfile + enable security) **Buffer Time:** - Address flaky tests discovered in previous phases - Fix any infrastructure issues - Improve test stability and reliability - Documentation updates --- ## Success Metrics & Acceptance Criteria ### Coverage Goals **Test Coverage Targets:** - 🎯 **Core Features:** 100% coverage (auth, navigation, dashboard) - 🎯 **Critical Features:** 100% coverage (proxy hosts, ACLs, certificates) - 🎯 **High Priority Features:** 90% coverage (security suite, backups) - 🎯 **Medium Priority Features:** 80% coverage (settings, monitoring) - 🎯 **Nice-to-Have Features:** 70% coverage (imports, plugins) **Feature Coverage Matrix:** | Feature | Priority | Target Coverage | Test Files | Status | |---------|----------|----------------|------------|--------| | Authentication | Critical | 100% | 1 | ✅ Covered (94% - 3 session tests deferred) | | Dashboard | Core | 100% | 1 | ✅ Covered | | Navigation | Core | 100% | 1 | ✅ Covered | | Proxy Hosts | Critical | 100% | 3 | ❌ Not started | | Certificates | Critical | 100% | 3 | ❌ Not started | | Access Lists | Critical | 100% | 2 | ❌ Not started | | CrowdSec | High | 90% | 3 | ❌ Not started | | WAF | High | 90% | 1 | ❌ Not started | | Rate Limiting | High | 90% | 1 | ❌ Not started | | Security Headers | Medium | 80% | 1 | ❌ Not started | | Audit Logs | Medium | 80% | 1 | ❌ Not started | | Backups | High | 90% | 2 | ❌ Not started | | Users | High | 90% | 2 | ❌ Not started | | Settings | Medium | 80% | 4 | ❌ Not started | | Monitoring | Medium | 80% | 3 | ❌ Not started | | Import/Export | Medium | 80% | 2 | ❌ Not started | | DNS Providers | Critical | 100% | 4 | ✅ Covered | --- ## Next Steps 1. ~~**Review and Approve Plan:** Stakeholder sign-off~~ ✅ 2. ~~**Set Up Test Infrastructure:** Fixtures, utilities, CI configuration~~ ✅ 3. ~~**Begin Phase 1 Implementation:** Foundation tests~~ ✅ 4. **Begin Phase 2 Implementation:** Critical Path (Proxy Hosts, Certificates, ACLs) 5. **Fix Session Expiration Tests:** See [docs/issues/e2e-session-expiration-tests.md](../issues/e2e-session-expiration-tests.md) 6. **Daily Standup Check-ins:** Progress tracking, blocker resolution 7. **Weekly Demo:** Show completed test coverage --- **Document Status:** In Progress - Phase 1 Complete **Last Updated:** January 17, 2026 **Phase 1 Completed:** January 17, 2026 (112/119 tests passing - 94%) **Next Review:** Upon Phase 2 completion (estimated Jan 31, 2026) **Owner:** Planning Agent / QA Team