Files
Charon/docs/plans/current_spec.md
GitHub Actions 4cecbea8db chore: add Phase 3 Security Features E2E tests (121 new tests)
Implement comprehensive Playwright E2E test coverage for Security Features:

security-dashboard.spec.ts: Module toggles, status indicators, navigation
crowdsec-config.spec.ts: Presets, config files, console enrollment
crowdsec-decisions.spec.ts: Decisions/bans management (skipped - no route)
waf-config.spec.ts: WAF mode toggle, rulesets, threshold settings
rate-limiting.spec.ts: RPS, burst, time window configuration
security-headers.spec.ts: Presets, individual headers, score display
audit-logs.spec.ts: Data table, filtering, export CSV, pagination
Bug fixes applied:

Fixed toggle selectors (checkbox instead of switch role)
Fixed card navigation selectors for Security page
Fixed rate-limiting route URL (/rate-limiting not /rate-limit)
Added proper loading state handling for audit-logs tests
Test results: 346 passed, 1 pre-existing flaky, 25 skipped (99.7%)

Part of E2E Testing Plan Phase 3 (Week 6-7)
2026-01-20 06:11:59 +00:00

2841 lines
92 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Charon E2E Testing Plan: Comprehensive Playwright Coverage
**Date:** January 18, 2026
**Status:** Phase 3 Complete - 346+ tests passing
**Priority:** Critical - Blocking new feature development
**Objective:** Establish comprehensive E2E test coverage for all existing Charon features
**Timeline:** 10 weeks (with proper infrastructure setup and comprehensive feature coverage)
> **Revision Note:** This document has been completely revised to address critical infrastructure gaps, expand underspecified sections, and provide implementation-ready specifications. Major additions include test data management, authentication strategy, CI/CD integration, flaky test prevention, and detailed security feature testing.
---
## Table of Contents
1. [Current State & Coverage Gaps](#1-current-state--coverage-gaps)
2. [Testing Infrastructure](#2-testing-infrastructure)
- 2.1 [Test Environment Setup](#21-test-environment-setup)
- 2.2 [Test Data Management Strategy](#22-test-data-management-strategy)
- 2.3 [Authentication Strategy](#23-authentication-strategy)
- 2.4 [Flaky Test Prevention](#24-flaky-test-prevention)
- 2.5 [CI/CD Integration](#25-cicd-integration)
3. [Test Organization](#3-test-organization)
4. [Implementation Plan](#4-implementation-plan)
- Phase 0: Infrastructure Setup (Week 1-2)
- Phase 1: Foundation (Week 3)
- Phase 2: Critical Path (Week 4-5)
- Phase 3: Security Features (Week 6-7)
- Phase 4: Settings (Week 8)
- Phase 5: Tasks (Week 9)
- Phase 6: Integration (Week 10)
5. [Security Feature Testing Strategy](#5-security-feature-testing-strategy)
6. [Risk Mitigation](#6-risk-mitigation)
7. [Success Metrics](#7-success-metrics)
8. [Next Steps](#8-next-steps)
---
## 1. Current State & Coverage Gaps
### Existing Test Files
**Current E2E Test Coverage:**
-`tests/auth.setup.ts` - Authentication setup (shared fixture)
-`tests/manual-dns-provider.spec.ts` - Manual DNS provider E2E tests (comprehensive)
-`tests/dns-provider-crud.spec.ts` - DNS provider CRUD operations
-`tests/dns-provider-types.spec.ts` - DNS provider type validation
-`tests/example.spec.js` - Legacy example (can be removed)
-`tests/fixtures/dns-providers.ts` - Shared DNS test fixtures
**Critical Infrastructure Gaps Identified:**
- ❌ No test data management system (causes data conflicts, FK violations)
- ❌ No per-test user creation (shared auth state breaks parallel execution)
- ❌ No CI/CD integration strategy (no automated testing on PRs)
- ❌ No flaky test prevention utilities (arbitrary timeouts everywhere)
- ❌ No environment setup documentation (manual setup, no verification)
- ❌ No mock external service strategy (tests depend on real services)
### Coverage Gaps
**All major features lack E2E test coverage except DNS providers:**
- ❌ Proxy Hosts management
- ❌ Access Lists (ACL)
- ❌ SSL Certificates
- ❌ CrowdSec integration
- ❌ Coraza WAF
- ❌ Rate Limiting
- ❌ Security Headers
- ❌ Backups & Restore
- ❌ User Management
- ❌ System Settings
- ❌ Audit Logs
- ❌ Remote Servers
- ❌ Uptime Monitoring
- ❌ Notifications
- ❌ Import/Export features
- ❌ Encryption Management
---
## 2. Testing Infrastructure
### 2.1 Test Environment Setup
**Objective:** Ensure consistent, reproducible test environments for local development and CI.
#### 2.1.1 Local Development Setup
**Prerequisites:**
- Docker and Docker Compose installed
- Node.js 18+ and npm
- Go 1.21+ (for backend development)
- Playwright browsers installed (`npx playwright install`)
**Environment Configuration:**
```bash
# .env.test (create in project root)
NODE_ENV=test
DATABASE_URL=sqlite:./data/charon_test.db
BASE_URL=http://localhost:8080
PLAYWRIGHT_BASE_URL=http://localhost:8080
TEST_USER_EMAIL=test-admin@charon.local
TEST_USER_PASSWORD=TestPassword123!
DOCKER_HOST=unix:///var/run/docker.sock
ENABLE_CROWDSEC=false # Disabled for unit tests, enabled for integration
ENABLE_WAF=false
LOG_LEVEL=warn
```
**Required Docker Services:**
> **Note:** Use the committed `docker-compose.playwright.yml` for E2E testing.
> The `docker-compose.test.yml` is gitignored and reserved for personal/local configurations.
```yaml
# .docker/compose/docker-compose.playwright.yml
# See the actual file for the full configuration with:
# - Charon app service with test environment
# - Optional CrowdSec profile: --profile security-tests
# - Optional MailHog profile: --profile notification-tests
#
# Usage:
# docker compose -f .docker/compose/docker-compose.playwright.yml up -d
# docker compose -f .docker/compose/docker-compose.playwright.yml --profile security-tests up -d
**Setup Script:**
```bash
#!/bin/bash
# scripts/setup-e2e-env.sh
set -euo pipefail
echo "🚀 Setting up E2E test environment..."
# 1. Check prerequisites
command -v docker >/dev/null 2>&1 || { echo "❌ Docker not found"; exit 1; }
command -v node >/dev/null 2>&1 || { echo "❌ Node.js not found"; exit 1; }
# 2. Install dependencies
echo "📦 Installing dependencies..."
npm ci
# 3. Install Playwright browsers
echo "🎭 Installing Playwright browsers..."
npx playwright install chromium
# 4. Create test environment file
if [ ! -f .env.test ]; then
echo "📝 Creating .env.test..."
cp .env.example .env.test
# Set test-specific values
sed -i 's/NODE_ENV=.*/NODE_ENV=test/' .env.test
sed -i 's/DATABASE_URL=.*/DATABASE_URL=sqlite:\.\/data\/charon_test.db/' .env.test
fi
# 5. Start test environment
echo "🐳 Starting Docker services..."
docker compose -f .docker/compose/docker-compose.playwright.yml up -d
# 6. Wait for service health
echo "⏳ Waiting for service to be healthy..."
timeout 60 bash -c 'until docker compose -f .docker/compose/docker-compose.playwright.yml exec -T charon-app curl -f http://localhost:8080/api/v1/health; do sleep 2; done'
# 7. Run database migrations
echo "🗄️ Running database migrations..."
docker compose -f .docker/compose/docker-compose.playwright.yml exec -T charon-app /app/backend/charon migrate
echo "✅ E2E environment ready!"
echo "📍 Application: http://localhost:8080"
echo "🧪 Run tests: npm run test:e2e"
```
**Environment Health Check:**
```typescript
// tests/utils/health-check.ts
export async function waitForHealthyEnvironment(baseURL: string, timeout = 60000): Promise<void> {
const startTime = Date.now();
while (Date.now() - startTime < timeout) {
try {
const response = await fetch(`${baseURL}/api/v1/health`);
if (response.ok) {
const health = await response.json();
if (health.status === 'healthy' && health.database === 'connected') {
console.log('✅ Environment is healthy');
return;
}
}
} catch (error) {
// Service not ready yet
}
await new Promise(resolve => setTimeout(resolve, 2000));
}
throw new Error(`Environment not healthy after ${timeout}ms`);
}
export async function verifyTestPrerequisites(): Promise<void> {
const checks = {
'Database writable': async () => {
// Attempt to create a test record
const response = await fetch(`${process.env.PLAYWRIGHT_BASE_URL}/api/v1/test/db-check`, {
method: 'POST'
});
return response.ok;
},
'Docker socket accessible': async () => {
// Check if Docker is available (for proxy host tests)
const response = await fetch(`${process.env.PLAYWRIGHT_BASE_URL}/api/v1/test/docker-check`);
return response.ok;
}
};
for (const [name, check] of Object.entries(checks)) {
try {
const result = await check();
if (!result) throw new Error(`Check failed: ${name}`);
console.log(`${name}`);
} catch (error) {
console.error(`${name}: ${error}`);
throw new Error(`Prerequisite check failed: ${name}`);
}
}
}
```
#### 2.1.2 CI Environment Configuration
**GitHub Actions Environment:**
- Use `localhost:8080` instead of Tailscale IP
- Run services in Docker containers
- Use GitHub Actions cache for dependencies and browsers
- Upload test artifacts on failure
**Network Configuration:**
```yaml
# In CI, all services communicate via Docker network
services:
charon:
networks:
- test-network
networks:
test-network:
driver: bridge
```
#### 2.1.3 Mock External Service Strategy
**DNS Provider API Mocks:**
```typescript
// tests/mocks/dns-provider-api.ts
import { rest } from 'msw';
import { setupServer } from 'msw/node';
export const dnsProviderMocks = [
// Mock Cloudflare API
rest.post('https://api.cloudflare.com/client/v4/zones/:zoneId/dns_records', (req, res, ctx) => {
return res(
ctx.status(200),
ctx.json({
success: true,
result: { id: 'mock-record-id', name: req.body.name }
})
);
}),
// Mock Route53 API
rest.post('https://route53.amazonaws.com/*', (req, res, ctx) => {
return res(ctx.status(200), ctx.xml('<ChangeInfo><Id>mock-change-id</Id></ChangeInfo>'));
})
];
export const mockServer = setupServer(...dnsProviderMocks);
```
**ACME Server Mock (for certificate tests):**
```typescript
// tests/mocks/acme-server.ts
export const acmeMocks = [
// Mock Let's Encrypt directory
rest.get('https://acme-v02.api.letsencrypt.org/directory', (req, res, ctx) => {
return res(ctx.json({
newNonce: 'https://mock-acme/new-nonce',
newAccount: 'https://mock-acme/new-account',
newOrder: 'https://mock-acme/new-order'
}));
})
];
```
---
### 2.2 Test Data Management Strategy
**Critical Problem:** Current approach uses shared test data, causing conflicts in parallel execution and leaving orphaned records.
**Solution:** Implement `TestDataManager` utility with namespaced isolation and guaranteed cleanup.
#### 2.2.1 TestDataManager Design
```typescript
// tests/utils/TestDataManager.ts
import { APIRequestContext } from '@playwright/test';
import crypto from 'crypto';
export interface ManagedResource {
id: string;
type: 'proxy-host' | 'certificate' | 'access-list' | 'dns-provider' | 'user';
namespace: string;
createdAt: Date;
}
export class TestDataManager {
private resources: ManagedResource[] = [];
private namespace: string;
private request: APIRequestContext;
constructor(request: APIRequestContext, testName?: string) {
this.request = request;
// Create unique namespace per test to avoid conflicts
this.namespace = testName
? `test-${this.sanitize(testName)}-${Date.now()}`
: `test-${crypto.randomUUID()}`;
}
private sanitize(name: string): string {
return name.toLowerCase().replace(/[^a-z0-9]/g, '-').substring(0, 30);
}
/**
* Create a proxy host with automatic cleanup tracking
*/
async createProxyHost(data: {
domain: string;
forwardHost: string;
forwardPort: number;
scheme?: 'http' | 'https';
}): Promise<{ id: string; domain: string }> {
const namespaced = {
...data,
domain: `${this.namespace}.${data.domain}` // Ensure unique domain
};
const response = await this.request.post('/api/v1/proxy-hosts', {
data: namespaced
});
if (!response.ok()) {
throw new Error(`Failed to create proxy host: ${await response.text()}`);
}
const result = await response.json();
this.resources.push({
id: result.uuid,
type: 'proxy-host',
namespace: this.namespace,
createdAt: new Date()
});
return { id: result.uuid, domain: namespaced.domain };
}
/**
* Create an access list with automatic cleanup
*/
async createAccessList(data: {
name: string;
rules: Array<{ type: 'allow' | 'deny'; value: string }>;
}): Promise<{ id: string }> {
const namespaced = {
...data,
name: `${this.namespace}-${data.name}`
};
const response = await this.request.post('/api/v1/access-lists', {
data: namespaced
});
if (!response.ok()) {
throw new Error(`Failed to create access list: ${await response.text()}`);
}
const result = await response.json();
this.resources.push({
id: result.id,
type: 'access-list',
namespace: this.namespace,
createdAt: new Date()
});
return { id: result.id };
}
/**
* Create a certificate with automatic cleanup
*/
async createCertificate(data: {
domains: string[];
type: 'letsencrypt' | 'custom';
privateKey?: string;
certificate?: string;
}): Promise<{ id: string }> {
const namespaced = {
...data,
domains: data.domains.map(d => `${this.namespace}.${d}`)
};
const response = await this.request.post('/api/v1/certificates', {
data: namespaced
});
if (!response.ok()) {
throw new Error(`Failed to create certificate: ${await response.text()}`);
}
const result = await response.json();
this.resources.push({
id: result.id,
type: 'certificate',
namespace: this.namespace,
createdAt: new Date()
});
return { id: result.id };
}
/**
* Create a DNS provider with automatic cleanup
*/
async createDNSProvider(data: {
type: 'manual' | 'cloudflare' | 'route53';
name: string;
credentials?: Record<string, string>;
}): Promise<{ id: string }> {
const namespaced = {
...data,
name: `${this.namespace}-${data.name}`
};
const response = await this.request.post('/api/v1/dns-providers', {
data: namespaced
});
if (!response.ok()) {
throw new Error(`Failed to create DNS provider: ${await response.text()}`);
}
const result = await response.json();
this.resources.push({
id: result.id,
type: 'dns-provider',
namespace: this.namespace,
createdAt: new Date()
});
return { id: result.id };
}
/**
* Create a test user with automatic cleanup
*/
async createUser(data: {
email: string;
password: string;
role: 'admin' | 'user' | 'guest';
}): Promise<{ id: string; email: string; token: string }> {
const namespaced = {
...data,
email: `${this.namespace}+${data.email}`
};
const response = await this.request.post('/api/v1/users', {
data: namespaced
});
if (!response.ok()) {
throw new Error(`Failed to create user: ${await response.text()}`);
}
const result = await response.json();
this.resources.push({
id: result.id,
type: 'user',
namespace: this.namespace,
createdAt: new Date()
});
// Automatically log in the user and return token
const loginResponse = await this.request.post('/api/v1/auth/login', {
data: { email: namespaced.email, password: data.password }
});
const { token } = await loginResponse.json();
return { id: result.id, email: namespaced.email, token };
}
/**
* Clean up all resources in reverse order (respects FK constraints)
*/
async cleanup(): Promise<void> {
// Sort by creation time (newest first) to respect dependencies
const sortedResources = [...this.resources].sort(
(a, b) => b.createdAt.getTime() - a.createdAt.getTime()
);
const errors: Error[] = [];
for (const resource of sortedResources) {
try {
await this.deleteResource(resource);
} catch (error) {
errors.push(error as Error);
console.error(`Failed to cleanup ${resource.type}:${resource.id}:`, error);
}
}
this.resources = [];
if (errors.length > 0) {
throw new Error(`Cleanup completed with ${errors.length} errors`);
}
}
private async deleteResource(resource: ManagedResource): Promise<void> {
const endpoints = {
'proxy-host': `/api/v1/proxy-hosts/${resource.id}`,
'certificate': `/api/v1/certificates/${resource.id}`,
'access-list': `/api/v1/access-lists/${resource.id}`,
'dns-provider': `/api/v1/dns-providers/${resource.id}`,
'user': `/api/v1/users/${resource.id}`
};
const endpoint = endpoints[resource.type];
const response = await this.request.delete(endpoint);
if (!response.ok() && response.status() !== 404) {
throw new Error(`Failed to delete ${resource.type}: ${await response.text()}`);
}
}
/**
* Get all resources created in this namespace
*/
getResources(): ManagedResource[] {
return [...this.resources];
}
/**
* Get namespace identifier
*/
getNamespace(): string {
return this.namespace;
}
}
```
#### 2.2.2 Usage Pattern
```typescript
// Example test using TestDataManager
import { test, expect } from '@playwright/test';
import { TestDataManager } from './utils/TestDataManager';
test.describe('Proxy Host Management', () => {
let testData: TestDataManager;
test.beforeEach(async ({ request }, testInfo) => {
testData = new TestDataManager(request, testInfo.title);
});
test.afterEach(async () => {
await testData.cleanup();
});
test('should create and delete proxy host', async ({ page, request }) => {
await test.step('Create proxy host', async () => {
const { id, domain } = await testData.createProxyHost({
domain: 'app.example.com',
forwardHost: '192.168.1.100',
forwardPort: 3000,
scheme: 'http'
});
await page.goto('/proxy-hosts');
await expect(page.getByText(domain)).toBeVisible();
});
// Cleanup happens automatically in afterEach
});
});
```
#### 2.2.3 Database Seeding Strategy
**Seed Data for Reference Tests:**
```typescript
// tests/fixtures/seed-data.ts
export async function seedReferenceData(request: APIRequestContext): Promise<void> {
// Create stable reference data that doesn't change
const referenceData = {
accessLists: [
{ name: 'Global Allowlist', rules: [{ type: 'allow', value: '0.0.0.0/0' }] }
],
dnsProviders: [
{ type: 'manual', name: 'Manual DNS (Default)' }
]
};
// Idempotent seeding - only create if not exists
for (const list of referenceData.accessLists) {
const response = await request.get('/api/v1/access-lists', {
params: { name: list.name }
});
const existing = await response.json();
if (existing.length === 0) {
await request.post('/api/v1/access-lists', { data: list });
}
}
}
```
#### 2.2.4 Parallel Execution Handling
**Test Isolation Strategy:**
- Each test worker gets its own namespace via `TestDataManager`
- Database transactions are NOT used (not supported in E2E context)
- Unique identifiers prevent collisions (domain names, usernames)
- Cleanup runs independently per test
**Worker ID Integration:**
```typescript
// playwright.config.ts adjustment
export default defineConfig({
workers: process.env.CI ? 4 : undefined,
use: {
storageState: ({ workerIndex }) => `auth/state-worker-${workerIndex}.json`
}
});
```
---
### 2.3 Authentication Strategy
**Critical Problem:** Current `auth.setup.ts` uses a single shared user, causing race conditions in parallel execution.
**Solution:** Per-test user creation with role-based fixtures.
#### 2.3.1 Per-Test User Creation
```typescript
// tests/fixtures/auth-fixtures.ts
import { test as base, expect, APIRequestContext } from '@playwright/test';
import { TestDataManager } from '../utils/TestDataManager';
export interface TestUser {
id: string;
email: string;
token: string;
role: 'admin' | 'user' | 'guest';
}
interface AuthFixtures {
authenticatedUser: TestUser;
adminUser: TestUser;
regularUser: TestUser;
guestUser: TestUser;
testData: TestDataManager;
}
export const test = base.extend<AuthFixtures>({
testData: async ({ request }, use, testInfo) => {
const manager = new TestDataManager(request, testInfo.title);
await use(manager);
await manager.cleanup();
},
// Default authenticated user (admin role)
authenticatedUser: async ({ testData }, use, testInfo) => {
const user = await testData.createUser({
email: `admin-${Date.now()}@test.local`,
password: 'TestPass123!',
role: 'admin'
});
await use(user);
},
// Explicit admin user fixture
adminUser: async ({ testData }, use) => {
const user = await testData.createUser({
email: `admin-${Date.now()}@test.local`,
password: 'TestPass123!',
role: 'admin'
});
await use(user);
},
// Regular user (non-admin)
regularUser: async ({ testData }, use) => {
const user = await testData.createUser({
email: `user-${Date.now()}@test.local`,
password: 'TestPass123!',
role: 'user'
});
await use(user);
},
// Guest user (read-only)
guestUser: async ({ testData }, use) => {
const user = await testData.createUser({
email: `guest-${Date.now()}@test.local`,
password: 'TestPass123!',
role: 'guest'
});
await use(user);
}
});
export { expect } from '@playwright/test';
```
#### 2.3.2 Usage Pattern
```typescript
// Example test with per-test authentication
import { test, expect } from './fixtures/auth-fixtures';
test.describe('User Management', () => {
test('admin can create users', async ({ page, adminUser }) => {
await test.step('Login as admin', async () => {
await page.goto('/login');
await page.getByLabel('Email').fill(adminUser.email);
await page.getByLabel('Password').fill('TestPass123!');
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('/');
});
await test.step('Create new user', async () => {
await page.goto('/users');
await page.getByRole('button', { name: 'Add User' }).click();
// ... rest of test
});
});
test('regular user cannot create users', async ({ page, regularUser }) => {
await test.step('Login as regular user', async () => {
await page.goto('/login');
await page.getByLabel('Email').fill(regularUser.email);
await page.getByLabel('Password').fill('TestPass123!');
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('/');
});
await test.step('Verify no access to user management', async () => {
await page.goto('/users');
await expect(page.getByText('Access Denied')).toBeVisible();
});
});
});
```
#### 2.3.3 Storage State Management
**Per-Worker Storage:**
```typescript
// tests/auth.setup.ts (revised)
import { test as setup, expect } from '@playwright/test';
import { TestDataManager } from './utils/TestDataManager';
// Generate storage state per worker
const authFile = process.env.CI
? `auth/state-worker-${process.env.TEST_WORKER_INDEX || 0}.json`
: 'auth/state.json';
setup('authenticate', async ({ request, page }) => {
const testData = new TestDataManager(request, 'setup');
try {
// Create a dedicated setup user for this worker
const user = await testData.createUser({
email: `setup-worker-${process.env.TEST_WORKER_INDEX || 0}@test.local`,
password: 'SetupPass123!',
role: 'admin'
});
await page.goto('/login');
await page.getByLabel('Email').fill(user.email);
await page.getByLabel('Password').fill('SetupPass123!');
await page.getByRole('button', { name: 'Login' }).click();
await page.waitForURL('/');
// Save authenticated state
await page.context().storageState({ path: authFile });
console.log(`✅ Auth state saved for worker ${process.env.TEST_WORKER_INDEX || 0}`);
} finally {
// Cleanup happens automatically via TestDataManager
await testData.cleanup();
}
});
```
---
### 2.4 Flaky Test Prevention
**Critical Problem:** Arbitrary timeouts (`page.waitForTimeout(1000)`) cause flaky tests and slow execution.
**Solution:** Deterministic wait utilities that poll for specific conditions.
#### 2.4.1 Wait Utilities
```typescript
// tests/utils/wait-helpers.ts
import { Page, Locator, expect } from '@playwright/test';
/**
* Wait for a toast notification with specific text
*/
export async function waitForToast(
page: Page,
text: string | RegExp,
options: { timeout?: number; type?: 'success' | 'error' | 'info' } = {}
): Promise<void> {
const { timeout = 10000, type } = options;
const toastSelector = type
? `[role="alert"][data-type="${type}"]`
: '[role="alert"]';
const toast = page.locator(toastSelector);
await expect(toast).toContainText(text, { timeout });
}
/**
* Wait for a specific API response
*/
export async function waitForAPIResponse(
page: Page,
urlPattern: string | RegExp,
options: { status?: number; timeout?: number } = {}
): Promise<Response> {
const { status, timeout = 30000 } = options;
const responsePromise = page.waitForResponse(
(response) => {
const matchesURL = typeof urlPattern === 'string'
? response.url().includes(urlPattern)
: urlPattern.test(response.url());
const matchesStatus = status ? response.status() === status : true;
return matchesURL && matchesStatus;
},
{ timeout }
);
return await responsePromise;
}
/**
* Wait for loading spinner to disappear
*/
export async function waitForLoadingComplete(
page: Page,
options: { timeout?: number } = {}
): Promise<void> {
const { timeout = 10000 } = options;
// Wait for any loading indicator to disappear
const loader = page.locator('[role="progressbar"], [aria-busy="true"], .loading-spinner');
await expect(loader).toHaveCount(0, { timeout });
}
/**
* Wait for specific element count (e.g., table rows)
*/
export async function waitForElementCount(
locator: Locator,
count: number,
options: { timeout?: number } = {}
): Promise<void> {
const { timeout = 10000 } = options;
await expect(locator).toHaveCount(count, { timeout });
}
/**
* Wait for WebSocket connection to be established
*/
export async function waitForWebSocketConnection(
page: Page,
urlPattern: string | RegExp,
options: { timeout?: number } = {}
): Promise<void> {
const { timeout = 10000 } = options;
await page.waitForEvent('websocket', {
predicate: (ws) => {
const matchesURL = typeof urlPattern === 'string'
? ws.url().includes(urlPattern)
: urlPattern.test(ws.url());
return matchesURL;
},
timeout
});
}
/**
* Wait for WebSocket message with specific content
*/
export async function waitForWebSocketMessage(
page: Page,
matcher: (data: string | Buffer) => boolean,
options: { timeout?: number } = {}
): Promise<string | Buffer> {
const { timeout = 10000 } = options;
return new Promise((resolve, reject) => {
const timer = setTimeout(() => {
reject(new Error(`WebSocket message not received within ${timeout}ms`));
}, timeout);
page.on('websocket', (ws) => {
ws.on('framereceived', (event) => {
const data = event.payload;
if (matcher(data)) {
clearTimeout(timer);
resolve(data);
}
});
});
});
}
/**
* Wait for progress bar to complete
*/
export async function waitForProgressComplete(
page: Page,
options: { timeout?: number } = {}
): Promise<void> {
const { timeout = 30000 } = options;
const progressBar = page.locator('[role="progressbar"]');
// Wait for progress to reach 100% or disappear
await page.waitForFunction(
() => {
const bar = document.querySelector('[role="progressbar"]');
if (!bar) return true; // Progress bar gone = complete
const value = bar.getAttribute('aria-valuenow');
return value === '100';
},
{ timeout }
);
// Wait for progress bar to disappear
await expect(progressBar).toHaveCount(0, { timeout: 5000 });
}
/**
* Wait for modal to open
*/
export async function waitForModal(
page: Page,
titleText: string | RegExp,
options: { timeout?: number } = {}
): Promise<Locator> {
const { timeout = 10000 } = options;
const modal = page.locator('[role="dialog"]');
await expect(modal).toBeVisible({ timeout });
if (titleText) {
await expect(modal.getByRole('heading')).toContainText(titleText);
}
return modal;
}
/**
* Wait for dropdown/listbox to open
*/
export async function waitForDropdown(
page: Page,
triggerId: string,
options: { timeout?: number } = {}
): Promise<Locator> {
const { timeout = 5000 } = options;
const trigger = page.locator(`#${triggerId}`);
const expanded = await trigger.getAttribute('aria-expanded');
if (expanded !== 'true') {
throw new Error(`Dropdown ${triggerId} is not expanded`);
}
const listboxId = await trigger.getAttribute('aria-controls');
if (!listboxId) {
throw new Error(`Dropdown ${triggerId} has no aria-controls`);
}
const listbox = page.locator(`#${listboxId}`);
await expect(listbox).toBeVisible({ timeout });
return listbox;
}
/**
* Wait for table to finish loading and render rows
*/
export async function waitForTableLoad(
page: Page,
tableRole: string = 'table',
options: { minRows?: number; timeout?: number } = {}
): Promise<void> {
const { minRows = 0, timeout = 10000 } = options;
const table = page.getByRole(tableRole);
await expect(table).toBeVisible({ timeout });
// Wait for loading state to clear
await waitForLoadingComplete(page);
// If minimum rows specified, wait for them
if (minRows > 0) {
const rows = table.locator('tbody tr');
await expect(rows).toHaveCount(minRows, { timeout });
}
}
/**
* Retry an action until it succeeds or timeout
*/
export async function retryAction<T>(
action: () => Promise<T>,
options: {
maxAttempts?: number;
interval?: number;
timeout?: number;
} = {}
): Promise<T> {
const { maxAttempts = 5, interval = 1000, timeout = 30000 } = options;
const startTime = Date.now();
let lastError: Error | undefined;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
if (Date.now() - startTime > timeout) {
throw new Error(`Retry timeout after ${timeout}ms`);
}
try {
return await action();
} catch (error) {
lastError = error as Error;
if (attempt < maxAttempts) {
await new Promise(resolve => setTimeout(resolve, interval));
}
}
}
throw lastError || new Error('Retry failed');
}
```
#### 2.4.2 Usage Examples
```typescript
import { test, expect } from '@playwright/test';
import {
waitForToast,
waitForAPIResponse,
waitForLoadingComplete,
waitForModal
} from './utils/wait-helpers';
test('create proxy host with deterministic waits', async ({ page }) => {
await test.step('Navigate and open form', async () => {
await page.goto('/proxy-hosts');
await page.getByRole('button', { name: 'Add Proxy Host' }).click();
await waitForModal(page, 'Create Proxy Host');
});
await test.step('Fill form and submit', async () => {
await page.getByLabel('Domain Name').fill('test.example.com');
await page.getByLabel('Forward Host').fill('192.168.1.100');
await page.getByLabel('Forward Port').fill('3000');
// Wait for API call to complete
const responsePromise = waitForAPIResponse(page, '/api/v1/proxy-hosts', { status: 201 });
await page.getByRole('button', { name: 'Save' }).click();
await responsePromise;
});
await test.step('Verify success', async () => {
await waitForToast(page, 'Proxy host created successfully', { type: 'success' });
await waitForLoadingComplete(page);
await expect(page.getByRole('row', { name: /test.example.com/ })).toBeVisible();
});
});
```
---
### 2.5 CI/CD Integration
**Objective:** Automate E2E test execution on every PR with parallel execution, comprehensive reporting, and failure artifacts.
#### 2.5.1 GitHub Actions Workflow
```yaml
# .github/workflows/e2e-tests.yml
name: E2E Tests (Playwright)
on:
pull_request:
branches: [main, develop]
paths:
- 'frontend/**'
- 'backend/**'
- 'tests/**'
- 'playwright.config.js'
- '.github/workflows/e2e-tests.yml'
push:
branches: [main]
workflow_dispatch:
inputs:
browser:
description: 'Browser to test'
required: false
default: 'chromium'
type: choice
options:
- chromium
- firefox
- webkit
- all
env:
NODE_VERSION: '18'
GO_VERSION: '1.21'
jobs:
# Build application once, share across test shards
build:
name: Build Application
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Build frontend
run: npm run build
working-directory: frontend
- name: Build backend
run: make build
working-directory: backend
- name: Build Docker image
run: |
docker build -t charon:test .
docker save charon:test -o charon-test-image.tar
- name: Upload Docker image
uses: actions/upload-artifact@v4
with:
name: docker-image
path: charon-test-image.tar
retention-days: 1
# Run tests in parallel shards
e2e-tests:
name: E2E Tests (Shard ${{ matrix.shard }})
runs-on: ubuntu-latest
needs: build
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
browser: [chromium] # Can be extended to [chromium, firefox, webkit]
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
- name: Download Docker image
uses: actions/download-artifact@v4
with:
name: docker-image
- name: Load Docker image
run: docker load -i charon-test-image.tar
- name: Start test environment
run: |
docker compose -f .docker/compose/docker-compose.playwright.yml up -d
- name: Wait for service healthy
run: |
timeout 60 bash -c 'until curl -f http://localhost:8080/api/v1/health; do sleep 2; done'
- name: Install dependencies
run: npm ci
- name: Install Playwright browsers
run: npx playwright install --with-deps ${{ matrix.browser }}
- name: Run E2E tests (Shard ${{ matrix.shard }})
run: |
npx playwright test \
--project=${{ matrix.browser }} \
--shard=${{ matrix.shard }}/4 \
--reporter=html,json,junit
env:
PLAYWRIGHT_BASE_URL: http://localhost:8080
CI: true
TEST_WORKER_INDEX: ${{ matrix.shard }}
- name: Upload test results
if: always()
uses: actions/upload-artifact@v4
with:
name: test-results-${{ matrix.browser }}-shard-${{ matrix.shard }}
path: |
playwright-report/
test-results/
retention-days: 7
- name: Upload test traces
if: failure()
uses: actions/upload-artifact@v4
with:
name: traces-${{ matrix.browser }}-shard-${{ matrix.shard }}
path: test-results/**/*.zip
retention-days: 7
- name: Collect Docker logs
if: failure()
run: |
docker compose -f .docker/compose/docker-compose.playwright.yml logs > docker-logs.txt
- name: Upload Docker logs
if: failure()
uses: actions/upload-artifact@v4
with:
name: docker-logs-shard-${{ matrix.shard }}
path: docker-logs.txt
retention-days: 7
# Merge reports from all shards
merge-reports:
name: Merge Test Reports
runs-on: ubuntu-latest
needs: e2e-tests
if: always()
steps:
- uses: actions/checkout@v4
- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Download all reports
uses: actions/download-artifact@v4
with:
pattern: test-results-*
path: all-results
- name: Merge Playwright HTML reports
run: npx playwright merge-reports --reporter html all-results
- name: Upload merged report
uses: actions/upload-artifact@v4
with:
name: merged-playwright-report
path: playwright-report/
retention-days: 30
- name: Generate summary
run: |
echo "## E2E Test Results" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
cat all-results/*/report.json | jq -s '
{
total: (map(.stats.expected + .stats.unexpected + .stats.flaky + .stats.skipped) | add),
passed: (map(.stats.expected) | add),
failed: (map(.stats.unexpected) | add),
flaky: (map(.stats.flaky) | add),
skipped: (map(.stats.skipped) | add)
}
' | jq -r '"- **Total**: \(.total)\n- **Passed**: \(.passed)\n- **Failed**: \(.failed)\n- **Flaky**: \(.flaky)\n- **Skipped**: \(.skipped)"' >> $GITHUB_STEP_SUMMARY
# Comment on PR with results
comment-results:
name: Comment Test Results on PR
runs-on: ubuntu-latest
needs: merge-reports
if: github.event_name == 'pull_request'
permissions:
pull-requests: write
steps:
- name: Download merged report
uses: actions/download-artifact@v4
with:
name: merged-playwright-report
path: playwright-report
- name: Extract test stats
id: stats
run: |
STATS=$(cat playwright-report/report.json | jq -c '.stats')
echo "stats=$STATS" >> $GITHUB_OUTPUT
- name: Comment on PR
uses: actions/github-script@v7
with:
script: |
const stats = JSON.parse('${{ steps.stats.outputs.stats }}');
const passed = stats.expected;
const failed = stats.unexpected;
const flaky = stats.flaky;
const total = passed + failed + flaky + stats.skipped;
const emoji = failed > 0 ? '❌' : flaky > 0 ? '⚠️' : '✅';
const status = failed > 0 ? 'FAILED' : flaky > 0 ? 'FLAKY' : 'PASSED';
const body = `## ${emoji} E2E Test Results: ${status}
| Metric | Count |
|--------|-------|
| Total | ${total} |
| Passed | ✅ ${passed} |
| Failed | ❌ ${failed} |
| Flaky | ⚠️ ${flaky} |
| Skipped | ⏭️ ${stats.skipped} |
[View full Playwright report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})
${failed > 0 ? '⚠️ **Tests failed!** Please review the failures and fix before merging.' : ''}
${flaky > 0 ? '⚠️ **Flaky tests detected!** Please investigate and stabilize before merging.' : ''}
`;
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: body
});
# Block merge if tests fail
e2e-results:
name: E2E Test Results
runs-on: ubuntu-latest
needs: e2e-tests
if: always()
steps:
- name: Check test results
run: |
if [ "${{ needs.e2e-tests.result }}" != "success" ]; then
echo "E2E tests failed or were cancelled"
exit 1
fi
```
#### 2.5.2 Test Sharding Strategy
**Why Shard:** Reduces CI run time from ~40 minutes to ~10 minutes with 4 parallel shards.
**Sharding Configuration:**
```typescript
// playwright.config.ts
export default defineConfig({
testDir: './tests',
fullyParallel: true,
workers: process.env.CI ? 4 : undefined,
retries: process.env.CI ? 2 : 0,
projects: [
{
name: 'chromium',
use: { ...devices['Desktop Chrome'] }
}
],
// CI-specific optimizations
...(process.env.CI && {
reporter: [
['html', { outputFolder: 'playwright-report' }],
['json', { outputFile: 'test-results/results.json' }],
['junit', { outputFile: 'test-results/junit.xml' }]
],
maxFailures: 10 // Stop after 10 failures to save CI time
})
});
```
**Shard Distribution:**
- Shard 1: `tests/core/**`, `tests/proxy/**` (~10 min)
- Shard 2: `tests/dns/**`, `tests/certificates/**` (~10 min)
- Shard 3: `tests/security/**` (~10 min)
- Shard 4: `tests/settings/**`, `tests/tasks/**`, `tests/monitoring/**`, `tests/integration/**` (~10 min)
#### 2.5.3 Cache Strategy
```yaml
# Cache Playwright browsers
- name: Cache Playwright browsers
uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: playwright-browsers-${{ hashFiles('package-lock.json') }}
# Cache npm dependencies
- name: Cache npm dependencies
uses: actions/cache@v4
with:
path: ~/.npm
key: npm-${{ hashFiles('package-lock.json') }}
# Cache Docker layers
- name: Cache Docker layers
uses: actions/cache@v4
with:
path: /tmp/.buildx-cache
key: docker-${{ github.sha }}
restore-keys: docker-
```
#### 2.5.4 Failure Notification Strategy
**Slack Notification (Optional):**
```yaml
- name: Notify Slack on failure
if: failure() && github.event_name == 'push'
uses: slackapi/slack-github-action@v1
with:
payload: |
{
"text": "❌ E2E tests failed on ${{ github.ref }}",
"blocks": [
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*E2E Tests Failed*\n\nBranch: `${{ github.ref }}`\nCommit: `${{ github.sha }}`\n<https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}|View Run>"
}
}
]
}
env:
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
```
---
## 3. Test Organization
**Directory Structure:**
```
tests/
├── auth.setup.ts # ✅ Exists - Authentication setup
├── fixtures/ # Shared test data and utilities
│ ├── dns-providers.ts # ✅ Exists - DNS fixtures
│ ├── proxy-hosts.ts # 📝 To create
│ ├── access-lists.ts # 📝 To create
│ ├── certificates.ts # 📝 To create
│ └── test-data.ts # 📝 To create - Common test data
├── core/ # Core application features
│ ├── authentication.spec.ts # 📝 Auth flows
│ ├── dashboard.spec.ts # 📝 Dashboard UI
│ └── navigation.spec.ts # 📝 App navigation
├── proxy/ # Proxy management
│ ├── proxy-hosts-crud.spec.ts # 📝 CRUD operations
│ ├── proxy-hosts-validation.spec.ts # 📝 Form validation
│ ├── proxy-hosts-docker.spec.ts # 📝 Docker discovery
│ └── remote-servers.spec.ts # 📝 Remote Docker servers
├── dns/ # DNS management
│ ├── manual-dns-provider.spec.ts # ✅ Exists - Manual provider
│ ├── dns-provider-crud.spec.ts # ✅ Exists - CRUD operations
│ ├── dns-provider-types.spec.ts # ✅ Exists - Provider types
│ ├── dns-provider-credentials.spec.ts # 📝 Credential management
│ └── dns-plugins.spec.ts # 📝 Plugin management
├── certificates/ # SSL certificate management
│ ├── certificates-list.spec.ts # 📝 List and view
│ ├── certificates-upload.spec.ts # 📝 Upload custom certs
│ └── certificates-acme.spec.ts # 📝 ACME integration
├── security/ # Cerberus security suite
│ ├── access-lists-crud.spec.ts # 📝 ACL CRUD
│ ├── access-lists-rules.spec.ts # 📝 ACL rule engine
│ ├── crowdsec-config.spec.ts # 📝 CrowdSec setup
│ ├── crowdsec-decisions.spec.ts # 📝 Ban management
│ ├── crowdsec-presets.spec.ts # 📝 Preset management
│ ├── waf-config.spec.ts # 📝 WAF configuration
│ ├── rate-limiting.spec.ts # 📝 Rate limit rules
│ ├── security-headers.spec.ts # 📝 Security headers
│ └── audit-logs.spec.ts # 📝 Audit trail
├── settings/ # System configuration
│ ├── system-settings.spec.ts # 📝 System config
│ ├── smtp-settings.spec.ts # 📝 Email config
│ ├── notifications.spec.ts # 📝 Notification config
│ ├── user-management.spec.ts # 📝 User CRUD
│ ├── encryption-management.spec.ts # 📝 Key rotation
│ └── account-settings.spec.ts # 📝 User profile
├── tasks/ # Background tasks & maintenance
│ ├── backups-create.spec.ts # 📝 Backup creation
│ ├── backups-restore.spec.ts # 📝 Backup restoration
│ ├── logs-viewing.spec.ts # 📝 Log viewer
│ ├── import-caddyfile.spec.ts # 📝 Caddy import
│ └── import-crowdsec.spec.ts # 📝 CrowdSec import
├── monitoring/ # Monitoring features
│ ├── uptime-monitoring.spec.ts # 📝 Uptime checks
│ └── real-time-logs.spec.ts # 📝 WebSocket logs
└── integration/ # Cross-feature integration
├── proxy-acl-integration.spec.ts # 📝 Proxy + ACL
├── proxy-certificate.spec.ts # 📝 Proxy + SSL
├── security-suite-integration.spec.ts # 📝 Full security stack
└── backup-restore-e2e.spec.ts # 📝 Full backup cycle
```
### Test Execution Strategy
**Playwright Configuration:**
- ✅ Base URL: `http://100.98.12.109:8080` (Tailscale IP) or `http://localhost:8080` (CI)
- ✅ Browser support: Chromium (primary), Firefox, WebKit
- ✅ Parallel execution: Enabled for faster runs
- ✅ Authentication: Shared state via `auth.setup.ts`
- ✅ Timeouts: 30s test, 5s expect
- ✅ Retries: 2 on CI, 0 on local
**Test Data Management:**
- Use fixtures for reusable test data
- Clean up created resources after tests
- Use unique identifiers for test resources (timestamps, UUIDs)
- Avoid hardcoded IDs or names that could conflict
**Accessibility Testing:**
- All tests must verify keyboard navigation
- Use `toMatchAriaSnapshot` for component structure validation
- Verify ARIA labels, roles, and live regions
- Test screen reader announcements for state changes
---
## 4. Implementation Plan
### Phase 0: Infrastructure Setup (Week 1-2)
**Goal:** Build robust test infrastructure before writing feature tests
#### Week 1: Core Infrastructure
**Priority:** Critical - Blocking all test development
**Estimated Effort:** 5 days
**Tasks:**
- [ ] Set up `TestDataManager` utility with namespace isolation
- [ ] Implement per-test user creation in `auth-fixtures.ts`
- [ ] Create all wait helper utilities (`waitForToast`, `waitForAPIResponse`, etc.)
- [ ] Configure test environment Docker Compose files (`.test.yml`)
- [ ] Write `setup-e2e-env.sh` script with health checks
- [ ] Implement mock external services (DNS providers, ACME servers)
- [ ] Configure test environment variables in `.env.test`
**Acceptance Criteria:**
- `TestDataManager` can create and cleanup all resource types
- Per-test users can be created with different roles
- Wait utilities replace all `page.waitForTimeout()` calls
- Test environment starts reliably with `npm run test:env:start`
- Mock services respond to API calls correctly
#### Week 2: CI/CD Integration
**Priority:** Critical - Required for PR automation
**Estimated Effort:** 5 days
**Tasks:**
- [ ] Create `.github/workflows/e2e-tests.yml`
- [ ] Implement test sharding strategy (4 shards)
- [ ] Configure artifact upload (reports, traces, logs)
- [ ] Set up PR comment reporting
- [ ] Configure caching for npm, Playwright browsers, Docker layers
- [ ] Test workflow end-to-end on a feature branch
- [ ] Document CI/CD troubleshooting guide
**Acceptance Criteria:**
- E2E tests run automatically on every PR
- Test results appear as PR comments
- Failed tests upload traces and logs
- CI run completes in <15 minutes with sharding
- Flaky test retries work correctly
### Phase 1: Foundation (Week 3)
**Status:** ✅ COMPLETE
**Completion Date:** January 17, 2026
**Test Results:** 112/119 passing (94%)
**Goal:** Establish core application testing patterns
#### 1.1 Test Fixtures & Helpers
**Priority:** Critical
**Status:** ✅ Complete
**Delivered Files:**
- [x] `tests/fixtures/test-data.ts` - Common test data generators
- [x] `tests/fixtures/proxy-hosts.ts` - Mock proxy host data
- [x] `tests/fixtures/access-lists.ts` - Mock ACL data
- [x] `tests/fixtures/certificates.ts` - Mock certificate data
- [x] `tests/fixtures/auth-fixtures.ts` - Per-test authentication
- [x] `tests/fixtures/navigation.ts` - Navigation helpers
- [x] `tests/utils/api-helpers.ts` - Common API operations
- [x] `tests/utils/wait-helpers.ts` - Deterministic wait utilities
- [x] `tests/utils/test-data-manager.ts` - Test data isolation
- [x] `tests/utils/accessibility-helpers.ts` - A11y testing utilities
**Acceptance Criteria:** ✅ Met
- Fixtures provide consistent, reusable test data
- API helpers reduce code duplication
- All utilities have JSDoc comments and usage examples
**Test File Template:**
```typescript
import { test, expect } from './fixtures/auth-fixtures'; // Use custom fixtures
import { TestDataManager } from './utils/TestDataManager';
test.describe('Feature Name', () => {
let testData: TestDataManager;
test.beforeEach(async ({ request, page }, testInfo) => {
testData = new TestDataManager(request, testInfo.title);
await page.goto('/feature-path');
});
test.afterEach(async () => {
await testData.cleanup(); // Guaranteed cleanup
});
test('should perform specific action', async ({ page, authenticatedUser }) => {
await test.step('User action', async () => {
// Use authenticatedUser fixture for API calls
await page.goto('/feature');
await page.getByRole('button', { name: 'Action' }).click();
});
await test.step('Verify result', async () => {
await expect(page.getByText('Success')).toBeVisible();
});
});
});
```
#### 1.2 Core Authentication & Navigation Tests
**Priority:** Critical
**Status:** ✅ Complete (with known issues tracked)
**Delivered Test Files:**
**`tests/core/authentication.spec.ts`** - 16 tests (13 passing, 3 failing - tracked)
- ✅ Login with valid credentials (covered by auth.setup.ts)
- ✅ Login with invalid credentials
- ✅ Logout functionality
- ✅ Session persistence
- ⚠️ Session expiration handling (3 tests failing - see [Issue: e2e-session-expiration-tests](../issues/e2e-session-expiration-tests.md))
- ✅ Password reset flow (if implemented)
**`tests/core/dashboard.spec.ts`** - All tests passing
- ✅ Dashboard loads successfully
- ✅ Summary cards display correct data
- ✅ Quick action buttons are functional
- ✅ Recent activity shows latest changes
- ✅ System status indicators work
**`tests/core/navigation.spec.ts`** - All tests passing
- ✅ All main menu items are clickable
- ✅ Sidebar navigation works
- ✅ Breadcrumbs display correctly
- ✅ Deep links resolve properly
- ✅ Back button navigation works
**Known Issues:**
- 3 session expiration tests require route mocking - tracked in [docs/issues/e2e-session-expiration-tests.md](../issues/e2e-session-expiration-tests.md)
**Acceptance Criteria:** ✅ Met (with known exceptions)
- All authentication flows covered (session expiration deferred)
- Dashboard displays without errors
- Navigation between all pages works
- No console errors during navigation
- Keyboard navigation fully functional
### Phase 2: Critical Path (Week 4-5)
**Goal:** Cover the most critical user journeys
#### 2.1 Proxy Hosts Management
**Priority:** Critical
**Estimated Effort:** 4 days
**Test Files:**
**`tests/proxy/proxy-hosts-crud.spec.ts`**
Test Scenarios:
- ✅ List all proxy hosts (empty state)
- ✅ Create new proxy host with basic configuration
- Enter domain name (e.g., `test-app.example.com`)
- Enter forward hostname (e.g., `192.168.1.100`)
- Enter forward port (e.g., `3000`)
- Select scheme (HTTP/HTTPS)
- Enable/disable WebSocket support
- Save and verify host appears in list
- ✅ View proxy host details
- ✅ Edit existing proxy host
- Update domain name
- Update forward hostname/port
- Toggle WebSocket support
- Save and verify changes
- ✅ Delete proxy host
- Delete single host
- Verify deletion confirmation dialog
- Verify host removed from list
- ✅ Bulk operations (if supported)
**Key User Flows:**
1. **Create Basic Proxy Host:**
```
Navigate → Click "Add Proxy Host" → Fill form → Save → Verify in list
```
2. **Edit Existing Host:**
```
Navigate → Select host → Click edit → Modify → Save → Verify changes
```
3. **Delete Host:**
```
Navigate → Select host → Click delete → Confirm → Verify removal
```
**Critical Assertions:**
- Host appears in list after creation
- Edit changes are persisted
- Deletion removes host from database
- Validation prevents invalid data
- Success/error messages display correctly
#### 2.2 SSL Certificates Management
**Priority:** Critical
**Estimated Effort:** 4 days
**Test Files:**
**`tests/certificates/certificates-list.spec.ts`**
Test Scenarios:
- ✅ List all certificates (empty state)
- ✅ Display certificate details (domain, expiry, issuer)
- ✅ Filter certificates by status (valid, expiring, expired)
- ✅ Sort certificates by expiry date
- ✅ Search certificates by domain name
- ✅ Show certificate chain details
**`tests/certificates/certificates-upload.spec.ts`**
Test Scenarios:
- ✅ Upload custom certificate with private key
- ✅ Validate PEM format
- ✅ Reject invalid certificate formats
- ✅ Reject mismatched certificate and key
- ✅ Support intermediate certificate chains
- ✅ Update existing certificate
- ✅ Delete custom certificate
**`tests/certificates/certificates-acme.spec.ts`**
Test Scenarios:
**ACME HTTP-01 Challenge:**
- ✅ Request certificate via HTTP-01 challenge
- Select domain from proxy hosts
- Choose HTTP-01 validation method
- Verify challenge file is served at `/.well-known/acme-challenge/`
- Mock ACME server validates challenge
- Certificate issued and stored
- ✅ HTTP-01 challenge fails if proxy host not accessible
- ✅ HTTP-01 challenge fails with invalid domain
**ACME DNS-01 Challenge:**
- ✅ Request certificate via DNS-01 challenge
- Select DNS provider (Cloudflare, Route53, Manual)
- Mock DNS provider API for TXT record creation
- Verify TXT record `_acme-challenge.domain.com` created
- Mock ACME server validates DNS record
- Certificate issued and stored
- ✅ DNS-01 challenge supports wildcard certificates
- Request `*.example.com` certificate
- Verify TXT record for `_acme-challenge.example.com`
- Certificate covers all subdomains
- ✅ DNS-01 challenge fails with invalid DNS credentials
- ✅ DNS-01 challenge retries on DNS propagation delay
**Certificate Renewal:**
- ✅ Automatic renewal triggered 30 days before expiry
- Mock certificate with expiry in 29 days
- Verify renewal task scheduled
- Renewal completes successfully
- Old certificate archived
- ✅ Manual certificate renewal
- Click "Renew Now" button
- Renewal process uses same validation method
- New certificate replaces old
- ✅ Renewal fails gracefully
- Old certificate remains active
- Error notification displayed
- Retry mechanism available
**Wildcard Certificates:**
- ✅ Request wildcard certificate (`*.example.com`)
- DNS-01 challenge required (HTTP-01 not supported)
- Verify TXT record created
- Certificate issued with wildcard SAN
- ✅ Wildcard certificate applies to all subdomains
- Create proxy host `app.example.com`
- Wildcard certificate auto-selected
- HTTPS works for any subdomain
**Certificate Revocation:**
- ✅ Revoke Let's Encrypt certificate
- Click "Revoke" button
- Confirm revocation reason
- Certificate marked as revoked
- ACME server notified
- ✅ Revoked certificate cannot be used
- Proxy hosts using certificate show warning
- HTTPS connections fail
**Validation Error Handling:**
- ✅ ACME account registration fails
- Invalid email address
- Rate limit exceeded
- Network error during registration
- ✅ Challenge validation fails
- HTTP-01: Challenge file not accessible
- DNS-01: TXT record not found
- DNS-01: DNS propagation timeout
- ✅ Certificate issuance fails
- ACME server error
- Domain validation failed
- Rate limit exceeded
**Mixed Certificate Sources:**
- ✅ Use Let's Encrypt and custom certificates together
- Some domains use Let's Encrypt
- Some domains use custom certificates
- Certificates don't conflict
- ✅ Migrate from custom to Let's Encrypt
- Replace custom certificate with Let's Encrypt
- No downtime during migration
- Old certificate archived
**Certificate Metadata:**
- ✅ Display certificate information
- Issuer, subject, validity period
- SAN (Subject Alternative Names)
- Signature algorithm
- Certificate chain
- ✅ Export certificate in various formats
- PEM, DER, PFX
- With or without private key
- Include full chain
**Key User Flows:**
1. **HTTP-01 Challenge Flow:**
```
Navigate → Click "Request Certificate" → Select domain → Choose HTTP-01 →
Monitor challenge → Certificate issued → Verify in list
```
2. **DNS-01 Wildcard Flow:**
```
Navigate → Click "Request Certificate" → Enter *.example.com → Choose DNS-01 →
Select DNS provider → Monitor DNS propagation → Certificate issued → Verify wildcard works
```
3. **Certificate Renewal Flow:**
```
Navigate → Select expiring certificate → Click "Renew" →
Automatic challenge re-validation → New certificate issued → Old certificate archived
```
**Critical Assertions:**
- Challenge files/records created correctly
- ACME server validates challenges
- Certificates issued with correct domains
- Renewal happens before expiry
- Validation errors display helpful messages
- Certificate chain is complete and valid
#### 2.3 Access Lists (ACL)
**Priority:** Critical
**Estimated Effort:** 3 days
**Test Files:**
**`tests/access-lists/access-lists-crud.spec.ts`**
Test Scenarios:
- ✅ List all access lists (empty state)
- Verify empty state message displayed
- "Create Access List" CTA visible
- ✅ Create IP whitelist (Allow Only)
- Enter name (e.g., "Office IPs")
- Add description
- Select type: IP Whitelist
- Add IP rules (single IP, CIDR ranges)
- Save and verify appears in list
- ✅ Create IP blacklist (Block Only)
- Select type: IP Blacklist
- Add blocked IPs/ranges
- Verify badge shows "Deny"
- ✅ Create geo-whitelist
- Select type: Geo Whitelist
- Select allowed countries (US, CA, GB)
- Verify country badges displayed
- ✅ Create geo-blacklist
- Select type: Geo Blacklist
- Block high-risk countries
- Apply security presets
- ✅ Enable/disable access list
- Toggle enabled state
- Verify badge shows correct status
- ✅ Edit access list
- Update name, description, rules
- Add/remove IP ranges
- Change type (whitelist ↔ blacklist)
- ✅ Delete access list
- Confirm backup creation before delete
- Verify removed from list
- Verify proxy hosts unaffected
**`tests/access-lists/access-lists-rules.spec.ts`**
Test Scenarios:
- ✅ Add single IP address
- Enter `192.168.1.100`
- Add optional description
- Verify appears in rules list
- ✅ Add CIDR range
- Enter `10.0.0.0/24`
- Verify covers 256 IPs
- Display IP count badge
- ✅ Add multiple rules
- Add 5+ IP rules
- Verify all rules displayed
- Support pagination/scrolling
- ✅ Remove individual rule
- Click delete on specific rule
- Verify removed from list
- Other rules unaffected
- ✅ RFC1918 Local Network Only
- Toggle "Local Network Only" switch
- Verify IP rules section hidden
- Description shows "RFC1918 ranges only"
- ✅ Invalid IP validation
- Enter invalid IP (e.g., `999.999.999.999`)
- Verify error message displayed
- Form not submitted
- ✅ Invalid CIDR validation
- Enter invalid CIDR (e.g., `192.168.1.0/99`)
- Verify error message displayed
- ✅ Get My IP feature
- Click "Get My IP" button
- Verify current IP populated in field
- Toast shows IP source
**`tests/access-lists/access-lists-geo.spec.ts`**
Test Scenarios:
- ✅ Select single country
- Click country in dropdown
- Country badge appears
- ✅ Select multiple countries
- Add US, CA, GB
- All badges displayed
- Deselect removes badge
- ✅ Country search/filter
- Type "united" in search
- Shows United States, United Kingdom, UAE
- Select from filtered list
- ✅ Geo-whitelist vs geo-blacklist behavior
- Whitelist: only selected countries allowed
- Blacklist: selected countries blocked
**`tests/access-lists/access-lists-presets.spec.ts`**
Test Scenarios:
- ✅ Show security presets (blacklist only)
- Presets section hidden for whitelists
- Presets section visible for blacklists
- ✅ Apply "Known Malicious Actors" preset
- Click "Apply" on preset
- IP rules populated
- Toast shows rules added count
- ✅ Apply "High-Risk Countries" preset
- Apply geo-blacklist preset
- Countries auto-selected
- Can add additional countries
- ✅ Preset warning displayed
- Shows data source and update frequency
- Warning for aggressive presets
**`tests/access-lists/access-lists-test-ip.spec.ts`**
Test Scenarios:
- ✅ Open Test IP dialog
- Click test tube icon on ACL row
- Dialog opens with IP input
- ✅ Test allowed IP
- Enter IP that should be allowed
- Click "Test"
- Success toast: "✅ IP Allowed: [reason]"
- ✅ Test blocked IP
- Enter IP that should be blocked
- Click "Test"
- Error toast: "🚫 IP Blocked: [reason]"
- ✅ Invalid IP test
- Enter invalid IP
- Error toast displayed
- ✅ Test RFC1918 detection
- Test with private IP (192.168.x.x)
- Verify local network detection
- ✅ Test IPv6 address
- Enter IPv6 address
- Verify correct allow/block decision
**`tests/access-lists/access-lists-integration.spec.ts`**
Test Scenarios:
- ✅ Assign ACL to proxy host
- Edit proxy host
- Select ACL from dropdown
- Save and verify assignment
- ✅ ACL selector shows only enabled lists
- Disabled ACLs hidden from dropdown
- Enabled ACLs visible with type badge
- ✅ Bulk update ACL on multiple hosts
- Select multiple hosts
- Click "Update ACL" bulk action
- Select ACL from modal
- Verify all hosts updated
- ✅ Remove ACL from proxy host
- Select "No Access Control (Public)"
- Verify ACL unassigned
- ✅ Delete ACL in use
- Attempt delete of assigned ACL
- Warning shows affected hosts
- Confirm or cancel
**Key UI Selectors:**
```typescript
// AccessLists.tsx page selectors
'button >> text=Create Access List' // Create button
'[role="table"]' // ACL list table
'[role="row"]' // Individual ACL rows
'button >> text=Edit' // Edit action (row)
'button >> text=Delete' // Delete action (row)
'button[title*="Test IP"]' // Test IP button (TestTube2 icon)
// AccessListForm.tsx selectors
'input#name' // Name input
'textarea#description' // Description input
'select#type' // Type dropdown (whitelist/blacklist/geo)
'[data-state="checked"]' // Enabled toggle (checked)
'button >> text=Get My IP' // Get current IP
'button >> text=Add' // Add IP rule
'input[placeholder*="192.168"]' // IP input field
// AccessListSelector.tsx selectors
'select >> text=Access Control List' // ACL selector in ProxyHostForm
'option >> text=No Access Control' // Public option
```
**API Endpoints:**
```typescript
// Access Lists CRUD
GET /api/v1/access-lists // List all
GET /api/v1/access-lists/:id // Get single
POST /api/v1/access-lists // Create
PUT /api/v1/access-lists/:id // Update
DELETE /api/v1/access-lists/:id // Delete
POST /api/v1/access-lists/:id/test // Test IP against ACL
GET /api/v1/access-lists/templates // Get presets
// Proxy Host ACL Integration
PUT /api/v1/proxy-hosts/bulk-update-acl // Bulk ACL update
```
**Critical Assertions:**
- ACL appears in list after creation
- IP rules correctly parsed and displayed
- Type badges match ACL configuration
- Test IP returns accurate allow/block decisions
- ACL assignment persists on proxy hosts
- Validation prevents invalid CIDR/IP input
- Security presets apply correctly
---
## Phase 2 Implementation Plan (Detailed)
**Timeline:** Week 4-5 (2 weeks)
**Total Tests Estimated:** 95-105 tests
**Based on Phase 1 Velocity:** 112 tests in ~1 week = ~16 tests/day
### Week 4: Proxy Hosts & Access Lists (Days 1-5)
#### Day 1-2: Proxy Hosts CRUD (30-35 tests)
**File: `tests/proxy/proxy-hosts-crud.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | displays empty state when no hosts exist | `[data-testid="empty-state"]`, `text=Add Proxy Host` | `GET /proxy-hosts` | P0 |
| 2 | shows skeleton loading while fetching | `[data-testid="skeleton-table"]` | `GET /proxy-hosts` | P1 |
| 3 | lists all proxy hosts in table | `role=table`, `role=row` | `GET /proxy-hosts` | P0 |
| 4 | displays host details (domain, forward, ssl) | `role=cell` | - | P0 |
| 5 | opens create form when Add clicked | `button >> text=Add Proxy Host`, `role=dialog` | - | P0 |
| 6 | creates basic HTTP proxy host | `#proxy-name`, `#domain-names`, `#forward-host`, `#forward-port` | `POST /proxy-hosts` | P0 |
| 7 | creates HTTPS proxy host with SSL | `[name="ssl_forced"]` | `POST /proxy-hosts` | P0 |
| 8 | creates proxy with WebSocket support | `[name="allow_websocket_upgrade"]` | `POST /proxy-hosts` | P1 |
| 9 | creates proxy with HTTP/2 support | `[name="http2_support"]` | `POST /proxy-hosts` | P1 |
| 10 | shows Docker containers in dropdown | `button >> text=Docker Discovery` | `GET /docker/containers` | P1 |
| 11 | auto-fills from Docker container | Docker container option | - | P1 |
| 12 | validates empty domain name | `#domain-names:invalid` | - | P0 |
| 13 | validates invalid domain format | Error toast | - | P0 |
| 14 | validates empty forward host | `#forward-host:invalid` | - | P0 |
| 15 | validates invalid forward port | `#forward-port:invalid` | - | P0 |
| 16 | validates port out of range (0, 65536) | Error message | - | P1 |
| 17 | rejects XSS in domain name | 422 response | `POST /proxy-hosts` | P0 |
| 18 | rejects SQL injection in fields | 422 response | `POST /proxy-hosts` | P0 |
| 19 | opens edit form for existing host | `button[aria-label="Edit"]` | `GET /proxy-hosts/:uuid` | P0 |
| 20 | updates domain name | Form submission | `PUT /proxy-hosts/:uuid` | P0 |
| 21 | updates forward host and port | Form submission | `PUT /proxy-hosts/:uuid` | P0 |
| 22 | toggles host enabled/disabled | `role=switch` | `PUT /proxy-hosts/:uuid` | P0 |
| 23 | assigns SSL certificate | Certificate selector | `PUT /proxy-hosts/:uuid` | P1 |
| 24 | assigns access list | ACL selector | `PUT /proxy-hosts/:uuid` | P1 |
| 25 | shows delete confirmation dialog | `role=alertdialog` | - | P0 |
| 26 | deletes single host | Confirm button | `DELETE /proxy-hosts/:uuid` | P0 |
| 27 | cancels delete operation | Cancel button | - | P1 |
| 28 | shows success toast after CRUD | `role=alert` | - | P0 |
| 29 | shows error toast on failure | `role=alert[data-type="error"]` | - | P0 |
| 30 | navigates back to list after save | URL check | - | P1 |
**File: `tests/proxy/proxy-hosts-bulk.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 31 | selects single host via checkbox | `role=checkbox` | - | P0 |
| 32 | selects all hosts via header checkbox | Header checkbox | - | P0 |
| 33 | shows bulk actions when selected | Bulk action buttons | - | P0 |
| 34 | bulk updates ACL on multiple hosts | `button >> text=Update ACL` | `PUT /proxy-hosts/bulk-update-acl` | P0 |
| 35 | bulk deletes multiple hosts | `button >> text=Delete` | Multiple `DELETE` | P1 |
| 36 | bulk updates security headers | Security headers modal | `PUT /proxy-hosts/bulk-update-security-headers` | P1 |
| 37 | clears selection after bulk action | Checkbox states | - | P1 |
**File: `tests/proxy/proxy-hosts-search-filter.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 38 | filters hosts by domain search | Search input | - | P1 |
| 39 | filters by enabled/disabled status | Status filter | - | P1 |
| 40 | filters by SSL status | SSL filter | - | P2 |
| 41 | sorts by domain name | Column header click | - | P2 |
| 42 | sorts by creation date | Column header click | - | P2 |
| 43 | paginates large host lists | Pagination controls | `GET /proxy-hosts?page=2` | P2 |
#### Day 3: Access Lists CRUD (20-25 tests)
**File: `tests/access-lists/access-lists-crud.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | displays empty state when no ACLs | `[data-testid="empty-state"]` | `GET /access-lists` | P0 |
| 2 | lists all access lists in table | `role=table` | `GET /access-lists` | P0 |
| 3 | shows ACL type badge (Allow/Deny) | `Badge[variant="success"]` | - | P0 |
| 4 | creates IP whitelist | `select#type`, `option[value="whitelist"]` | `POST /access-lists` | P0 |
| 5 | creates IP blacklist | `option[value="blacklist"]` | `POST /access-lists` | P0 |
| 6 | creates geo-whitelist | `option[value="geo_whitelist"]` | `POST /access-lists` | P0 |
| 7 | creates geo-blacklist | `option[value="geo_blacklist"]` | `POST /access-lists` | P0 |
| 8 | validates empty name | `input#name:invalid` | - | P0 |
| 9 | adds single IP rule | IP input, Add button | - | P0 |
| 10 | adds CIDR range rule | `10.0.0.0/24` input | - | P0 |
| 11 | shows IP count for CIDR | IP count badge | - | P1 |
| 12 | removes IP rule | Delete button on rule | - | P0 |
| 13 | validates invalid CIDR | Error message | - | P0 |
| 14 | enables RFC1918 local network only | Toggle switch | - | P1 |
| 15 | Get My IP populates field | `button >> text=Get My IP` | `GET /system/my-ip` | P1 |
| 16 | edits existing ACL | Edit button, form | `PUT /access-lists/:id` | P0 |
| 17 | deletes ACL with backup | Delete, confirm | `DELETE /access-lists/:id` | P0 |
| 18 | toggles ACL enabled/disabled | Enable switch | `PUT /access-lists/:id` | P0 |
| 19 | shows CGNAT warning on first load | Alert component | - | P2 |
| 20 | dismisses CGNAT warning | Dismiss button | - | P2 |
**File: `tests/access-lists/access-lists-geo.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 21 | selects country from list | Country dropdown | - | P0 |
| 22 | adds multiple countries | Country badges | - | P0 |
| 23 | removes country | Badge X button | - | P0 |
| 24 | shows all 40+ countries | Country list | - | P1 |
**File: `tests/access-lists/access-lists-test.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 25 | opens Test IP dialog | TestTube2 icon button | - | P0 |
| 26 | tests allowed IP shows success | Success toast | `POST /access-lists/:id/test` | P0 |
| 27 | tests blocked IP shows error | Error toast | `POST /access-lists/:id/test` | P0 |
| 28 | validates invalid IP input | Error message | - | P1 |
#### Day 4-5: Access Lists Integration & Presets (10-15 tests)
**File: `tests/access-lists/access-lists-presets.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | shows presets section for blacklist | Presets toggle | - | P1 |
| 2 | hides presets for whitelist | - | - | P1 |
| 3 | applies security preset | Apply button | - | P1 |
| 4 | shows preset warning | Warning icon | - | P2 |
| 5 | shows data source link | External link | - | P2 |
**File: `tests/access-lists/access-lists-integration.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 6 | assigns ACL to proxy host | ACL selector | `PUT /proxy-hosts/:uuid` | P0 |
| 7 | shows only enabled ACLs in selector | Dropdown options | `GET /access-lists` | P0 |
| 8 | bulk assigns ACL to hosts | Bulk ACL modal | `PUT /proxy-hosts/bulk-update-acl` | P0 |
| 9 | removes ACL from proxy host | "No Access Control" | `PUT /proxy-hosts/:uuid` | P0 |
| 10 | warns when deleting ACL in use | Warning dialog | - | P1 |
### Week 5: SSL Certificates (Days 6-10)
#### Day 6-7: Certificate List & Upload (25-30 tests)
**File: `tests/certificates/certificates-list.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | displays empty state when no certs | Empty state | `GET /certificates` | P0 |
| 2 | lists all certificates | Table rows | `GET /certificates` | P0 |
| 3 | shows certificate details | Name, domain, expiry | - | P0 |
| 4 | shows status badge (valid) | `Badge[variant="success"]` | - | P0 |
| 5 | shows status badge (expiring) | `Badge[variant="warning"]` | - | P0 |
| 6 | shows status badge (expired) | `Badge[variant="error"]` | - | P0 |
| 7 | sorts by name column | Header click | - | P1 |
| 8 | sorts by expiry date | Header click | - | P1 |
| 9 | shows associated proxy hosts | Host count/badges | - | P2 |
**File: `tests/certificates/certificates-upload.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 10 | opens upload modal | `button >> text=Add Certificate` | - | P0 |
| 11 | uploads valid cert and key | File inputs | `POST /certificates` (multipart) | P0 |
| 12 | validates PEM format | Error on invalid | - | P0 |
| 13 | rejects mismatched cert/key | Error toast | - | P0 |
| 14 | rejects expired certificate | Error toast | - | P1 |
| 15 | shows upload progress | Progress indicator | - | P2 |
| 16 | closes modal after success | Modal hidden | - | P1 |
| 17 | shows success toast | `role=alert` | - | P0 |
| 18 | deletes certificate | Delete button | `DELETE /certificates/:id` | P0 |
| 19 | shows delete confirmation | Confirm dialog | - | P0 |
| 20 | creates backup before delete | Backup API | `POST /backups` | P1 |
**File: `tests/certificates/certificates-validation.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 21 | rejects empty name | Validation error | - | P0 |
| 22 | rejects missing cert file | Required error | - | P0 |
| 23 | rejects missing key file | Required error | - | P0 |
| 24 | rejects self-signed (if configured) | Warning/Error | - | P2 |
| 25 | handles network error gracefully | Error toast | - | P1 |
#### Day 8-9: ACME Certificates (15-20 tests)
**File: `tests/certificates/certificates-acme.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | shows ACME certificate info | Let's Encrypt badge | - | P0 |
| 2 | displays HTTP-01 challenge type | Challenge type indicator | - | P1 |
| 3 | displays DNS-01 challenge type | Challenge type indicator | - | P1 |
| 4 | shows certificate renewal date | Expiry countdown | - | P0 |
| 5 | shows "Renew Now" for expiring | Renew button visible | - | P1 |
| 6 | hides "Renew Now" for valid | Renew button hidden | - | P1 |
| 7 | displays wildcard indicator | Wildcard badge | - | P1 |
| 8 | shows SAN (multiple domains) | Domain list | - | P2 |
**Note:** Full ACME flow testing requires mocked ACME server (staging.letsencrypt.org) - these tests verify UI behavior with pre-existing ACME certificates.
**File: `tests/certificates/certificates-status.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 9 | dashboard shows certificate stats | CertificateStatusCard | - | P1 |
| 10 | shows valid certificate count | Valid count badge | - | P1 |
| 11 | shows expiring certificate count | Warning count | - | P1 |
| 12 | shows pending certificate count | Pending count | - | P2 |
| 13 | links to certificates page | Card link | - | P2 |
| 14 | progress bar shows coverage | Progress component | - | P2 |
#### Day 10: Certificate Integration & Cleanup (10-15 tests)
**File: `tests/certificates/certificates-integration.spec.ts`**
| # | Test Name | UI Selectors | API Endpoint | Priority |
|---|-----------|-------------|--------------|----------|
| 1 | assigns certificate to proxy host | Certificate selector | `PUT /proxy-hosts/:uuid` | P0 |
| 2 | shows only valid certs in selector | Dropdown filtered | `GET /certificates` | P0 |
| 3 | certificate cleanup dialog on host delete | CertificateCleanupDialog | - | P0 |
| 4 | deletes orphan certs option | Checkbox in dialog | - | P1 |
| 5 | keeps certs option | Default unchecked | - | P1 |
| 6 | shows affected hosts on cert delete | Host list | - | P1 |
| 7 | warns about hosts using certificate | Warning message | - | P1 |
---
### Fixtures Reference
**Proxy Hosts:** `tests/fixtures/proxy-hosts.ts`
- `basicProxyHost` - HTTP proxy to internal service
- `proxyHostWithSSL` - HTTPS with forced SSL
- `proxyHostWithWebSocket` - WebSocket enabled
- `proxyHostFullSecurity` - All security features
- `wildcardProxyHost` - Wildcard domain
- `dockerProxyHost` - From Docker discovery
- `invalidProxyHosts` - Validation test cases (XSS, SQL injection)
**Access Lists:** `tests/fixtures/access-lists.ts`
- `emptyAccessList` - No rules
- `allowOnlyAccessList` - IP whitelist
- `denyOnlyAccessList` - IP blacklist
- `mixedRulesAccessList` - Multiple IP ranges
- `authEnabledAccessList` - With HTTP basic auth
- `ipv6AccessList` - IPv6 ranges
- `invalidACLConfigs` - Validation test cases
**Certificates:** `tests/fixtures/certificates.ts`
- `letsEncryptCertificate` - HTTP-01 ACME
- `multiDomainLetsEncrypt` - SAN certificate
- `wildcardCertificate` - DNS-01 wildcard
- `customCertificateMock` - Uploaded PEM
- `expiredCertificate` - For error testing
- `expiringCertificate` - 25 days to expiry
- `invalidCertificates` - Validation test cases
---
### Acceptance Criteria for Phase 2
**Proxy Hosts (40 tests minimum):**
- [ ] All CRUD operations covered
- [ ] Bulk operations functional
- [ ] Docker discovery integration works
- [ ] Validation prevents all invalid input
- [ ] XSS/SQL injection rejected
**SSL Certificates (30 tests minimum):**
- [ ] List/upload/delete operations covered
- [ ] PEM validation enforced
- [ ] Certificate status displayed correctly
- [ ] Dashboard stats accurate
- [ ] Cleanup dialog handles orphan certs
**Access Lists (25 tests minimum):**
- [ ] All 4 ACL types covered (IP/Geo × Allow/Block)
- [ ] IP/CIDR rule management works
- [ ] Country selection works
- [ ] Test IP feature functional
- [ ] Integration with proxy hosts works
**Overall:**
- [ ] 95+ tests passing
- [ ] <5% flaky test rate
- [ ] All P0 tests complete
- [ ] 90%+ P1 tests complete
- [ ] No hardcoded waits
- [ ] All tests use TestDataManager for cleanup
---
### Phase 3: Security Features (Week 6-7)
**Goal:** Cover all Cerberus security features
#### 3.1 CrowdSec Integration
**Priority:** High
**Estimated Effort:** 4 days
**Test Files:**
**`tests/security/crowdsec-startup.spec.ts`**
**Objective:** Verify CrowdSec container lifecycle and connectivity
Test Scenarios:
- ✅ CrowdSec container starts successfully
- Verify container health check passes
- Verify LAPI (Local API) is accessible
- Verify logs show successful initialization
- ✅ CrowdSec LAPI connection from Charon
- Backend connects to CrowdSec LAPI
- Authentication succeeds
- API health endpoint returns 200
- ✅ CrowdSec bouncer registration
- Charon registers as a bouncer
- Bouncer API key generated
- Bouncer appears in CrowdSec bouncer list
- ✅ CrowdSec graceful shutdown
- Stop CrowdSec container
- Charon handles disconnection gracefully
- No errors in Charon logs
- Restart CrowdSec, Charon reconnects
- ✅ CrowdSec container restart recovery
- Kill CrowdSec container abruptly
- Charon detects connection loss
- Auto-reconnect after CrowdSec restarts
- ✅ CrowdSec version compatibility
- Verify minimum version check
- Warn if CrowdSec version too old
- Block connection if incompatible
**Key Assertions:**
- Container health: `docker inspect crowdsec --format '{{.State.Health.Status}}' == 'healthy'`
- LAPI reachable: `curl http://crowdsec:8080/health` returns 200
- Bouncer registered: API call to `/v1/bouncers` shows Charon
- Logs clean: No error/warning logs after startup
**`tests/security/crowdsec-decisions.spec.ts`**
**Objective:** Test IP ban management and decision enforcement
Test Scenarios:
**Manual IP Ban:**
- ✅ Add IP ban via Charon UI
- Navigate to CrowdSec → Decisions
- Click "Add Decision"
- Enter IP address (e.g., `192.168.1.100`)
- Select ban duration (1h, 4h, 24h, permanent)
- Select scope (IP, range, country)
- Add ban reason (e.g., "Suspicious activity")
- Save decision
- ✅ Verify decision appears in CrowdSec
- Query CrowdSec LAPI `/v1/decisions`
- Verify decision contains correct IP, duration, reason
- ✅ Banned IP cannot access proxy hosts
- Make HTTP request from banned IP
- Verify 403 Forbidden response
- Verify block logged in audit log
**Automatic IP Ban (Scenario-based):**
- ✅ Trigger ban via brute force scenario
- Simulate 10 failed login attempts from IP
- CrowdSec scenario detects pattern
- Decision created automatically
- IP banned for configured duration
- ✅ Trigger ban via HTTP flood scenario
- Send 100 requests/second from IP
- CrowdSec detects flood pattern
- IP banned automatically
**Ban Duration & Expiration:**
- ✅ Temporary ban expires automatically
- Create 5-second ban
- Verify IP blocked immediately
- Wait 6 seconds
- Verify IP can access again
- ✅ Permanent ban persists
- Create permanent ban
- Restart Charon and CrowdSec
- Verify ban still active
- ✅ Manual ban removal
- Select active ban
- Click "Remove Decision"
- Confirm removal
- Verify IP can access immediately
**Ban Scope Testing:**
- ✅ Single IP ban: `192.168.1.100`
- Only that IP blocked
- `192.168.1.101` can access
- ✅ IP range ban: `192.168.1.0/24`
- All IPs in range blocked
- `192.168.2.1` can access
- ✅ Country-level ban: `CN` (China)
- All Chinese IPs blocked
- Uses GeoIP database
- Other countries can access
**Decision Priority & Conflicts:**
- ✅ Allow decision overrides ban decision
- Ban IP range `10.0.0.0/8`
- Allow specific IP `10.0.0.5`
- Verify `10.0.0.5` can access
- Verify `10.0.0.6` is blocked
- ✅ Multiple decisions for same IP
- Add 1-hour ban
- Add 24-hour ban
- Longer duration takes precedence
**Decision Metadata:**
- ✅ Display decision details
- IP/Range/Country
- Ban duration and expiry time
- Scenario that triggered ban
- Reason/origin
- Creation timestamp
- ✅ Decision history
- View past decisions (expired/deleted)
- Filter by IP, scenario, date range
- Export decision history
**Key User Flows:**
1. **Manual Ban Flow:**
```
CrowdSec page → Decisions tab → Add Decision →
Enter IP → Select duration → Save → Verify block
```
2. **Automatic Ban Flow:**
```
Trigger scenario (e.g., brute force) →
CrowdSec detects → Decision created →
View in Decisions tab → Verify block
```
3. **Ban Removal Flow:**
```
CrowdSec page → Decisions tab → Select decision →
Remove → Confirm → Verify access restored
```
**Critical Assertions:**
- Banned IPs receive 403 status code
- Decision sync between Charon and CrowdSec
- Expiration timing accurate (±5 seconds)
- Allow decisions override ban decisions
- Decision changes appear in audit log
**`tests/security/crowdsec-presets.spec.ts`**
**Objective:** Test CrowdSec scenario preset management
Test Scenarios:
**Preset Listing:**
- ✅ View available presets
- Navigate to CrowdSec → Presets
- Display preset categories (Web, SSH, System)
- Show preset descriptions
- Indicate enabled/disabled status
**Enable/Disable Presets:**
- ✅ Enable web attack preset
- Select "Web Attacks" preset
- Click "Enable"
- Verify scenarios installed in CrowdSec
- Verify collection appears in CrowdSec collections list
- ✅ Disable web attack preset
- Select enabled preset
- Click "Disable"
- Verify scenarios removed
- Existing decisions preserved
- ✅ Bulk enable multiple presets
- Select multiple presets
- Click "Enable Selected"
- All scenarios installed
**Custom Scenarios:**
- ✅ Create custom scenario
- Click "Add Custom Scenario"
- Enter scenario name (e.g., "api-abuse")
- Define pattern (e.g., 50 requests to /api in 10s)
- Set ban duration (e.g., 1h)
- Save scenario
- Verify scenario YAML created
- ✅ Test custom scenario
- Trigger scenario conditions
- Verify decision created
- Verify ban enforced
- ✅ Edit custom scenario
- Modify pattern thresholds
- Save changes
- Reload CrowdSec scenarios
- ✅ Delete custom scenario
- Select scenario
- Confirm deletion
- Scenario removed from CrowdSec
**Preset Configuration:**
- ✅ Configure scenario thresholds
- Select scenario (e.g., "http-bf" brute force)
- Modify threshold (e.g., 5 → 10 failed attempts)
- Modify time window (e.g., 30s → 60s)
- Save configuration
- Verify new thresholds apply
- ✅ Configure ban duration per scenario
- Different scenarios have different ban times
- Brute force: 4h ban
- Port scan: 24h ban
- Verify durations respected
**Scenario Testing & Validation:**
- ✅ Test scenario before enabling
- View scenario details
- Simulate trigger conditions in test mode
- Verify pattern matching works
- No actual bans created (dry-run)
- ✅ Scenario validation on save
- Invalid regex pattern rejected
- Impossible thresholds rejected (e.g., 0 requests)
- Missing required fields flagged
**Key User Flows:**
1. **Enable Preset Flow:**
```
CrowdSec page → Presets tab → Select preset →
Enable → Verify scenarios active
```
2. **Custom Scenario Flow:**
```
CrowdSec page → Presets tab → Add Custom →
Define pattern → Set duration → Save → Test trigger
```
3. **Configure Scenario Flow:**
```
CrowdSec page → Presets tab → Select scenario →
Edit thresholds → Save → Reload CrowdSec
```
**Critical Assertions:**
- Enabled scenarios appear in CrowdSec collections
- Scenario triggers create correct decisions
- Custom scenarios persist after restart
- Threshold changes take effect immediately
- Invalid scenarios rejected with clear error messages
#### 3.2 Coraza WAF (Web Application Firewall)
**Priority:** High
**Estimated Effort:** 3 days
**Test Files:**
**`tests/security/waf-config.spec.ts`**
Test Scenarios:
- ✅ Enable/disable WAF globally
- ✅ Configure WAF for specific proxy host
- ✅ Set WAF rule sets (OWASP Core Rule Set)
- ✅ Configure anomaly scoring thresholds
- ✅ Set blocking/logging mode
- ✅ Custom rule creation
- ✅ Rule exclusions (false positive handling)
**`tests/security/waf-blocking.spec.ts`**
Test Scenarios:
- ✅ Block SQL injection attempts
- Send request with `' OR 1=1--` in query
- Verify 403 response
- Verify attack logged
- ✅ Block XSS attempts
- Send request with `<script>alert('xss')</script>`
- Verify 403 response
- ✅ Block path traversal attempts
- Send request with `../../etc/passwd`
- Verify 403 response
- ✅ Block command injection attempts
- ✅ Block file upload attacks
- ✅ Allow legitimate requests
- Normal user traffic passes through
- No false positives
**Key Assertions:**
- Malicious requests blocked (403 status)
- Legitimate requests allowed (200/3xx status)
- Attacks logged in audit log with details
- WAF performance overhead <50ms
#### 3.3 Rate Limiting
**Priority:** High
**Estimated Effort:** 2 days
### Phase 4: Settings (Week 8)
**Goal:** Cover system configuration and user management
**Estimated Effort:** 5 days
**Test Files:**
- `tests/settings/system-settings.spec.ts` - System configuration
- `tests/settings/smtp-settings.spec.ts` - Email configuration
- `tests/settings/notifications.spec.ts` - Notification rules
- `tests/settings/user-management.spec.ts` - User CRUD and roles
- `tests/settings/encryption-management.spec.ts` - Encryption key rotation
- `tests/settings/account-settings.spec.ts` - User profile management
**Key Features:**
- System configuration (timezone, language, theme)
- Email settings (SMTP, templates)
- Notification rules (email, webhook)
- User management (CRUD, roles, permissions)
- Encryption management (key rotation, backup)
- Account settings (profile, password, 2FA)
### Phase 5: Tasks (Week 9)
**Goal:** Cover backup, logs, and monitoring features
**Estimated Effort:** 5 days
**Test Files:**
- `tests/tasks/backups-create.spec.ts` - Backup creation
- `tests/tasks/backups-restore.spec.ts` - Backup restoration
- `tests/tasks/logs-viewing.spec.ts` - Log viewer functionality
- `tests/tasks/import-caddyfile.spec.ts` - Caddyfile import
- `tests/tasks/import-crowdsec.spec.ts` - CrowdSec config import
- `tests/monitoring/uptime-monitoring.spec.ts` - Uptime checks
- `tests/monitoring/real-time-logs.spec.ts` - WebSocket log streaming
**Key Features:**
- Backup creation (manual, scheduled)
- Backup restoration (full, selective)
- Log viewing (filtering, search, export)
- Caddyfile import (validation, migration)
- CrowdSec import (scenarios, decisions)
- Uptime monitoring (HTTP checks, alerts)
- Real-time logs (WebSocket, filtering)
### Phase 6: Integration & Buffer (Week 10)
**Goal:** Test cross-feature interactions, edge cases, and provide buffer for overruns
**Estimated Effort:** 5 days (3 days testing + 2 days buffer)
**Test Files:**
- `tests/integration/proxy-acl-integration.spec.ts` - Proxy + ACL
- `tests/integration/proxy-certificate.spec.ts` - Proxy + SSL
- `tests/integration/security-suite-integration.spec.ts` - Full security stack
- `tests/integration/backup-restore-e2e.spec.ts` - Full backup cycle
**Key Scenarios:**
- Create proxy host with ACL and SSL certificate
- Test security stack: WAF + CrowdSec + Rate Limiting
- Full backup → Restore → Verify all data intact
- Multi-feature workflows (e.g., import Caddyfile + enable security)
**Buffer Time:**
- Address flaky tests discovered in previous phases
- Fix any infrastructure issues
- Improve test stability and reliability
- Documentation updates
---
## Success Metrics & Acceptance Criteria
### Coverage Goals
**Test Coverage Targets:**
- 🎯 **Core Features:** 100% coverage (auth, navigation, dashboard)
- 🎯 **Critical Features:** 100% coverage (proxy hosts, ACLs, certificates)
- 🎯 **High Priority Features:** 90% coverage (security suite, backups)
- 🎯 **Medium Priority Features:** 80% coverage (settings, monitoring)
- 🎯 **Nice-to-Have Features:** 70% coverage (imports, plugins)
**Feature Coverage Matrix:**
| Feature | Priority | Target Coverage | Test Files | Status |
|---------|----------|----------------|------------|--------|
| Authentication | Critical | 100% | 1 | ✅ Covered (94% - 3 session tests deferred) |
| Dashboard | Core | 100% | 1 | ✅ Covered |
| Navigation | Core | 100% | 1 | ✅ Covered |
| Proxy Hosts | Critical | 100% | 3 | ❌ Not started |
| Certificates | Critical | 100% | 3 | ❌ Not started |
| Access Lists | Critical | 100% | 2 | ❌ Not started |
| CrowdSec | High | 90% | 3 | ❌ Not started |
| WAF | High | 90% | 1 | ❌ Not started |
| Rate Limiting | High | 90% | 1 | ❌ Not started |
| Security Headers | Medium | 80% | 1 | ❌ Not started |
| Audit Logs | Medium | 80% | 1 | ❌ Not started |
| Backups | High | 90% | 2 | ❌ Not started |
| Users | High | 90% | 2 | ❌ Not started |
| Settings | Medium | 80% | 4 | ❌ Not started |
| Monitoring | Medium | 80% | 3 | ❌ Not started |
| Import/Export | Medium | 80% | 2 | ❌ Not started |
| DNS Providers | Critical | 100% | 4 | ✅ Covered |
---
## Next Steps
1. ~~**Review and Approve Plan:** Stakeholder sign-off~~
2. ~~**Set Up Test Infrastructure:** Fixtures, utilities, CI configuration~~
3. ~~**Begin Phase 1 Implementation:** Foundation tests~~
4. **Begin Phase 2 Implementation:** Critical Path (Proxy Hosts, Certificates, ACLs)
5. **Fix Session Expiration Tests:** See [docs/issues/e2e-session-expiration-tests.md](../issues/e2e-session-expiration-tests.md)
6. **Daily Standup Check-ins:** Progress tracking, blocker resolution
7. **Weekly Demo:** Show completed test coverage
---
**Document Status:** In Progress - Phase 1 Complete
**Last Updated:** January 17, 2026
**Phase 1 Completed:** January 17, 2026 (112/119 tests passing - 94%)
**Next Review:** Upon Phase 2 completion (estimated Jan 31, 2026)
**Owner:** Planning Agent / QA Team