# CI Flake Triage Implementation - Frontend_Dev

**Date**: January 26, 2026
**Feature Branch**: feature/beta-release
**Focus**: Playwright/tests and global setup (not app UI)

## Summary

Implemented deterministic fixes for CI flakes in Playwright E2E tests, focusing on health checks, ACL reset verification, shared helpers, and shard-specific improvements.

## Changes Made

### 1. Global Setup - Health Probes & Deterministic ACL Disable

**File**: `tests/global-setup.ts`

**Changes**:
- Added `checkEmergencyServerHealth()` function to probe `http://localhost:2019/config` with 3s timeout
- Added `checkTier2ServerHealth()` function to probe `http://localhost:2020/health` with 3s timeout
- Both health checks are non-blocking (skip if unavailable, don't fail setup)
- Added URL analysis logging (IPv4 vs IPv6, localhost detection) for debugging cookie domain issues
- Implemented `verifySecurityDisabled()` with 2-attempt retry and fail-fast:
  - Checks `/api/v1/security/config` for ACL and rate-limit state
  - Retries emergency reset once if still enabled
  - Fails with actionable error if security remains enabled after retry
- Logs include emojis for easy scanning in CI output

**Rationale**: Emergency and tier-2 servers are optional; tests should skip gracefully if unavailable. ACL/rate-limit must be disabled deterministically or tests fail with clear diagnostics.

### 2. TestDataManager - ACL Safety Check

**File**: `tests/utils/TestDataManager.ts`

**Changes**:
- Added `assertSecurityDisabled()` method
- Checks `/api/v1/security/config` before operations
- Throws actionable error if ACL or rate-limit is enabled
- Idempotent: skips check if endpoint unavailable (no-op in environments without endpoint)

**Usage**:
```typescript
await testData.assertSecurityDisabled(); // Before creating resources
const host = await testData.createProxyHost(config);
```

**Rationale**: Fail-fast with clear error when security is blocking operations, rather than cryptic 403 errors.

### 3. Shared UI Helpers

**File**: `tests/utils/ui-helpers.ts` (new)

**Helpers Created**:

#### `getToastLocator(page, text?, options)`
- Uses `data-testid="toast-{type}"` for role-based selection
- Avoids strict-mode violations with `.first()`
- Short retry timeout (default 5s)
- Filters by text if provided

#### `waitForToast(page, text, options)`
- Wrapper around `getToastLocator` with built-in wait
- Replaces `page.locator('[data-testid="toast-success"]').first()` pattern

#### `getRowScopedButton(page, rowIdentifier, buttonName, options)`
- Finds button within specific table row
- Avoids strict-mode collisions when multiple rows have same button
- Example: Find "Resend" button in row containing "user@example.com"

#### `getRowScopedIconButton(page, rowIdentifier, iconClass)`
- Finds button by icon class (e.g., `lucide-mail`) within row
- Fallback for buttons without proper accessible names

#### `getCertificateValidationMessage(page, messagePattern)`
- Targets validation message with proper role (`alert`, `status`) or error class
- Avoids brittle `getByText()` that can match unrelated elements

#### `refreshListAndWait(page, options)`
- Reloads page and waits for table to stabilize
- Ensures list reflects changes after create/update operations

**Rationale**: DRY principle, consistent locator strategies, avoid strict-mode violations, improve test reliability.

### 4. Shard 1 Fixes - DNS Provider CRUD

**File**: `tests/dns-provider-crud.spec.ts`

**Changes**:
- Imported `getToastLocator` and `refreshListAndWait` from `ui-helpers`
- Updated "Manual DNS provider" test:
  - Replaced raw toast locator with `getToastLocator(page, /success|created/i, { type: 'success' })`
  - Added `refreshListAndWait(page)` after create to ensure list updates
- Updated "Webhook DNS provider" test:
  - Replaced raw toast locator with `getToastLocator`
- Updated "Update provider name" test:
  - Replaced raw toast locator with `getToastLocator`

**Rationale**: Toast helper reduces duplication and ensures consistent detection. Refresh ensures provider appears in list after creation.

### 5. Shard 2 Fixes - Emergency & Tier-2 Tests

**File**: `tests/emergency-server/emergency-server.spec.ts`

**Changes**:
- Added `checkEmergencyServerHealth()` function
- Added `test.beforeAll()` hook to check health before suite
- Skips entire suite if emergency server unavailable (port 2019)

**File**: `tests/emergency-server/tier2-validation.spec.ts`

**Changes**:
- Added `test.beforeAll()` hook to check tier-2 health (port 2020)
- Skips entire suite if tier-2 server unavailable
- Logs health check result for CI visibility

**Rationale**: Emergency and tier-2 servers are optional. Tests should skip gracefully rather than hang or timeout.

### 6. Shard 3 Fixes - Certificate Email Validation

**File**: `tests/settings/account-settings.spec.ts`

**Changes**:
- Imported `getCertificateValidationMessage` from `ui-helpers`
- Updated "Validate certificate email format" test:
  - Replaced `page.getByText(/invalid.*email|email.*invalid/i)` with `getCertificateValidationMessage(page, /invalid.*email|email.*invalid/i)`
  - Targets visible validation message with proper role/text

**Rationale**: Brittle `getByText` can match unrelated elements. Helper targets proper validation message role.

### 7. Shard 4 Fixes - System Settings & User Management

**File**: `tests/settings/system-settings.spec.ts`

**Changes**:
- Imported `getToastLocator` from `ui-helpers`
- Updated 3 toast locators:
  - "Save general settings" test: success toast
  - "Show error for unreachable URL" test: error toast
  - "Update public URL setting" test: success toast
- Replaced complex `.or()` chains with single `getToastLocator` call

**File**: `tests/settings/user-management.spec.ts`

**Changes**:
- Imported `getRowScopedButton` and `getRowScopedIconButton` from `ui-helpers`
- Updated "Resend invite" test:
  - Replaced `page.getByRole('button', { name: /resend invite/i }).first()` with `getRowScopedButton(page, testEmail, /resend invite/i)`
  - Added fallback to `getRowScopedIconButton(page, testEmail, 'lucide-mail')` for icon-only buttons
  - Avoids strict-mode violations when multiple pending users exist

**Rationale**: Row-scoped helpers avoid strict-mode violations in parallel tests. Toast helper ensures consistent detection.

## Files Changed (7 files)

1. `tests/global-setup.ts` - Health probes, URL analysis, ACL verification
2. `tests/utils/TestDataManager.ts` - ACL safety check
3. `tests/utils/ui-helpers.ts` - NEW: Shared helpers
4. `tests/dns-provider-crud.spec.ts` - Toast helper, refresh list
5. `tests/emergency-server/emergency-server.spec.ts` - Health check, skip if unavailable
6. `tests/emergency-server/tier2-validation.spec.ts` - Health check, skip if unavailable
7. `tests/settings/account-settings.spec.ts` - Certificate validation helper
8. `tests/settings/system-settings.spec.ts` - Toast helper (3 usages)
9. `tests/settings/user-management.spec.ts` - Row-scoped button helpers

## Observability

### Global Setup Logs (Non-secret)

Example output:
```
🧹 Running global test setup...
📍 Base URL: http://localhost:8080
  🔍 URL Analysis: host=localhost port=8080 IPv6=false localhost=true
🔍 Checking emergency server health at http://localhost:2019...
  ✅ Emergency server (port 2019) is healthy
🔍 Checking tier-2 server health at http://localhost:2020...
  ⏭️  Tier-2 server unavailable (tests will skip tier-2 features)
⏭️  Pre-auth security reset skipped (fresh container, no custom token)
🧹 Cleaning up orphaned test data...
   No orphaned test data found
✅ Global setup complete

🔓 Performing emergency security reset...
  ✅ Emergency reset successful
  ✅ Disabled modules: security.acl.enabled, security.waf.enabled, security.rate_limit.enabled
  ⏳ Waiting for security reset to propagate...
  ✅ Security reset complete
✓ Authenticated security reset complete

🔒 Verifying security modules are disabled...
  ✅ Security modules confirmed disabled
```

### Emergency/Tier-2 Health Checks

Each shard logs its health check:
```
🔍 Checking emergency server health before tests...
✅ Emergency server is healthy
```

Or:
```
🔍 Checking tier-2 server health before tests...
❌ Tier-2 server is unavailable: connect ECONNREFUSED
[Suite skipped]
```

### ACL State Per Project

Logged in TestDataManager when `assertSecurityDisabled()` is called:
```
❌ SECURITY MODULES ARE ENABLED - OPERATION WILL FAIL
   ACL: true, Rate Limiting: true
   Cannot proceed with resource creation.
   Check: global-setup.ts emergency reset completed successfully
```

## Not Implemented (Per Task)

- **Coverage/Vite**: Not re-enabled (remains disabled per task 5)
- **Security tests**: Remain disabled (per task 5)
- **Backend changes**: None made (per task constraint)

## Test Execution

**Recommended**:
```bash
# Run specific shard for quick validation
npx playwright test tests/dns-provider-crud.spec.ts --project=chromium

# Or run full suite
npx playwright test --project=chromium
```

**Not executed** in this session due to time constraints. Recommend running focused tests on relevant shards to validate:
- Shard 1: `tests/dns-provider-crud.spec.ts`
- Shard 2: `tests/emergency-server/emergency-server.spec.ts`
- Shard 3: `tests/settings/account-settings.spec.ts` (certificate email validation test)
- Shard 4: `tests/settings/system-settings.spec.ts`, `tests/settings/user-management.spec.ts`

## Design Decisions

1. **Health Checks**: Non-blocking, 3s timeout, graceful skip if unavailable
2. **ACL Verification**: 2-attempt retry with fail-fast and actionable error
3. **Shared Helpers**: DRY principle, consistent patterns, avoid strict-mode
4. **Row-Scoped Locators**: Prevent strict-mode violations in parallel tests
5. **Observability**: Emoji-rich logs for easy CI scanning (no secrets logged)

## Next Steps (Optional)

1. Run Playwright tests per shard to validate changes
2. Monitor CI runs for reduced flake rate
3. Consider extracting health check logic to a separate utility module if reused elsewhere
4. Add more row-scoped helpers if other tests need similar patterns

## References

- Plan: `docs/plans/current_spec.md` (CI flake triage section)
- Playwright docs: https://playwright.dev/docs/best-practices
- Object Calisthenics: `docs/.github/instructions/object-calisthenics.instructions.md`
- Testing protocols: `docs/.github/instructions/testing.instructions.md`