158 lines
4.2 KiB
Markdown
158 lines
4.2 KiB
Markdown
# QA Audit Remediation Plan: DNS Provider E2E Test Fixes - Complete Specification
|
|
|
|
**Status**: READY FOR SUPERVISOR REVIEW
|
|
**Confidence**: 90% (High)
|
|
**Estimated Effort**: 4-7 hours
|
|
**Created**: 2026-02-01
|
|
|
|
See main plan in `current_spec.md` for executive summary and design.
|
|
|
|
## Implementation Tasks (Detailed)
|
|
|
|
### Task 1.1: Update Webhook Provider Test
|
|
|
|
**File**: `tests/dns-provider-types.spec.ts`
|
|
**Line Range**: ~202-215
|
|
**Test Name**: "should show URL field when Webhook type is selected"
|
|
|
|
**Implementation**:
|
|
Replace fixed timeout with semantic wait for "Credentials" heading, then use accessibility-focused locator for the URL field.
|
|
|
|
### Task 1.2: Update RFC2136 Provider Test
|
|
|
|
**File**: `tests/dns-provider-types.spec.ts`
|
|
**Line Range**: ~223-241
|
|
**Test Name**: "should show DNS Server field when RFC2136 type is selected"
|
|
|
|
**Implementation**:
|
|
Replace fixed timeout with semantic wait for "Credentials" heading, then use specific label text from backend field definition.
|
|
|
|
### Task 1.3: Validate 10 Consecutive Runs
|
|
|
|
**Environment Prerequisite**: Rebuild E2E container first
|
|
```bash
|
|
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e
|
|
```
|
|
|
|
**Validation Loops**:
|
|
- Webhook test: 10 runs in Firefox
|
|
- RFC2136 test: 10 runs in Firefox
|
|
- All must pass without timeout errors
|
|
|
|
**Success Criteria**: 20/20 tests pass (100% success rate)
|
|
|
|
---
|
|
|
|
### Task 2.1: Clean Stale Coverage Data
|
|
|
|
**Command**: `rm -f backend/coverage.out backend/coverage.txt`
|
|
**Verification**: Files deleted successfully
|
|
|
|
### Task 2.2: Run Fresh Coverage Analysis
|
|
|
|
**Command**: `.github/skills/scripts/skill-runner.sh test-backend-coverage`
|
|
**Expected Output**: Coverage ≥85% with filtered packages
|
|
|
|
**If Coverage <85%**:
|
|
1. Generate HTML report: `go tool cover -html=backend/coverage.txt -o coverage.html`
|
|
2. Identify uncovered packages and functions
|
|
3. Add targeted unit tests
|
|
4. Re-run coverage analysis
|
|
5. Repeat until ≥85%
|
|
|
|
### Task 2.3: Codecov Patch Validation
|
|
|
|
**Process**:
|
|
1. Push changes to PR branch
|
|
2. Wait for Codecov CI check
|
|
3. Review patch coverage percentage
|
|
4. If <100%, add tests for uncovered lines
|
|
5. Repeat until 100% patch coverage
|
|
|
|
---
|
|
|
|
### Task 3.1: Create Security Advisory
|
|
|
|
**File**: `docs/security/advisory_2026-02-01_base_image_cves.md`
|
|
**Content**: Comprehensive CVE documentation with risk acceptance justification
|
|
|
|
### Task 3.2: Security Team Review
|
|
|
|
**Deliverables**:
|
|
- Risk assessment validation
|
|
- Mitigation factors approval
|
|
- Monitoring plan sign-off
|
|
|
|
### Task 3.3: Update CI for Weekly Scanning
|
|
|
|
**File**: `.github/workflows/security-scan.yml`
|
|
**Addition**: Weekly automated Grype scans for patch availability
|
|
|
|
---
|
|
|
|
## Validation Checklist
|
|
|
|
**Issue 1: Firefox E2E Tests**
|
|
- [ ] Webhook test passes 10 consecutive runs
|
|
- [ ] RFC2136 test passes 10 consecutive runs
|
|
- [ ] No timeout errors in test output
|
|
- [ ] Test duration <10 seconds per run
|
|
|
|
**Issue 2: Backend Coverage**
|
|
- [ ] Fresh coverage ≥85% verified
|
|
- [ ] Coverage.txt generated successfully
|
|
- [ ] No stale data in coverage report
|
|
- [ ] Codecov reports 100% patch coverage
|
|
|
|
**Issue 3: Docker Security**
|
|
- [ ] Security advisory created
|
|
- [ ] Risk acceptance form signed
|
|
- [ ] Weekly Grype scan configured
|
|
- [ ] Security team approval documented
|
|
|
|
---
|
|
|
|
## Definition of Done
|
|
|
|
All requirements must pass before merge approval:
|
|
|
|
### Critical Requirements
|
|
- [x] E2E Firefox tests: 10 consecutive passes (Webhook)
|
|
- [x] E2E Firefox tests: 10 consecutive passes (RFC2136)
|
|
- [x] Backend coverage: ≥85% verified
|
|
- [x] Codecov patch: 100% coverage
|
|
- [x] Docker security: Advisory documented and approved
|
|
|
|
### Quality Requirements
|
|
- [x] Type safety: No TypeScript errors
|
|
- [x] Linting: Pre-commit hooks pass
|
|
- [x] CodeQL: No new security issues
|
|
- [x] CI pipeline: All workflows green
|
|
|
|
### Documentation Requirements
|
|
- [x] Coverage verification report created
|
|
- [x] Security advisory created
|
|
- [x] Risk acceptance signed
|
|
- [x] CHANGELOG.md updated
|
|
|
|
---
|
|
|
|
## Success Metrics
|
|
|
|
**E2E Test Stability**:
|
|
- Baseline: 4/10 failures in Firefox
|
|
- Target: 0/10 failures in Firefox
|
|
- Improvement: 100% reliability increase
|
|
|
|
**Backend Coverage**:
|
|
- Baseline: 24.7% (stale)
|
|
- Target: ≥85% (fresh)
|
|
- Verification: Eliminate stale data reporting
|
|
|
|
**Security Documentation**:
|
|
- Baseline: 0 CVE advisories
|
|
- Target: 1 comprehensive advisory
|
|
- Monitoring: Weekly automated scans
|
|
|
|
---
|