akanealw/Charon
1
0
Fork 0
Code Issues Pull Requests Actions 14 Packages Projects Releases Wiki Activity

fix(integration): migrate wget-style curl syntax for Debian compatibility

Browse Source 
After migrating base image from Alpine to Debian Trixie (PR #550),
integration test scripts were using wget-style options with curl
that don't work correctly on Debian.

Changed curl -q -O- (wget syntax) to curl -sf (proper curl):

waf_integration.sh
cerberus_integration.sh
rate_limit_integration.sh
crowdsec_startup_test.sh
install-go-1.25.5.sh
Also added future phase to plan for Playwright security test helpers
to prevent ACL deadlock issues during E2E testing.

Refs: #550
This commit is contained in:
GitHub Actions
2026-01-25 09:17:50 +00:00
parent aa74d37a3a
commit a41cfaae10
12 changed files with 866 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

288
docs/plans/current_spec.md
View File

@@ -1,7 +1,215 @@
# Git & Workflow Recovery Plan
# WAF Integration Workflow Fix: wget-style curl Syntax Migration
**Plan ID**: WAF-2026-001
**Status**: 📋 PENDING
**Priority**: High
**Created**: 2026-01-25
**Scope**: Fix integration test scripts using incorrect wget-style curl syntax
---
## Problem Summary
After migrating the Docker base image from Alpine to Debian Trixie (PR #550), the WAF integration workflow is failing. The root cause is **not** a missing `wget` command, but rather several integration test scripts using **wget-style options with curl** that don't work correctly.
### Root Cause
Multiple scripts use `curl -q -O-` which is **wget syntax, not curl syntax**:
| Syntax | Tool | Meaning |
|--------|------|---------|
| `-q` | **wget** | Quiet mode |
| `-q` | **curl** | **Invalid** - does nothing useful |
| `-O-` | **wget** | Output to stdout |
| `-O-` | **curl** | **Wrong** - `-O` means "save with remote filename", `-` is treated as a separate URL |
The correct curl equivalents are:
| wget | curl | Notes |
|------|------|-------|
| `wget -q` | `curl -s` | Silent mode |
| `wget -O-` | `curl -s` | stdout is curl's default output |
| `wget -q -O- URL` | `curl -s URL` | Full equivalent |
| `wget -O filename` | `curl -o filename` | Note: lowercase `-o` in curl |
---
## Files Requiring Changes
### Priority 1: Integration Test Scripts (Blocking WAF Workflow)
| File | Line | Current Code | Issue |
|------|------|--------------|-------|
| [scripts/waf_integration.sh](../../scripts/waf_integration.sh#L205) | 205 | `curl -q -O- http://${BACKEND_CONTAINER}/get` | wget syntax |
| [scripts/cerberus_integration.sh](../../scripts/cerberus_integration.sh#L214) | 214 | `curl -q -O- http://${BACKEND_CONTAINER}/get` | wget syntax |
| [scripts/rate_limit_integration.sh](../../scripts/rate_limit_integration.sh#L190) | 190 | `curl -q -O- http://${BACKEND_CONTAINER}/get` | wget syntax |
| [scripts/crowdsec_startup_test.sh](../../scripts/crowdsec_startup_test.sh#L178) | 178 | `curl -q -O- http://127.0.0.1:8085/health` | wget syntax |
### Priority 2: Utility Scripts
| File | Line | Current Code | Issue |
|------|------|--------------|-------|
| [scripts/install-go-1.25.5.sh](../../scripts/install-go-1.25.5.sh#L18) | 18 | `curl -q -O "$TMPFILE" "URL"` | Wrong syntax - `-O` doesn't take an argument in curl |
---
## Detailed Fixes
### Fix 1: scripts/waf_integration.sh (Line 205)
**Current (broken):**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -q -O- http://${BACKEND_CONTAINER}/get 2>/dev/null || curl -s http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
**Fixed:**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -sf http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
**Notes:**
- `-s` = silent (no progress meter)
- `-f` = fail silently on HTTP errors (returns non-zero exit code)
- Removed redundant fallback since the fix makes the command work correctly
---
### Fix 2: scripts/cerberus_integration.sh (Line 214)
**Current (broken):**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -q -O- http://${BACKEND_CONTAINER}/get 2>/dev/null || curl -s http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
**Fixed:**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -sf http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
---
### Fix 3: scripts/rate_limit_integration.sh (Line 190)
**Current (broken):**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -q -O- http://${BACKEND_CONTAINER}/get 2>/dev/null || curl -s http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
**Fixed:**
```bash
if docker exec ${CONTAINER_NAME} sh -c "curl -sf http://${BACKEND_CONTAINER}/get" >/dev/null 2>&1; then
```
---
### Fix 4: scripts/crowdsec_startup_test.sh (Line 178)
**Current (broken):**
```bash
LAPI_HEALTH=$(docker exec ${CONTAINER_NAME} curl -q -O- http://127.0.0.1:8085/health 2>/dev/null || echo "FAILED")
```
**Fixed:**
```bash
LAPI_HEALTH=$(docker exec ${CONTAINER_NAME} curl -sf http://127.0.0.1:8085/health 2>/dev/null || echo "FAILED")
```
---
### Fix 5: scripts/install-go-1.25.5.sh (Line 18)
**Current (broken):**
```bash
curl -q -O "$TMPFILE" "https://go.dev/dl/${TARFILE}"
```
**Fixed:**
```bash
curl -sSfL -o "$TMPFILE" "https://go.dev/dl/${TARFILE}"
```
**Notes:**
- `-s` = silent
- `-S` = show errors even in silent mode
- `-f` = fail on HTTP errors
- `-L` = follow redirects (important for go.dev downloads)
- `-o filename` = output to specified file (lowercase `-o`)
---
## Verification Commands
After applying fixes, verify each script works:
```bash
# Test WAF integration
./scripts/waf_integration.sh
# Test Cerberus integration
./scripts/cerberus_integration.sh
# Test Rate Limit integration
./scripts/rate_limit_integration.sh
# Test CrowdSec startup
./scripts/crowdsec_startup_test.sh
# Verify Go install script syntax
bash -n ./scripts/install-go-1.25.5.sh
```
---
## Behavior Differences: wget vs curl
When migrating from wget to curl, be aware of these differences:
| Behavior | wget | curl |
|----------|------|------|
| Output destination | File by default | stdout by default |
| Follow redirects | Yes by default | Requires `-L` flag |
| Retry on failure | Built-in retry | Requires `--retry N` |
| Progress display | Text progress bar | Progress meter (use `-s` to hide) |
| HTTP error handling | Non-zero exit on 404 | Requires `-f` for non-zero exit on HTTP errors |
| Quiet mode | `-q` | `-s` (silent) |
| Output to file | `-O filename` (uppercase) | `-o filename` (lowercase) |
| Save with remote name | `-O` (no arg) | `-O` (uppercase, no arg) |
---
## Execution Checklist
- [ ] **Fix 1**: Update `scripts/waf_integration.sh` line 205
- [ ] **Fix 2**: Update `scripts/cerberus_integration.sh` line 214
- [ ] **Fix 3**: Update `scripts/rate_limit_integration.sh` line 190
- [ ] **Fix 4**: Update `scripts/crowdsec_startup_test.sh` line 178
- [ ] **Fix 5**: Update `scripts/install-go-1.25.5.sh` line 18
- [ ] **Verify**: Run each integration test locally
- [ ] **CI**: Confirm WAF integration workflow passes
---
## Notes
1. **Deprecated Scripts**: Several affected scripts are marked deprecated (will be removed in v2.0.0). However, they are still used by CI workflows, so fixes are required.
2. **Skill-Based Replacements**: The `.github/skills/scripts/` directory was checked and contains no wget usage - those scripts already use correct curl syntax.
3. **Docker Compose Files**: All health checks in docker-compose files already use correct curl syntax (`curl -f`, `curl -fsS`).
4. **Dockerfile**: The main Dockerfile correctly installs `curl` and uses correct curl syntax in the HEALTHCHECK instruction.
---
# Previous Plan (Archived)
The previous Git & Workflow Recovery Plan has been archived below.
---
# Git & Workflow Recovery Plan (ARCHIVED)
**Plan ID**: GIT-2026-001
**Status**: 📋 PENDING
**Status**: ✅ ARCHIVED
**Priority**: High
**Created**: 2026-01-25
**Scope**: Git recovery, Renovate fix, Workflow simplification
@@ -366,3 +574,79 @@ docker compose -f .docker/compose/docker-compose.playwright.yml config
### References
- **OWASP**: [A02:2021 – Cryptographic Failures](https://owasp.org/Top10/A02_2021-Cryptographic_Failures/)
---
# Future Phase: Playwright Security Test Helpers
**Plan ID**: E2E-SEC-001
**Status**: 📋 TODO (Follow-up Task)
**Priority**: Medium
**Created**: 2026-01-25
**Scope**: Add security test helpers to prevent ACL deadlock in E2E tests
---
## Problem Summary
During E2E testing, if ACL is left enabled from a previous test run (e.g., due to test failure), it can create a **deadlock**:
1. ACL blocks API requests → returns 403 Forbidden
2. Global cleanup can't run → API blocked
3. Auth setup fails → tests skip
4. Manual intervention required to reset volumes
## Solution: Security Test Helpers
Create `tests/utils/security-helpers.ts` with API helpers for:
### 1. Get Security Status
```typescript
export async function getSecurityStatus(request: APIRequestContext): Promise<SecurityStatus>
```
### 2. Toggle ACL via API
```typescript
export async function setAclEnabled(request: APIRequestContext, enabled: boolean): Promise<void>
```
### 3. Ensure ACL Enabled with Cleanup
```typescript
export async function ensureAclEnabled(request: APIRequestContext): Promise<OriginalState>
export async function restoreSecurityState(request: APIRequestContext, originalState: OriginalState): Promise<void>
```
## Usage Pattern
```typescript
test.describe('ACL Enforcement Tests', () => {
let originalState: OriginalState;
test.beforeAll(async ({ request }) => {
originalState = await ensureAclEnabled(request);
});
test.afterAll(async ({ request }) => {
await restoreSecurityState(request, originalState); // Runs even on failure
});
// ACL tests here...
});
```
## Benefits
1. **No deadlock**: Tests can safely enable/disable ACL
2. **Cleanup guaranteed**: `test.afterAll` runs even on failure
3. **Realistic testing**: ACL tests use the same toggle mechanism as users
4. **Isolation**: Other tests unaffected by ACL state
## Files to Create/Modify
| File | Action |
|------|--------|
| `tests/utils/security-helpers.ts` | **Create** - New helper module |
| `tests/security/security-dashboard.spec.ts` | **Modify** - Use new helpers |
| `tests/integration/proxy-acl-integration.spec.ts` | **Modify** - Use new helpers |

157
docs/reports/qa_report_waf_integration_fix_2026-01-25.md Normal file
View File

@@ -0,0 +1,157 @@
# QA Report: WAF Integration Fix
**Date**: 2026-01-25
**Branch**: feature/beta-release (merged from development)
**Context**: Fixed integration scripts using wget-style curl syntax
---
## Definition of Done Audit Summary
| Check | Status | Details |
|-------|--------|---------|
| Playwright E2E Tests | ❌ **FAILED** | 230/707 tests failed - ACL blocking test user creation |
| Backend Coverage | ✅ **PASS** | 86.5% (minimum: 85%) |
| Frontend Coverage | ⚠️ **1 FAILURE** | 1499/1500 passed, 1 failed test |
| TypeScript Type Check | ✅ **PASS** | No errors |
| Pre-commit Hooks | ✅ **PASS** | All hooks passed |
| Trivy Security Scan | ⚠️ **WARNING** | 2 HIGH (OS-level, no fix available) |
| Grype Security Scan | ✅ **PASS** | No fixable HIGH/CRITICAL issues |
---
## Detailed Results
### 1. Playwright E2E Tests ❌ FAILED
**Result**: 230 tests failed, 472 passed, 39 skipped
**Root Cause**: All failures show identical error:
```
Error: Failed to create user: {"error":"Blocked by access control list"}
```
**Analysis**: The WAF/ACL configuration is blocking the test fixture's ability to create test users via the API. This is a configuration issue in the Docker container's Cerberus security layer, not a code defect.
**Affected Test Suites**:
- `tests/security/` - Security dashboard, WAF config, rate limiting
- `tests/settings/` - Account settings, user management, notifications, SMTP
- `tests/tasks/` - Backups, imports, logs viewing
**Remediation Required**:
1. Review Cerberus ACL whitelist configuration for test environment
2. Ensure test API endpoints or test user IPs are whitelisted
3. Check if `DISABLE_ACL_FOR_TESTS` environment variable is needed
---
### 2. Backend Coverage ✅ PASS
**Result**: 86.5% statement coverage
- Minimum required: 85%
- All test suites passed
- No test failures
---
### 3. Frontend Coverage ⚠️ 1 FAILURE
**Result**: 1499 passed, 1 failed, 2 skipped
**Failed Test**:
```
FAIL src/components/__tests__/SecurityNotificationSettingsModal.test.tsx
> loads and displays existing settings
AssertionError: expected false to be true
- enableSwitch.checked expected to be true
```
**Analysis**: This appears to be a timing/async issue in the test where the modal's settings aren't loaded before the assertion runs. This is a **test flakiness issue**, not a production bug.
**Remediation**: Add `waitFor` or increase timeout for settings load in the test.
---
### 4. TypeScript Type Check ✅ PASS
**Result**: `tsc --noEmit` completed with zero errors
---
### 5. Pre-commit Hooks ✅ PASS
All hooks passed:
- fix end of files
- trim trailing whitespace
- check yaml
- check for added large files
- dockerfile validation
- Go Vet
- golangci-lint
- .version tag match
- LFS checks
- CodeQL DB artifact prevention
- Frontend TypeScript Check
- Frontend Lint
---
### 6. Security Scans
#### Trivy ⚠️ WARNING
**Findings**: 2 HIGH, 0 CRITICAL
| Library | CVE | Severity | Status | Notes |
|---------|-----|----------|--------|-------|
| libc-bin | CVE-2026-0861 | HIGH | affected | glibc heap corruption - no fix available |
| libc6 | CVE-2026-0861 | HIGH | affected | Same as above |
**Assessment**: These are base OS (Debian 13.3) vulnerabilities in glibc with no upstream fix available. The application code (Go binaries, Caddy, CrowdSec) has **zero vulnerabilities**.
#### Grype ✅ PASS
**Result**: No fixable vulnerabilities found
---
## Issues Blocking Merge
### Critical (Must Fix Before Merge)
1. **Playwright E2E Test ACL Blocking**
- **Issue**: Cerberus ACL blocks test user creation
- **Impact**: 230 E2E tests cannot run
- **Owner**: DevOps/Security configuration
- **Fix**: Whitelist test API or add test environment bypass
### Minor (Can Be Fixed Post-Merge)
2. **Flaky Frontend Test**
- **Issue**: `SecurityNotificationSettingsModal` test timing issue
- **Impact**: 1 test failure
- **Owner**: Frontend team
- **Fix**: Add proper async waiting in test
---
## Recommendations
1. **Immediate**: Investigate and fix the ACL configuration blocking E2E tests
2. **Before Merge**: Re-run full E2E suite after ACL fix
3. **Post-Merge**: Fix the flaky frontend test
4. **Ongoing**: Monitor CVE-2026-0861 for upstream glibc fix
---
## Sign-off
- [ ] ACL blocking issue resolved
- [ ] E2E tests passing (aim: >95%)
- [ ] Frontend flaky test fixed or documented
- [ ] Security scan reviewed and accepted
**Auditor**: GitHub Copilot (Claude Opus 4.5)
**Audit Date**: 2026-01-25