Enhance documentation and testing plans
- Added references to existing test files in the UI/UX testing plan. - Updated CI failure remediation plan with improved file paths and clarity. - Expanded CrowdSec full implementation documentation with detailed configuration steps and scripts. - Improved CrowdSec testing plan with clearer objectives and expected results. - Updated current specification documentation with additional context on CVE remediation. - Enhanced docs-to-issues workflow documentation for better issue tracking. - Corrected numbering in UI/UX bugfixes specification for clarity. - Improved WAF testing plan with detailed curl commands and expected results. - Updated QA reports for CrowdSec implementation and UI/UX testing with detailed results and coverage metrics. - Fixed rate limit integration test summary with clear identification of issues and resolutions. - Enhanced rate limit test status report with detailed root causes and next steps for follow-up.
This commit is contained in:
@@ -498,73 +498,79 @@ We will add a scripted integration test to run inside CI or locally with Docker.
|
||||
```json
|
||||
{"name":"default","enabled":true,"rate_limit_enable":true,"rate_limit_requests":3,"rate_limit_window_sec":10,"rate_limit_burst":1}
|
||||
```
|
||||
- Validate that Caddy Admin API at `http://localhost:2019/config` includes a `rate_limit` handler and, where applicable, a `subroute` with bypass CIDRs (if `RateLimitBypassList` set).
|
||||
- Execute the runtime checks:
|
||||
- Using a single client IP, send 3 requests in quick succession expecting HTTP 200.
|
||||
- The 4th request (same client IP) should return HTTP 429 (Too Many Requests) and include a `Retry-After` header.
|
||||
- On allowed responses, assert that `X-RateLimit-Limit` equals 3 and `X-RateLimit-Remaining` decrements.
|
||||
- Wait until the configured `RateLimitWindowSec` elapses, and confirm requests are allowed again (headers reset).
|
||||
|
||||
- Bypass List Validation:
|
||||
- Set `RateLimitBypassList` to contain the requester's IP (or `127.0.0.1/32` when client runs from the host). Confirm repeated requests do not get `429`, and `X-RateLimit-*` headers may be absent or indicate non-enforcement.
|
||||
- Validate that Caddy Admin API at `http://localhost:2019/config` includes a `rate_limit` handler and, where applicable, a `subroute` with bypass CIDRs (if `RateLimitBypassList` set).
|
||||
- Execute the runtime checks:
|
||||
- Using a single client IP, send 3 requests in quick succession expecting HTTP 200.
|
||||
- The 4th request (same client IP) should return HTTP 429 (Too Many Requests) and include a `Retry-After` header.
|
||||
- On allowed responses, assert that `X-RateLimit-Limit` equals 3 and `X-RateLimit-Remaining` decrements.
|
||||
- Wait until the configured `RateLimitWindowSec` elapses, and confirm requests are allowed again (headers reset).
|
||||
|
||||
- Multi-IP Isolation:
|
||||
- Spin up two client containers with different IPs (via Docker network `--subnet` + `--ip`). Each should have independent counters; both able to make configured number requests without affecting the other.
|
||||
- Bypass List Validation:
|
||||
- Set `RateLimitBypassList` to contain the requester's IP (or `127.0.0.1/32` when client runs from the host). Confirm repeated requests do not get `429`, and `X-RateLimit-*` headers may be absent or indicate non-enforcement.
|
||||
|
||||
- X-Forwarded-For behavior (Confirm remote.host is used as key):
|
||||
- Send requests with `X-Forwarded-For` different than the container IP; observe rate counters still use the connection IP unless Caddy remote_ip plugin explicitly configured to respect XFF.
|
||||
- Multi-IP Isolation:
|
||||
- Spin up two client containers with different IPs (via Docker network `--subnet` + `--ip`). Each should have independent counters; both able to make configured number requests without affecting the other.
|
||||
|
||||
- X-Forwarded-For behavior (Confirm remote.host is used as key):
|
||||
- Send requests with `X-Forwarded-For` different than the container IP; observe rate counters still use the connection IP unless Caddy remote_ip plugin explicitly configured to respect XFF.
|
||||
|
||||
- Test Example (Shell Snippet to assert headers)
|
||||
|
||||
- Test Example (Shell Snippet to assert headers)
|
||||
```bash
|
||||
# Single request driver - check headers
|
||||
curl -s -D - -o /dev/null -H "Host: ratelimit.local" http://localhost/post
|
||||
# Expect headers: X-RateLimit-Limit: 3, X-RateLimit-Remaining: <number>
|
||||
```
|
||||
|
||||
- Script name: `scripts/rate_limit_integration.sh` (mirrors style of `coraza_integration.sh`).
|
||||
- Script name: `scripts/rate_limit_integration.sh` (mirrors style of `coraza_integration.sh`).
|
||||
|
||||
- Manage flaky behavior:
|
||||
- Retry a couple times and log Caddy admin API output on failure for debugging.
|
||||
-
|
||||
|
||||
- Manage flaky behavior:
|
||||
- Retry a couple times and log Caddy admin API output on failure for debugging.
|
||||
+
|
||||
2.3.4 E2E Tests (Longer, optional)
|
||||
|
||||
- Create `scripts/rate_limit_e2e.sh` which spins up the same environment but runs broader scenarios:
|
||||
- High-rate bursts (WindowSec small and Requests small) to test burst allowance/consumption.
|
||||
- Multi-minute stress run (not for every CI pass) to check long-term behavior and reset across windows.
|
||||
- SPA / browser test using Playwright / Cypress to validate UI controls (admin toggles rate limit presets and sets bypass list) and ensures that applied config is effective at runtime.
|
||||
- High-rate bursts (WindowSec small and Requests small) to test burst allowance/consumption.
|
||||
- Multi-minute stress run (not for every CI pass) to check long-term behavior and reset across windows.
|
||||
- SPA / browser test using Playwright / Cypress to validate UI controls (admin toggles rate limit presets and sets bypass list) and ensures that applied config is effective at runtime.
|
||||
|
||||
2.3.5 Mock/Stub Guidance
|
||||
|
||||
- IP Addresses
|
||||
- Use Docker network subnets and `docker run --network containers_default --ip 172.25.0.10` to guarantee client IP addresses for tests and to exercise bypass list behavior.
|
||||
- For tests run from host with `curl`, include `--interface` or `--local-port` if needed to force source IP (less reliable than container-based approach).
|
||||
- Use Docker network subnets and `docker run --network containers_default --ip 172.25.0.10` to guarantee client IP addresses for tests and to exercise bypass list behavior.
|
||||
- For tests run from host with `curl`, include `--interface` or `--local-port` if needed to force source IP (less reliable than container-based approach).
|
||||
- X-Forwarded-For
|
||||
- Add `-H "X-Forwarded-For: 10.0.0.5"` to `curl` requests; assert that plugin uses real connection IP by default. If future changes enable `real_ip` handling in Caddy, tests should be updated to reflect the new behavior.
|
||||
- Add `-H "X-Forwarded-For: 10.0.0.5"` to `curl` requests; assert that plugin uses real connection IP by default. If future changes enable `real_ip` handling in Caddy, tests should be updated to reflect the new behavior.
|
||||
- Timing Windows
|
||||
- Keep small values (2-10 seconds) while maintaining reliability (1s windows are often flaky). For CI environment, `RateLimitWindowSec=10` with `RateLimitRequests=3` and `Burst=1` is a stable, fast choice.
|
||||
- Keep small values (2-10 seconds) while maintaining reliability (1s windows are often flaky). For CI environment, `RateLimitWindowSec=10` with `RateLimitRequests=3` and `Burst=1` is a stable, fast choice.
|
||||
|
||||
2.3.6 Test Data and Assertions (Explicit)
|
||||
|
||||
- Unit Test: `TestBuildRateLimitHandler_ValidConfig`
|
||||
- Input: secCfg{Requests:100, WindowSec:60, Burst:25}
|
||||
- Assert: `h["handler"] == "rate_limit"`, `static".max_events == 100`, `burst == 25`.
|
||||
- Input: secCfg{Requests:100, WindowSec:60, Burst:25}
|
||||
- Assert: `h["handler"] == "rate_limit"`, `static".max_events == 100`, `burst == 25`.
|
||||
|
||||
- Integration Test: `TestRateLimit_Enforcement_Basic`
|
||||
- Input: RateLimitRequests=3, RateLimitWindowSec=10, Burst=1, no bypass list
|
||||
- Actions: Send 4 rapid requests using client container
|
||||
- Expected outputs: [200, 200, 200, 429], 4th returns Retry-After or explicit block message
|
||||
- Assert: Allowed responses include `X-RateLimit-Limit: 3`, and `X-RateLimit-Remaining` decreasing
|
||||
- Input: RateLimitRequests=3, RateLimitWindowSec=10, Burst=1, no bypass list
|
||||
- Actions: Send 4 rapid requests using client container
|
||||
- Expected outputs: [200, 200, 200, 429], 4th returns Retry-After or explicit block message
|
||||
- Assert: Allowed responses include `X-RateLimit-Limit: 3`, and `X-RateLimit-Remaining` decreasing
|
||||
|
||||
- Integration Test: `TestRateLimit_BypassList_SkipsLimit`
|
||||
- Input: Same as above + `RateLimitBypassList` contains client IP CIDR
|
||||
- Expected outputs: All requests 200 (no 429)
|
||||
- Input: Same as above + `RateLimitBypassList` contains client IP CIDR
|
||||
- Expected outputs: All requests 200 (no 429)
|
||||
|
||||
- Integration Test: `TestRateLimit_MultiClient_Isolation`
|
||||
- Input: As above
|
||||
- Actions: Client A sends 3 requests, Client B sends 3 requests
|
||||
- Expected: Both clients unaffected by the other; both get 200 responses for their first 3 requests
|
||||
- Input: As above
|
||||
- Actions: Client A sends 3 requests, Client B sends 3 requests
|
||||
- Expected: Both clients unaffected by the other; both get 200 responses for their first 3 requests
|
||||
|
||||
- Integration Test: `TestRateLimit_Window_Reset`
|
||||
- Input: As above
|
||||
- Actions: Exhaust quota (get 429), wait `RateLimitWindowSec + 1`, issue a new request
|
||||
- Expected: New request is 200 again
|
||||
- Input: As above
|
||||
- Actions: Exhaust quota (get 429), wait `RateLimitWindowSec + 1`, issue a new request
|
||||
- Expected: New request is 200 again
|
||||
|
||||
2.3.7 Test Harness - Example Go Integration Test
|
||||
Use the same approach as `backend/integration/coraza_integration_test.go`, run the script and check output for expected messages. Example test file: `backend/integration/rate_limit_integration_test.go`:
|
||||
@@ -608,14 +614,14 @@ func TestRateLimitIntegration(t *testing.T) {
|
||||
2.3.9 .gitignore / .codecov.yml / Dockerfile changes
|
||||
|
||||
- .gitignore
|
||||
- Add `test-results/rate_limit/` to avoid committing local script logs.
|
||||
- Add `scripts/rate_limit_integration.sh` output files (if any) to ignore.
|
||||
- Add `test-results/rate_limit/` to avoid committing local script logs.
|
||||
- Add `scripts/rate_limit_integration.sh` output files (if any) to ignore.
|
||||
- .codecov.yml
|
||||
- Optional: If you want integration test coverage included, remove `**/integration/**` from `ignore` or add a specific `backend/integration/*_test.go` to be included. (Caveat: integration coverage may not be reproducible across CI).
|
||||
- Optional: If you want integration test coverage included, remove `**/integration/**` from `ignore` or add a specific `backend/integration/*_test.go` to be included. (Caveat: integration coverage may not be reproducible across CI).
|
||||
- .dockerignore
|
||||
- Ensure `scripts/` and `backend/integration` are not copied to reduce build context size if not needed in Docker build.
|
||||
- Ensure `scripts/` and `backend/integration` are not copied to reduce build context size if not needed in Docker build.
|
||||
- Dockerfile
|
||||
- Confirm presence of `--with github.com/mholt/caddy-ratelimit` in the xcaddy build (it is present in base Dockerfile). Add comment and assert plugin presence in integration script by checking `caddy version` or `caddy list` available modules.
|
||||
- Confirm presence of `--with github.com/mholt/caddy-ratelimit` in the xcaddy build (it is present in base Dockerfile). Add comment and assert plugin presence in integration script by checking `caddy version` or `caddy list` available modules.
|
||||
|
||||
2.3.10 Prioritization
|
||||
|
||||
|
||||
@@ -540,6 +540,7 @@ apply-preset-btn
|
||||
### A. Existing Test Patterns (Reference)
|
||||
|
||||
See existing test files for patterns:
|
||||
|
||||
- [Security.test.tsx](frontend/src/pages/__tests__/Security.test.tsx)
|
||||
- [WafConfig.spec.tsx](frontend/src/pages/__tests__/WafConfig.spec.tsx)
|
||||
- [RateLimiting.spec.tsx](frontend/src/pages/__tests__/RateLimiting.spec.tsx)
|
||||
|
||||
@@ -14,7 +14,7 @@ Three GitHub Actions workflows have failed. This document provides root cause an
|
||||
|
||||
### 1.1 Frontend Test Timeout
|
||||
|
||||
**File:** [frontend/src/components/__tests__/LiveLogViewer.test.tsx](../../frontend/src/components/__tests__/LiveLogViewer.test.tsx#L374)
|
||||
**File:** [frontend/src/components/**tests**/LiveLogViewer.test.tsx](../../frontend/src/components/__tests__/LiveLogViewer.test.tsx#L374)
|
||||
**Test:** "displays blocked requests with special styling" under "Security Mode"
|
||||
**Error:** `Test timed out in 5000ms`
|
||||
|
||||
@@ -319,6 +319,7 @@ The workflow at [.github/workflows/pr-checklist.yml](../../.github/workflows/pr-
|
||||
**When this check triggers:**
|
||||
|
||||
The check only runs if the PR modifies files matching:
|
||||
|
||||
- `scripts/history-rewrite/*`
|
||||
- `docs/plans/history_rewrite.md`
|
||||
- Any file containing `history-rewrite` in the path
|
||||
@@ -342,6 +343,7 @@ Update the PR description to include all required checklist items from [.github/
|
||||
**Option B: If PR doesn't need history-rewrite validation**
|
||||
|
||||
Ensure the PR doesn't modify files in:
|
||||
|
||||
- `scripts/history-rewrite/`
|
||||
- `docs/plans/history_rewrite.md`
|
||||
- Any files with `history-rewrite` in the name
|
||||
@@ -359,6 +361,7 @@ If the workflow is triggering incorrectly, check the file list detection logic a
|
||||
**Root Cause:**
|
||||
|
||||
The `benchmark-action/github-action-benchmark@v1` action requires write permissions to push benchmark results to the repository. This fails on:
|
||||
|
||||
- Pull requests from forks (restricted permissions)
|
||||
- PRs where `GITHUB_TOKEN` doesn't have `contents: write` permission
|
||||
|
||||
@@ -371,6 +374,7 @@ permissions:
|
||||
```
|
||||
|
||||
The error occurs because:
|
||||
|
||||
1. On PRs, the token may not have write access
|
||||
2. The `auto-push: true` setting tries to push on main branch only, but the action still needs permissions to access the benchmark data
|
||||
|
||||
@@ -432,11 +436,13 @@ The 1.51x regression (165768 ns vs 109674 ns ≈ 56μs increase) likely comes fr
|
||||
**Investigation Steps:**
|
||||
|
||||
1. Run benchmarks locally to establish baseline:
|
||||
|
||||
```bash
|
||||
cd backend && go test -bench=. -benchmem -benchtime=3s ./internal/api/handlers/... -run=^$
|
||||
```
|
||||
|
||||
2. Compare with previous commit:
|
||||
|
||||
```bash
|
||||
git stash
|
||||
git checkout HEAD~1
|
||||
@@ -455,11 +461,13 @@ The 1.51x regression (165768 ns vs 109674 ns ≈ 56μs increase) likely comes fr
|
||||
**Recommended Actions:**
|
||||
|
||||
**If real regression:**
|
||||
|
||||
- Profile the affected handler using `go test -cpuprofile`
|
||||
- Review recent commits for inefficient code
|
||||
- Optimize the specific slow path
|
||||
|
||||
**If CI flakiness:**
|
||||
|
||||
- Increase `alert-threshold` to `175%` or `200%`
|
||||
- Add `-benchtime=3s` for more stable results
|
||||
- Consider running benchmarks multiple times and averaging
|
||||
|
||||
@@ -63,6 +63,7 @@ This indicates that while CrowdSec binaries are installed and configuration file
|
||||
### The Fatal Error Explained
|
||||
|
||||
CrowdSec requires **datasources** to function. A datasource tells CrowdSec:
|
||||
|
||||
1. Where to find logs (file path, journald, etc.)
|
||||
2. What parser to use for those logs
|
||||
3. Optional labels for categorization
|
||||
@@ -72,11 +73,13 @@ Without datasources configured in `acquis.yaml`, CrowdSec has nothing to monitor
|
||||
### Missing Acquisition Configuration
|
||||
|
||||
The CrowdSec release tarball includes default config files, but the `acquis.yaml` in the tarball is either:
|
||||
|
||||
1. Empty
|
||||
2. Contains example datasources that don't exist in the container (like syslog)
|
||||
3. Not present at all
|
||||
|
||||
**Current entrypoint flow:**
|
||||
|
||||
```bash
|
||||
# Step 1: Copy base config (MISSING acquis.yaml or empty)
|
||||
cp -r /etc/crowdsec.dist/* /etc/crowdsec/
|
||||
@@ -115,6 +118,7 @@ crowdsec &
|
||||
- `crowdsecurity/base-http-scenarios` for generic HTTP attacks
|
||||
|
||||
4. **Acquisition Config**: Tells CrowdSec where to read logs
|
||||
|
||||
```yaml
|
||||
# /etc/crowdsec/acquis.yaml
|
||||
source: file
|
||||
@@ -196,6 +200,7 @@ crowdsec &
|
||||
Create a default acquisition configuration that reads Caddy logs:
|
||||
|
||||
**New file: `configs/crowdsec/acquis.yaml`**
|
||||
|
||||
```yaml
|
||||
# Charon/Caddy Log Acquisition Configuration
|
||||
# This file tells CrowdSec what logs to monitor
|
||||
@@ -219,6 +224,7 @@ labels:
|
||||
#### 1.2 Create Default Config Template
|
||||
|
||||
**New file: `configs/crowdsec/config.yaml.template`**
|
||||
|
||||
```yaml
|
||||
# CrowdSec Configuration for Charon
|
||||
# Generated at container startup
|
||||
@@ -288,6 +294,7 @@ prometheus:
|
||||
#### 1.3 Create Local API Credentials Template
|
||||
|
||||
**New file: `configs/crowdsec/local_api_credentials.yaml.template`**
|
||||
|
||||
```yaml
|
||||
# CrowdSec Local API Credentials
|
||||
# This file is auto-generated - do not edit manually
|
||||
@@ -300,6 +307,7 @@ password: ${CROWDSEC_MACHINE_PASSWORD}
|
||||
#### 1.4 Create Bouncer Registration Script
|
||||
|
||||
**New file: `configs/crowdsec/register_bouncer.sh`**
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
# Register the Caddy bouncer with CrowdSec LAPI
|
||||
@@ -346,6 +354,7 @@ echo "API Key: $API_KEY"
|
||||
#### 1.5 Create Hub Setup Script
|
||||
|
||||
**New file: `configs/crowdsec/install_hub_items.sh`**
|
||||
|
||||
```bash
|
||||
#!/bin/sh
|
||||
# Install required CrowdSec hub items (parsers, scenarios, collections)
|
||||
@@ -597,6 +606,7 @@ The existing `buildCrowdSecHandler` function already generates the correct forma
|
||||
**File: `backend/internal/caddy/config.go`**
|
||||
|
||||
The function at line 752 is mostly correct. Verify it includes:
|
||||
|
||||
- `api_url`: Points to `http://127.0.0.1:8085` (already done)
|
||||
- `api_key`: From environment variable (already done)
|
||||
- `enable_streaming`: For real-time updates (already done)
|
||||
@@ -606,6 +616,7 @@ The function at line 752 is mostly correct. Verify it includes:
|
||||
Since there may not be an official `crowdsecurity/caddy-logs` parser, we need to create a custom parser or use the generic HTTP parser with appropriate normalization.
|
||||
|
||||
**New file: `configs/crowdsec/parsers/caddy-json-logs.yaml`**
|
||||
|
||||
```yaml
|
||||
# Custom parser for Caddy JSON access logs
|
||||
# Install with: cscli parsers install ./caddy-json-logs.yaml --force
|
||||
@@ -1996,11 +2007,13 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items
|
||||
### Post-Implementation Testing
|
||||
|
||||
1. **Build Test:**
|
||||
|
||||
```bash
|
||||
docker build -t charon:local .
|
||||
```
|
||||
|
||||
2. **Startup Test:**
|
||||
|
||||
```bash
|
||||
docker run --rm -d --name charon-test \
|
||||
-p 8080:8080 \
|
||||
@@ -2011,11 +2024,13 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items
|
||||
```
|
||||
|
||||
3. **LAPI Health Test:**
|
||||
|
||||
```bash
|
||||
docker exec charon-test wget -q -O- http://127.0.0.1:8085/health
|
||||
```
|
||||
|
||||
4. **Integration Test:**
|
||||
|
||||
```bash
|
||||
bash scripts/crowdsec_decision_integration.sh
|
||||
```
|
||||
@@ -2028,6 +2043,7 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items
|
||||
- Verify removal
|
||||
|
||||
6. **Unified Logging Test:**
|
||||
|
||||
```bash
|
||||
# Verify log watcher connects to Caddy logs
|
||||
curl -s http://localhost:8080/api/v1/status | jq '.log_watcher'
|
||||
|
||||
@@ -94,27 +94,32 @@ All endpoints are under `/api/v1/admin/crowdsec/` and require authentication.
|
||||
**Objective:** Verify CrowdSec can be started via the Security dashboard
|
||||
|
||||
**Prerequisites:**
|
||||
|
||||
- Charon running with `FEATURE_CERBERUS_ENABLED=true`
|
||||
- CrowdSec binary available in container
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to Security Dashboard (`/security`)
|
||||
2. Locate CrowdSec status card
|
||||
3. Click "Start" button
|
||||
4. Observe loading animation
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- API returns `{"status": "started", "pid": <number>}`
|
||||
- Status changes to "Running"
|
||||
- PID file created at `data/crowdsec/crowdsec.pid`
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/start
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{"status": "started", "pid": 12345}
|
||||
```
|
||||
@@ -126,21 +131,25 @@ curl -X POST -b "$COOKIE_FILE" \
|
||||
**Objective:** Verify CrowdSec status is correctly reported
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. After TC-1, check status endpoint
|
||||
2. Verify UI shows "Running" badge
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/status
|
||||
```
|
||||
|
||||
**Expected Response (when running):**
|
||||
|
||||
```json
|
||||
{"running": true, "pid": 12345}
|
||||
```
|
||||
|
||||
**Expected Response (when stopped):**
|
||||
|
||||
```json
|
||||
{"running": false, "pid": 0}
|
||||
```
|
||||
@@ -152,28 +161,33 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Verify banned IPs table displays correctly
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to `/security/crowdsec`
|
||||
2. Scroll to "Banned IPs" section
|
||||
3. Verify table columns: IP, Reason, Duration, Banned At, Source, Actions
|
||||
|
||||
**Curl Command (via cscli):**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/decisions
|
||||
```
|
||||
|
||||
**Curl Command (via LAPI - preferred):**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/decisions/lapi
|
||||
```
|
||||
|
||||
**Expected Response (empty):**
|
||||
|
||||
```json
|
||||
{"decisions": [], "total": 0}
|
||||
```
|
||||
|
||||
**Expected Response (with bans):**
|
||||
|
||||
```json
|
||||
{
|
||||
"decisions": [
|
||||
@@ -200,11 +214,13 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Ban a test IP address with custom duration
|
||||
|
||||
**Test Data:**
|
||||
|
||||
- IP: `192.168.100.100`
|
||||
- Duration: `1h`
|
||||
- Reason: `Integration test ban`
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to `/security/crowdsec`
|
||||
2. Click "Ban IP" button
|
||||
3. Enter IP: `192.168.100.100`
|
||||
@@ -213,6 +229,7 @@ curl -b "$COOKIE_FILE" \
|
||||
6. Click "Ban IP"
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
-H "Content-Type: application/json" \
|
||||
@@ -221,11 +238,13 @@ curl -X POST -b "$COOKIE_FILE" \
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{"status": "banned", "ip": "192.168.100.100", "duration": "1h"}
|
||||
```
|
||||
|
||||
**Validation:**
|
||||
|
||||
```bash
|
||||
# Verify via decisions list
|
||||
curl -b "$COOKIE_FILE" \
|
||||
@@ -239,11 +258,13 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Confirm banned IP appears in the UI table
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. After TC-4, refresh the page or observe real-time update
|
||||
2. Verify table shows the new ban entry
|
||||
3. Check columns display correct data
|
||||
|
||||
**Expected Table Row:**
|
||||
|
||||
| IP | Reason | Duration | Banned At | Source | Actions |
|
||||
|----|--------|----------|-----------|--------|---------|
|
||||
| 192.168.100.100 | manual ban: Integration test ban | 1h | (timestamp) | manual | [Unban] |
|
||||
@@ -255,18 +276,21 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Remove ban from test IP
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. In Banned IPs table, find `192.168.100.100`
|
||||
2. Click "Unban" button
|
||||
3. Confirm in modal dialog
|
||||
4. Observe IP removed from table
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -X DELETE -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/ban/192.168.100.100
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{"status": "unbanned", "ip": "192.168.100.100"}
|
||||
```
|
||||
@@ -278,16 +302,19 @@ curl -X DELETE -b "$COOKIE_FILE" \
|
||||
**Objective:** Confirm IP no longer appears in banned list
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. After TC-6, verify table no longer shows the IP
|
||||
2. Query decisions endpoint to confirm
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/decisions
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
- IP `192.168.100.100` not present in decisions array
|
||||
|
||||
---
|
||||
@@ -297,22 +324,26 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Export CrowdSec configuration as tar.gz
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to `/security/crowdsec`
|
||||
2. Click "Export" button
|
||||
3. Verify file downloads with timestamp filename
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" -o crowdsec-export.tar.gz \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/export
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
- HTTP 200 with `Content-Type: application/gzip`
|
||||
- `Content-Disposition: attachment; filename=crowdsec-config-YYYYMMDD-HHMMSS.tar.gz`
|
||||
- Valid tar.gz archive containing config files
|
||||
|
||||
**Validation:**
|
||||
|
||||
```bash
|
||||
tar -tzf crowdsec-export.tar.gz
|
||||
# Should list config files
|
||||
@@ -325,15 +356,18 @@ tar -tzf crowdsec-export.tar.gz
|
||||
**Objective:** Import a CrowdSec configuration package
|
||||
|
||||
**Prerequisites:**
|
||||
|
||||
- Export file from TC-8 or test config archive
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Navigate to `/security/crowdsec`
|
||||
2. Select file for import
|
||||
3. Click "Import" button
|
||||
4. Verify backup created and config applied
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
-F "file=@crowdsec-export.tar.gz" \
|
||||
@@ -341,6 +375,7 @@ curl -X POST -b "$COOKIE_FILE" \
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{"status": "imported", "backup": "data/crowdsec.backup.YYYYMMDD-HHMMSS"}
|
||||
```
|
||||
@@ -352,17 +387,20 @@ curl -X POST -b "$COOKIE_FILE" \
|
||||
**Objective:** Verify LAPI connectivity status
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/lapi/health
|
||||
```
|
||||
|
||||
**Expected Response (healthy):**
|
||||
|
||||
```json
|
||||
{"healthy": true, "lapi_url": "http://127.0.0.1:8085", "status": 200}
|
||||
```
|
||||
|
||||
**Expected Response (unhealthy):**
|
||||
|
||||
```json
|
||||
{"healthy": false, "error": "LAPI unreachable", "lapi_url": "http://127.0.0.1:8085"}
|
||||
```
|
||||
@@ -374,21 +412,25 @@ curl -b "$COOKIE_FILE" \
|
||||
**Objective:** Verify CrowdSec can be stopped
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. With CrowdSec running, click "Stop" button
|
||||
2. Verify status changes to "Stopped"
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/stop
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{"status": "stopped"}
|
||||
```
|
||||
|
||||
**Validation:**
|
||||
|
||||
- PID file removed from `data/crowdsec/`
|
||||
- Status endpoint returns `{"running": false, "pid": 0}`
|
||||
|
||||
@@ -397,6 +439,7 @@ curl -X POST -b "$COOKIE_FILE" \
|
||||
## Integration Test Script Requirements
|
||||
|
||||
### Script Location
|
||||
|
||||
`scripts/crowdsec_decision_integration.sh`
|
||||
|
||||
### Script Outline
|
||||
@@ -668,41 +711,50 @@ func TestCrowdsecDecisionsIntegration(t *testing.T) {
|
||||
## Error Scenarios
|
||||
|
||||
### Invalid IP Format
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"ip": "invalid-ip"}' \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/ban
|
||||
```
|
||||
|
||||
**Expected:** HTTP 400 or underlying cscli error
|
||||
|
||||
### Missing IP Parameter
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"duration": "1h"}' \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/ban
|
||||
```
|
||||
|
||||
**Expected:** HTTP 400 `{"error": "ip is required"}`
|
||||
|
||||
### Empty IP String
|
||||
|
||||
```bash
|
||||
curl -X POST -b "$COOKIE_FILE" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"ip": " "}' \
|
||||
http://localhost:8080/api/v1/admin/crowdsec/ban
|
||||
```
|
||||
|
||||
**Expected:** HTTP 400 `{"error": "ip cannot be empty"}`
|
||||
|
||||
### CrowdSec Not Available
|
||||
|
||||
When `cscli` is not in PATH:
|
||||
**Expected:** HTTP 200 with `{"decisions": [], "error": "cscli not available or failed"}`
|
||||
|
||||
### Export When No Config
|
||||
|
||||
```bash
|
||||
# When data/crowdsec doesn't exist
|
||||
curl -b "$COOKIE_FILE" http://localhost:8080/api/v1/admin/crowdsec/export
|
||||
```
|
||||
|
||||
**Expected:** HTTP 404 `{"error": "crowdsec config not found"}`
|
||||
|
||||
---
|
||||
|
||||
@@ -15,6 +15,7 @@ Trivy has identified CVE-2025-62408 in c-ares 1.34.5-r0. The fix requires rebuil
|
||||
**No Dockerfile changes required** - the existing `apk upgrade` command will automatically pull the patched version on the next build.
|
||||
|
||||
See the full remediation plan for:
|
||||
|
||||
- Root cause analysis
|
||||
- CVE details and impact assessment
|
||||
- Step-by-step implementation guide
|
||||
|
||||
@@ -672,6 +672,7 @@ docs/
|
||||
### 7.2 Issue Tracking
|
||||
|
||||
Each created issue includes footer:
|
||||
|
||||
```markdown
|
||||
---
|
||||
*Auto-created from [filename.md](link-to-source-commit)*
|
||||
@@ -746,17 +747,20 @@ console.log(JSON.stringify(result.data, null, 2));
|
||||
## 10. Implementation Phases
|
||||
|
||||
### Phase 1: Setup (15 min)
|
||||
|
||||
1. Create `.github/workflows/docs-to-issues.yml`
|
||||
2. Create `docs/issues/created/.gitkeep`
|
||||
3. Create `docs/issues/_TEMPLATE.md`
|
||||
4. Create `docs/issues/README.md`
|
||||
|
||||
### Phase 2: File Migration (30 min)
|
||||
|
||||
1. Add frontmatter to existing files (in order of priority)
|
||||
2. Test with dry_run mode
|
||||
3. Create one test issue to verify
|
||||
|
||||
### Phase 3: Validation (15 min)
|
||||
|
||||
1. Verify issue creation
|
||||
2. Verify label creation
|
||||
3. Verify project board integration
|
||||
|
||||
@@ -306,7 +306,7 @@ if (!status) return <div className="p-8 text-center text-gray-400">No security s
|
||||
}
|
||||
```
|
||||
|
||||
2. **App.tsx** - Update routes:
|
||||
1. **App.tsx** - Update routes:
|
||||
|
||||
```tsx
|
||||
// Remove: <Route path="users" element={<UsersPage />} />
|
||||
|
||||
@@ -132,6 +132,7 @@ The hash is derived from content to ensure Caddy reloads when rules change.
|
||||
### 2.3 Existing Integration Test Analysis
|
||||
|
||||
The existing `coraza_integration.sh` tests:
|
||||
|
||||
- ✅ XSS payload blocking (`<script>alert(1)</script>`)
|
||||
- ✅ BLOCK mode (expects HTTP 403)
|
||||
- ✅ MONITOR mode switching (expects HTTP 200 after mode change)
|
||||
@@ -234,6 +235,7 @@ curl -s -X POST -H "Content-Type: application/json" \
|
||||
**Objective:** Create a ruleset that blocks SQL injection patterns
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-1: Create SQLi Ruleset ==="
|
||||
|
||||
@@ -252,6 +254,7 @@ echo "$RESP" | jq .
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
{
|
||||
"ruleset": {
|
||||
@@ -271,6 +274,7 @@ echo "$RESP" | jq .
|
||||
**Objective:** Create a ruleset that blocks XSS patterns
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-2: Create XSS Ruleset ==="
|
||||
|
||||
@@ -294,6 +298,7 @@ echo "$RESP" | jq .
|
||||
**Objective:** Set WAF mode to blocking with a specific ruleset
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-3: Enable WAF (Block Mode) ==="
|
||||
|
||||
@@ -317,6 +322,7 @@ sleep 5
|
||||
```
|
||||
|
||||
**Verification:**
|
||||
|
||||
```bash
|
||||
# Check WAF status
|
||||
curl -s -b ${TMP_COOKIE} http://localhost:8080/api/v1/security/status | jq '.waf'
|
||||
@@ -362,6 +368,7 @@ echo "SQLi POST body: HTTP $RESP (expect 403)"
|
||||
```
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- All requests return HTTP 403
|
||||
|
||||
---
|
||||
@@ -371,6 +378,7 @@ echo "SQLi POST body: HTTP $RESP (expect 403)"
|
||||
**Objective:** Verify XSS patterns are blocked with HTTP 403
|
||||
|
||||
**Curl Commands:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-5: XSS Blocking ==="
|
||||
|
||||
@@ -404,6 +412,7 @@ echo "XSS script tag (JSON): HTTP $RESP (expect 403)"
|
||||
```
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- All requests return HTTP 403
|
||||
|
||||
---
|
||||
@@ -413,6 +422,7 @@ echo "XSS script tag (JSON): HTTP $RESP (expect 403)"
|
||||
**Objective:** Verify requests pass but are logged in monitor mode
|
||||
|
||||
**Curl Commands:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-6: Detection Mode ==="
|
||||
|
||||
@@ -440,6 +450,7 @@ docker exec charon-waf-test sh -c 'tail -50 /var/log/caddy/access.log 2>/dev/nul
|
||||
```
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- HTTP 200 response (request passes through)
|
||||
- WAF detection logged (in Caddy access logs or Coraza logs)
|
||||
|
||||
@@ -450,6 +461,7 @@ docker exec charon-waf-test sh -c 'tail -50 /var/log/caddy/access.log 2>/dev/nul
|
||||
**Objective:** Verify both SQLi and XSS rules can be combined
|
||||
|
||||
**Curl Commands:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-7: Multiple Rulesets (Combined) ==="
|
||||
|
||||
@@ -498,6 +510,7 @@ echo "Combined - Legitimate: HTTP $RESP (expect 200)"
|
||||
**Objective:** Verify all rulesets are listed correctly
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-8: List Rulesets ==="
|
||||
|
||||
@@ -506,6 +519,7 @@ echo "$RESP" | jq '.rulesets[] | {name, mode, last_updated}'
|
||||
```
|
||||
|
||||
**Expected Response:**
|
||||
|
||||
```json
|
||||
[
|
||||
{"name": "sqli-protection", "mode": "", "last_updated": "..."},
|
||||
@@ -521,6 +535,7 @@ echo "$RESP" | jq '.rulesets[] | {name, mode, last_updated}'
|
||||
**Objective:** Add and remove WAF rule exclusions for false positives
|
||||
|
||||
**Curl Commands:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-9: WAF Rule Exclusions ==="
|
||||
|
||||
@@ -548,6 +563,7 @@ echo "Delete exclusion: $RESP"
|
||||
**Objective:** Confirm WAF handler is present in running Caddy config
|
||||
|
||||
**Curl Command:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-10: Verify Caddy Config ==="
|
||||
|
||||
@@ -585,6 +601,7 @@ fi
|
||||
**Objective:** Verify ruleset can be deleted
|
||||
|
||||
**Curl Commands:**
|
||||
|
||||
```bash
|
||||
echo "=== TC-11: Delete Ruleset ==="
|
||||
|
||||
@@ -793,33 +810,33 @@ Location: `backend/integration/waf_integration_test.go`
|
||||
package integration
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
"context"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TestWAFIntegration runs the scripts/waf_integration.sh and ensures it completes successfully.
|
||||
func TestWAFIntegration(t *testing.T) {
|
||||
t.Parallel()
|
||||
t.Parallel()
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
cmd := exec.CommandContext(ctx, "bash", "./scripts/waf_integration.sh")
|
||||
cmd.Dir = "../.."
|
||||
cmd := exec.CommandContext(ctx, "bash", "./scripts/waf_integration.sh")
|
||||
cmd.Dir = "../.."
|
||||
|
||||
out, err := cmd.CombinedOutput()
|
||||
t.Logf("waf_integration script output:\n%s", string(out))
|
||||
out, err := cmd.CombinedOutput()
|
||||
t.Logf("waf_integration script output:\n%s", string(out))
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("waf integration failed: %v", err)
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("waf integration failed: %v", err)
|
||||
}
|
||||
|
||||
if !strings.Contains(string(out), "All WAF tests passed") {
|
||||
t.Fatalf("unexpected script output, expected pass assertion not found")
|
||||
}
|
||||
if !strings.Contains(string(out), "All WAF tests passed") {
|
||||
t.Fatalf("unexpected script output, expected pass assertion not found")
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user