diff --git a/.markdownlintrc b/.markdownlintrc new file mode 100644 index 00000000..7d009840 --- /dev/null +++ b/.markdownlintrc @@ -0,0 +1,10 @@ +{ + "default": true, + "MD013": { + "line_length": 150, + "tables": false, + "code_blocks": false + }, + "MD033": false, + "MD041": false +} diff --git a/.vscode/tasks.json b/.vscode/tasks.json index d0e12181..0cacbd39 100644 --- a/.vscode/tasks.json +++ b/.vscode/tasks.json @@ -113,14 +113,14 @@ { "label": "Lint: Markdownlint", "type": "shell", - "command": "npx markdownlint '**/*.md' --ignore node_modules --ignore .venv --ignore test-results --ignore codeql-db --ignore codeql-agent-results", + "command": "markdownlint '**/*.md' --ignore node_modules --ignore frontend/node_modules --ignore .venv --ignore test-results --ignore codeql-db --ignore codeql-agent-results", "group": "test", "problemMatcher": [] }, { "label": "Lint: Markdownlint (Fix)", "type": "shell", - "command": "npx markdownlint '**/*.md' --fix --ignore node_modules --ignore .venv --ignore test-results --ignore codeql-db --ignore codeql-agent-results", + "command": "markdownlint '**/*.md' --fix --ignore node_modules --ignore frontend/node_modules --ignore .venv --ignore test-results --ignore codeql-db --ignore codeql-agent-results", "group": "test", "problemMatcher": [] }, diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 441d9014..793c1a33 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -41,7 +41,7 @@ git clone https://github.com/YOUR_USERNAME/charon.git cd charon ``` -3. Add the upstream remote: +1. Add the upstream remote: ```bash git remote add upstream https://github.com/Wikid82/charon.git @@ -265,7 +265,7 @@ go test ./... npm test -- --run ``` -2. **Check code quality:** +1. **Check code quality:** ```bash # Go formatting @@ -275,9 +275,9 @@ go fmt ./... npm run lint ``` -3. **Update documentation** if needed -4. **Add tests** for new functionality -5. **Rebase on latest development** branch +1. **Update documentation** if needed +2. **Add tests** for new functionality +3. **Rebase on latest development** branch ### Submitting a Pull Request @@ -287,10 +287,10 @@ npm run lint git push origin feature/your-feature-name ``` -2. Open a Pull Request on GitHub -3. Fill out the PR template completely -4. Link related issues using "Closes #123" or "Fixes #456" -5. Request review from maintainers +1. Open a Pull Request on GitHub +2. Fill out the PR template completely +3. Link related issues using "Closes #123" or "Fixes #456" +4. Request review from maintainers ### PR Template diff --git a/README.md b/README.md index dd4b50e5..6a89254b 100644 --- a/README.md +++ b/README.md @@ -14,6 +14,9 @@ Turn multiple websites and apps into one simple dashboard. Click, save, done. No

Project Status: Active – The project is being actively developed.License: MIT + + Code Coverage + Release Build Status

diff --git a/SECURITY_CONFIG_PRIORITY.md b/SECURITY_CONFIG_PRIORITY.md index 0f1643e3..7e89df71 100644 --- a/SECURITY_CONFIG_PRIORITY.md +++ b/SECURITY_CONFIG_PRIORITY.md @@ -35,19 +35,24 @@ When the `/api/v1/security/status` endpoint is called, the system: ## Supported Settings Table Keys ### Cerberus (Master Switch) + - `feature.cerberus.enabled` - "true"/"false" - Enables/disables all security features ### WAF (Web Application Firewall) + - `security.waf.enabled` - "true"/"false" - Overrides WAF mode ### Rate Limiting + - `security.rate_limit.enabled` - "true"/"false" - Overrides rate limit mode ### CrowdSec + - `security.crowdsec.enabled` - "true"/"false" - Sets CrowdSec to local/disabled - `security.crowdsec.mode` - "local"/"disabled" - Direct mode override ### ACL (Access Control Lists) + - `security.acl.enabled` - "true"/"false" - Overrides ACL mode ## Examples @@ -127,6 +132,7 @@ config.SecurityConfig{ ## Testing Comprehensive unit tests verify the priority chain: + - `TestSecurityHandler_Priority_SettingsOverSecurityConfig` - Tests all three priority levels - `TestSecurityHandler_Priority_AllModules` - Tests all security modules together - `TestSecurityHandler_GetStatus_RespectsSettingsTable` - Tests Settings table overrides @@ -178,6 +184,7 @@ func (h *SecurityHandler) GetStatus(c *gin.Context) { ## QA Verification All previously failing tests now pass: + - ✅ `TestCertificateHandler_Delete_NotificationRateLimiting` - ✅ `TestSecurityHandler_ACL_DBOverride` - ✅ `TestSecurityHandler_CrowdSec_Mode_DBOverride` @@ -188,6 +195,7 @@ All previously failing tests now pass: ## Migration Notes For existing deployments: + 1. No database migration required - Settings table already exists 2. SecurityConfig records work as before 3. New Settings table overrides are optional diff --git a/docs/issues/created/20251213-orthrus.md b/docs/issues/created/20251213-orthrus.md index 587743ee..6629a0c3 100644 --- a/docs/issues/created/20251213-orthrus.md +++ b/docs/issues/created/20251213-orthrus.md @@ -173,6 +173,7 @@ To maintain a lightweight footprint (< 20MB), Orthrus uses a separate Go module Orthrus should be distributed in multiple formats so users can choose one that fits their environment and security posture. ### 9.1 Supported Distribution Formats + * **Docker / Docker Compose**: easiest for container-based hosts. * **Standalone static binary (recommended)**: small, copy to `/usr/local/bin`, run via `systemd`. * **Deb / RPM packages**: for managed installs via `apt`/`yum`. @@ -198,7 +199,7 @@ services: - /var/run/docker.sock:/var/run/docker.sock:ro ``` -2) Standalone binary + `systemd` (Linux) +1) Standalone binary + `systemd` (Linux) ```bash # download and install @@ -227,7 +228,7 @@ systemctl daemon-reload systemctl enable --now orthrus ``` -3) Tarball + install script +1) Tarball + install script ```bash curl -L -o orthrus.tar.gz https://example.com/orthrus/vX.Y.Z/orthrus-linux-amd64.tar.gz @@ -237,18 +238,19 @@ chmod +x /usr/local/bin/orthrus # then use the systemd unit above ``` -4) Homebrew (macOS / Linuxbrew) +1) Homebrew (macOS / Linuxbrew) ``` brew tap wikid82/charon brew install orthrus ``` -5) Kubernetes DaemonSet +1) Kubernetes DaemonSet Provide a DaemonSet YAML referencing the `orthrus` image and the required env vars (`AUTH_KEY`, `CHARON_LINK`), optionally mounting the Docker socket or using hostNetworking. ### 9.3 Security & UX Notes + * Provide SHA256 checksums and GPG signatures for binary downloads. * Avoid recommending `curl | sh`; prefer explicit steps and checksum verification. * The Hecate UI should present each snippet as a selectable tab with a copy button and an inline checksum. diff --git a/docs/plans/cerberus_remediation_plan.md b/docs/plans/cerberus_remediation_plan.md index 1febcce8..99495553 100644 --- a/docs/plans/cerberus_remediation_plan.md +++ b/docs/plans/cerberus_remediation_plan.md @@ -498,73 +498,79 @@ We will add a scripted integration test to run inside CI or locally with Docker. ```json {"name":"default","enabled":true,"rate_limit_enable":true,"rate_limit_requests":3,"rate_limit_window_sec":10,"rate_limit_burst":1} ``` - - Validate that Caddy Admin API at `http://localhost:2019/config` includes a `rate_limit` handler and, where applicable, a `subroute` with bypass CIDRs (if `RateLimitBypassList` set). - - Execute the runtime checks: - - Using a single client IP, send 3 requests in quick succession expecting HTTP 200. - - The 4th request (same client IP) should return HTTP 429 (Too Many Requests) and include a `Retry-After` header. - - On allowed responses, assert that `X-RateLimit-Limit` equals 3 and `X-RateLimit-Remaining` decrements. - - Wait until the configured `RateLimitWindowSec` elapses, and confirm requests are allowed again (headers reset). - - Bypass List Validation: - - Set `RateLimitBypassList` to contain the requester's IP (or `127.0.0.1/32` when client runs from the host). Confirm repeated requests do not get `429`, and `X-RateLimit-*` headers may be absent or indicate non-enforcement. +- Validate that Caddy Admin API at `http://localhost:2019/config` includes a `rate_limit` handler and, where applicable, a `subroute` with bypass CIDRs (if `RateLimitBypassList` set). +- Execute the runtime checks: +- Using a single client IP, send 3 requests in quick succession expecting HTTP 200. +- The 4th request (same client IP) should return HTTP 429 (Too Many Requests) and include a `Retry-After` header. +- On allowed responses, assert that `X-RateLimit-Limit` equals 3 and `X-RateLimit-Remaining` decrements. +- Wait until the configured `RateLimitWindowSec` elapses, and confirm requests are allowed again (headers reset). - - Multi-IP Isolation: - - Spin up two client containers with different IPs (via Docker network `--subnet` + `--ip`). Each should have independent counters; both able to make configured number requests without affecting the other. +- Bypass List Validation: +- Set `RateLimitBypassList` to contain the requester's IP (or `127.0.0.1/32` when client runs from the host). Confirm repeated requests do not get `429`, and `X-RateLimit-*` headers may be absent or indicate non-enforcement. - - X-Forwarded-For behavior (Confirm remote.host is used as key): - - Send requests with `X-Forwarded-For` different than the container IP; observe rate counters still use the connection IP unless Caddy remote_ip plugin explicitly configured to respect XFF. +- Multi-IP Isolation: +- Spin up two client containers with different IPs (via Docker network `--subnet` + `--ip`). Each should have independent counters; both able to make configured number requests without affecting the other. + +- X-Forwarded-For behavior (Confirm remote.host is used as key): +- Send requests with `X-Forwarded-For` different than the container IP; observe rate counters still use the connection IP unless Caddy remote_ip plugin explicitly configured to respect XFF. + +- Test Example (Shell Snippet to assert headers) - - Test Example (Shell Snippet to assert headers) ```bash # Single request driver - check headers curl -s -D - -o /dev/null -H "Host: ratelimit.local" http://localhost/post # Expect headers: X-RateLimit-Limit: 3, X-RateLimit-Remaining: ``` - - Script name: `scripts/rate_limit_integration.sh` (mirrors style of `coraza_integration.sh`). +- Script name: `scripts/rate_limit_integration.sh` (mirrors style of `coraza_integration.sh`). + +- Manage flaky behavior: +- Retry a couple times and log Caddy admin API output on failure for debugging. +- - - Manage flaky behavior: - - Retry a couple times and log Caddy admin API output on failure for debugging. -+ 2.3.4 E2E Tests (Longer, optional) + - Create `scripts/rate_limit_e2e.sh` which spins up the same environment but runs broader scenarios: - - High-rate bursts (WindowSec small and Requests small) to test burst allowance/consumption. - - Multi-minute stress run (not for every CI pass) to check long-term behavior and reset across windows. - - SPA / browser test using Playwright / Cypress to validate UI controls (admin toggles rate limit presets and sets bypass list) and ensures that applied config is effective at runtime. +- High-rate bursts (WindowSec small and Requests small) to test burst allowance/consumption. +- Multi-minute stress run (not for every CI pass) to check long-term behavior and reset across windows. +- SPA / browser test using Playwright / Cypress to validate UI controls (admin toggles rate limit presets and sets bypass list) and ensures that applied config is effective at runtime. 2.3.5 Mock/Stub Guidance + - IP Addresses - - Use Docker network subnets and `docker run --network containers_default --ip 172.25.0.10` to guarantee client IP addresses for tests and to exercise bypass list behavior. - - For tests run from host with `curl`, include `--interface` or `--local-port` if needed to force source IP (less reliable than container-based approach). +- Use Docker network subnets and `docker run --network containers_default --ip 172.25.0.10` to guarantee client IP addresses for tests and to exercise bypass list behavior. +- For tests run from host with `curl`, include `--interface` or `--local-port` if needed to force source IP (less reliable than container-based approach). - X-Forwarded-For - - Add `-H "X-Forwarded-For: 10.0.0.5"` to `curl` requests; assert that plugin uses real connection IP by default. If future changes enable `real_ip` handling in Caddy, tests should be updated to reflect the new behavior. +- Add `-H "X-Forwarded-For: 10.0.0.5"` to `curl` requests; assert that plugin uses real connection IP by default. If future changes enable `real_ip` handling in Caddy, tests should be updated to reflect the new behavior. - Timing Windows - - Keep small values (2-10 seconds) while maintaining reliability (1s windows are often flaky). For CI environment, `RateLimitWindowSec=10` with `RateLimitRequests=3` and `Burst=1` is a stable, fast choice. +- Keep small values (2-10 seconds) while maintaining reliability (1s windows are often flaky). For CI environment, `RateLimitWindowSec=10` with `RateLimitRequests=3` and `Burst=1` is a stable, fast choice. 2.3.6 Test Data and Assertions (Explicit) + - Unit Test: `TestBuildRateLimitHandler_ValidConfig` - - Input: secCfg{Requests:100, WindowSec:60, Burst:25} - - Assert: `h["handler"] == "rate_limit"`, `static".max_events == 100`, `burst == 25`. +- Input: secCfg{Requests:100, WindowSec:60, Burst:25} +- Assert: `h["handler"] == "rate_limit"`, `static".max_events == 100`, `burst == 25`. - Integration Test: `TestRateLimit_Enforcement_Basic` - - Input: RateLimitRequests=3, RateLimitWindowSec=10, Burst=1, no bypass list - - Actions: Send 4 rapid requests using client container - - Expected outputs: [200, 200, 200, 429], 4th returns Retry-After or explicit block message - - Assert: Allowed responses include `X-RateLimit-Limit: 3`, and `X-RateLimit-Remaining` decreasing +- Input: RateLimitRequests=3, RateLimitWindowSec=10, Burst=1, no bypass list +- Actions: Send 4 rapid requests using client container +- Expected outputs: [200, 200, 200, 429], 4th returns Retry-After or explicit block message +- Assert: Allowed responses include `X-RateLimit-Limit: 3`, and `X-RateLimit-Remaining` decreasing - Integration Test: `TestRateLimit_BypassList_SkipsLimit` - - Input: Same as above + `RateLimitBypassList` contains client IP CIDR - - Expected outputs: All requests 200 (no 429) +- Input: Same as above + `RateLimitBypassList` contains client IP CIDR +- Expected outputs: All requests 200 (no 429) - Integration Test: `TestRateLimit_MultiClient_Isolation` - - Input: As above - - Actions: Client A sends 3 requests, Client B sends 3 requests - - Expected: Both clients unaffected by the other; both get 200 responses for their first 3 requests +- Input: As above +- Actions: Client A sends 3 requests, Client B sends 3 requests +- Expected: Both clients unaffected by the other; both get 200 responses for their first 3 requests - Integration Test: `TestRateLimit_Window_Reset` - - Input: As above - - Actions: Exhaust quota (get 429), wait `RateLimitWindowSec + 1`, issue a new request - - Expected: New request is 200 again +- Input: As above +- Actions: Exhaust quota (get 429), wait `RateLimitWindowSec + 1`, issue a new request +- Expected: New request is 200 again 2.3.7 Test Harness - Example Go Integration Test Use the same approach as `backend/integration/coraza_integration_test.go`, run the script and check output for expected messages. Example test file: `backend/integration/rate_limit_integration_test.go`: @@ -608,14 +614,14 @@ func TestRateLimitIntegration(t *testing.T) { 2.3.9 .gitignore / .codecov.yml / Dockerfile changes - .gitignore - - Add `test-results/rate_limit/` to avoid committing local script logs. - - Add `scripts/rate_limit_integration.sh` output files (if any) to ignore. + - Add `test-results/rate_limit/` to avoid committing local script logs. + - Add `scripts/rate_limit_integration.sh` output files (if any) to ignore. - .codecov.yml - - Optional: If you want integration test coverage included, remove `**/integration/**` from `ignore` or add a specific `backend/integration/*_test.go` to be included. (Caveat: integration coverage may not be reproducible across CI). + - Optional: If you want integration test coverage included, remove `**/integration/**` from `ignore` or add a specific `backend/integration/*_test.go` to be included. (Caveat: integration coverage may not be reproducible across CI). - .dockerignore - - Ensure `scripts/` and `backend/integration` are not copied to reduce build context size if not needed in Docker build. + - Ensure `scripts/` and `backend/integration` are not copied to reduce build context size if not needed in Docker build. - Dockerfile - - Confirm presence of `--with github.com/mholt/caddy-ratelimit` in the xcaddy build (it is present in base Dockerfile). Add comment and assert plugin presence in integration script by checking `caddy version` or `caddy list` available modules. + - Confirm presence of `--with github.com/mholt/caddy-ratelimit` in the xcaddy build (it is present in base Dockerfile). Add comment and assert plugin presence in integration script by checking `caddy version` or `caddy list` available modules. 2.3.10 Prioritization diff --git a/docs/plans/cerberus_uiux_testing_plan.md b/docs/plans/cerberus_uiux_testing_plan.md index 0b0ffed7..ffb2df7b 100644 --- a/docs/plans/cerberus_uiux_testing_plan.md +++ b/docs/plans/cerberus_uiux_testing_plan.md @@ -540,6 +540,7 @@ apply-preset-btn ### A. Existing Test Patterns (Reference) See existing test files for patterns: + - [Security.test.tsx](frontend/src/pages/__tests__/Security.test.tsx) - [WafConfig.spec.tsx](frontend/src/pages/__tests__/WafConfig.spec.tsx) - [RateLimiting.spec.tsx](frontend/src/pages/__tests__/RateLimiting.spec.tsx) diff --git a/docs/plans/ci_failure_remediation_plan.md b/docs/plans/ci_failure_remediation_plan.md index 3a8e8c5c..f851b002 100644 --- a/docs/plans/ci_failure_remediation_plan.md +++ b/docs/plans/ci_failure_remediation_plan.md @@ -14,7 +14,7 @@ Three GitHub Actions workflows have failed. This document provides root cause an ### 1.1 Frontend Test Timeout -**File:** [frontend/src/components/__tests__/LiveLogViewer.test.tsx](../../frontend/src/components/__tests__/LiveLogViewer.test.tsx#L374) +**File:** [frontend/src/components/**tests**/LiveLogViewer.test.tsx](../../frontend/src/components/__tests__/LiveLogViewer.test.tsx#L374) **Test:** "displays blocked requests with special styling" under "Security Mode" **Error:** `Test timed out in 5000ms` @@ -319,6 +319,7 @@ The workflow at [.github/workflows/pr-checklist.yml](../../.github/workflows/pr- **When this check triggers:** The check only runs if the PR modifies files matching: + - `scripts/history-rewrite/*` - `docs/plans/history_rewrite.md` - Any file containing `history-rewrite` in the path @@ -342,6 +343,7 @@ Update the PR description to include all required checklist items from [.github/ **Option B: If PR doesn't need history-rewrite validation** Ensure the PR doesn't modify files in: + - `scripts/history-rewrite/` - `docs/plans/history_rewrite.md` - Any files with `history-rewrite` in the name @@ -359,6 +361,7 @@ If the workflow is triggering incorrectly, check the file list detection logic a **Root Cause:** The `benchmark-action/github-action-benchmark@v1` action requires write permissions to push benchmark results to the repository. This fails on: + - Pull requests from forks (restricted permissions) - PRs where `GITHUB_TOKEN` doesn't have `contents: write` permission @@ -371,6 +374,7 @@ permissions: ``` The error occurs because: + 1. On PRs, the token may not have write access 2. The `auto-push: true` setting tries to push on main branch only, but the action still needs permissions to access the benchmark data @@ -432,11 +436,13 @@ The 1.51x regression (165768 ns vs 109674 ns ≈ 56μs increase) likely comes fr **Investigation Steps:** 1. Run benchmarks locally to establish baseline: + ```bash cd backend && go test -bench=. -benchmem -benchtime=3s ./internal/api/handlers/... -run=^$ ``` 2. Compare with previous commit: + ```bash git stash git checkout HEAD~1 @@ -455,11 +461,13 @@ The 1.51x regression (165768 ns vs 109674 ns ≈ 56μs increase) likely comes fr **Recommended Actions:** **If real regression:** + - Profile the affected handler using `go test -cpuprofile` - Review recent commits for inefficient code - Optimize the specific slow path **If CI flakiness:** + - Increase `alert-threshold` to `175%` or `200%` - Add `-benchtime=3s` for more stable results - Consider running benchmarks multiple times and averaging diff --git a/docs/plans/crowdsec_full_implementation.md b/docs/plans/crowdsec_full_implementation.md index d1431ae6..271b30dd 100644 --- a/docs/plans/crowdsec_full_implementation.md +++ b/docs/plans/crowdsec_full_implementation.md @@ -63,6 +63,7 @@ This indicates that while CrowdSec binaries are installed and configuration file ### The Fatal Error Explained CrowdSec requires **datasources** to function. A datasource tells CrowdSec: + 1. Where to find logs (file path, journald, etc.) 2. What parser to use for those logs 3. Optional labels for categorization @@ -72,11 +73,13 @@ Without datasources configured in `acquis.yaml`, CrowdSec has nothing to monitor ### Missing Acquisition Configuration The CrowdSec release tarball includes default config files, but the `acquis.yaml` in the tarball is either: + 1. Empty 2. Contains example datasources that don't exist in the container (like syslog) 3. Not present at all **Current entrypoint flow:** + ```bash # Step 1: Copy base config (MISSING acquis.yaml or empty) cp -r /etc/crowdsec.dist/* /etc/crowdsec/ @@ -115,6 +118,7 @@ crowdsec & - `crowdsecurity/base-http-scenarios` for generic HTTP attacks 4. **Acquisition Config**: Tells CrowdSec where to read logs + ```yaml # /etc/crowdsec/acquis.yaml source: file @@ -196,6 +200,7 @@ crowdsec & Create a default acquisition configuration that reads Caddy logs: **New file: `configs/crowdsec/acquis.yaml`** + ```yaml # Charon/Caddy Log Acquisition Configuration # This file tells CrowdSec what logs to monitor @@ -219,6 +224,7 @@ labels: #### 1.2 Create Default Config Template **New file: `configs/crowdsec/config.yaml.template`** + ```yaml # CrowdSec Configuration for Charon # Generated at container startup @@ -288,6 +294,7 @@ prometheus: #### 1.3 Create Local API Credentials Template **New file: `configs/crowdsec/local_api_credentials.yaml.template`** + ```yaml # CrowdSec Local API Credentials # This file is auto-generated - do not edit manually @@ -300,6 +307,7 @@ password: ${CROWDSEC_MACHINE_PASSWORD} #### 1.4 Create Bouncer Registration Script **New file: `configs/crowdsec/register_bouncer.sh`** + ```bash #!/bin/sh # Register the Caddy bouncer with CrowdSec LAPI @@ -346,6 +354,7 @@ echo "API Key: $API_KEY" #### 1.5 Create Hub Setup Script **New file: `configs/crowdsec/install_hub_items.sh`** + ```bash #!/bin/sh # Install required CrowdSec hub items (parsers, scenarios, collections) @@ -597,6 +606,7 @@ The existing `buildCrowdSecHandler` function already generates the correct forma **File: `backend/internal/caddy/config.go`** The function at line 752 is mostly correct. Verify it includes: + - `api_url`: Points to `http://127.0.0.1:8085` (already done) - `api_key`: From environment variable (already done) - `enable_streaming`: For real-time updates (already done) @@ -606,6 +616,7 @@ The function at line 752 is mostly correct. Verify it includes: Since there may not be an official `crowdsecurity/caddy-logs` parser, we need to create a custom parser or use the generic HTTP parser with appropriate normalization. **New file: `configs/crowdsec/parsers/caddy-json-logs.yaml`** + ```yaml # Custom parser for Caddy JSON access logs # Install with: cscli parsers install ./caddy-json-logs.yaml --force @@ -1996,11 +2007,13 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items ### Post-Implementation Testing 1. **Build Test:** + ```bash docker build -t charon:local . ``` 2. **Startup Test:** + ```bash docker run --rm -d --name charon-test \ -p 8080:8080 \ @@ -2011,11 +2024,13 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items ``` 3. **LAPI Health Test:** + ```bash docker exec charon-test wget -q -O- http://127.0.0.1:8085/health ``` 4. **Integration Test:** + ```bash bash scripts/crowdsec_decision_integration.sh ``` @@ -2028,6 +2043,7 @@ RUN chmod +x /usr/local/bin/register_bouncer.sh /usr/local/bin/install_hub_items - Verify removal 6. **Unified Logging Test:** + ```bash # Verify log watcher connects to Caddy logs curl -s http://localhost:8080/api/v1/status | jq '.log_watcher' diff --git a/docs/plans/crowdsec_testing_plan.md b/docs/plans/crowdsec_testing_plan.md index 06ef992f..0d4c268f 100644 --- a/docs/plans/crowdsec_testing_plan.md +++ b/docs/plans/crowdsec_testing_plan.md @@ -94,27 +94,32 @@ All endpoints are under `/api/v1/admin/crowdsec/` and require authentication. **Objective:** Verify CrowdSec can be started via the Security dashboard **Prerequisites:** + - Charon running with `FEATURE_CERBERUS_ENABLED=true` - CrowdSec binary available in container **Steps:** + 1. Navigate to Security Dashboard (`/security`) 2. Locate CrowdSec status card 3. Click "Start" button 4. Observe loading animation **Expected Results:** + - API returns `{"status": "started", "pid": }` - Status changes to "Running" - PID file created at `data/crowdsec/crowdsec.pid` **Curl Command:** + ```bash curl -X POST -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/start ``` **Expected Response:** + ```json {"status": "started", "pid": 12345} ``` @@ -126,21 +131,25 @@ curl -X POST -b "$COOKIE_FILE" \ **Objective:** Verify CrowdSec status is correctly reported **Steps:** + 1. After TC-1, check status endpoint 2. Verify UI shows "Running" badge **Curl Command:** + ```bash curl -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/status ``` **Expected Response (when running):** + ```json {"running": true, "pid": 12345} ``` **Expected Response (when stopped):** + ```json {"running": false, "pid": 0} ``` @@ -152,28 +161,33 @@ curl -b "$COOKIE_FILE" \ **Objective:** Verify banned IPs table displays correctly **Steps:** + 1. Navigate to `/security/crowdsec` 2. Scroll to "Banned IPs" section 3. Verify table columns: IP, Reason, Duration, Banned At, Source, Actions **Curl Command (via cscli):** + ```bash curl -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/decisions ``` **Curl Command (via LAPI - preferred):** + ```bash curl -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/decisions/lapi ``` **Expected Response (empty):** + ```json {"decisions": [], "total": 0} ``` **Expected Response (with bans):** + ```json { "decisions": [ @@ -200,11 +214,13 @@ curl -b "$COOKIE_FILE" \ **Objective:** Ban a test IP address with custom duration **Test Data:** + - IP: `192.168.100.100` - Duration: `1h` - Reason: `Integration test ban` **Steps:** + 1. Navigate to `/security/crowdsec` 2. Click "Ban IP" button 3. Enter IP: `192.168.100.100` @@ -213,6 +229,7 @@ curl -b "$COOKIE_FILE" \ 6. Click "Ban IP" **Curl Command:** + ```bash curl -X POST -b "$COOKIE_FILE" \ -H "Content-Type: application/json" \ @@ -221,11 +238,13 @@ curl -X POST -b "$COOKIE_FILE" \ ``` **Expected Response:** + ```json {"status": "banned", "ip": "192.168.100.100", "duration": "1h"} ``` **Validation:** + ```bash # Verify via decisions list curl -b "$COOKIE_FILE" \ @@ -239,11 +258,13 @@ curl -b "$COOKIE_FILE" \ **Objective:** Confirm banned IP appears in the UI table **Steps:** + 1. After TC-4, refresh the page or observe real-time update 2. Verify table shows the new ban entry 3. Check columns display correct data **Expected Table Row:** + | IP | Reason | Duration | Banned At | Source | Actions | |----|--------|----------|-----------|--------|---------| | 192.168.100.100 | manual ban: Integration test ban | 1h | (timestamp) | manual | [Unban] | @@ -255,18 +276,21 @@ curl -b "$COOKIE_FILE" \ **Objective:** Remove ban from test IP **Steps:** + 1. In Banned IPs table, find `192.168.100.100` 2. Click "Unban" button 3. Confirm in modal dialog 4. Observe IP removed from table **Curl Command:** + ```bash curl -X DELETE -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/ban/192.168.100.100 ``` **Expected Response:** + ```json {"status": "unbanned", "ip": "192.168.100.100"} ``` @@ -278,16 +302,19 @@ curl -X DELETE -b "$COOKIE_FILE" \ **Objective:** Confirm IP no longer appears in banned list **Steps:** + 1. After TC-6, verify table no longer shows the IP 2. Query decisions endpoint to confirm **Curl Command:** + ```bash curl -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/decisions ``` **Expected Response:** + - IP `192.168.100.100` not present in decisions array --- @@ -297,22 +324,26 @@ curl -b "$COOKIE_FILE" \ **Objective:** Export CrowdSec configuration as tar.gz **Steps:** + 1. Navigate to `/security/crowdsec` 2. Click "Export" button 3. Verify file downloads with timestamp filename **Curl Command:** + ```bash curl -b "$COOKIE_FILE" -o crowdsec-export.tar.gz \ http://localhost:8080/api/v1/admin/crowdsec/export ``` **Expected Response:** + - HTTP 200 with `Content-Type: application/gzip` - `Content-Disposition: attachment; filename=crowdsec-config-YYYYMMDD-HHMMSS.tar.gz` - Valid tar.gz archive containing config files **Validation:** + ```bash tar -tzf crowdsec-export.tar.gz # Should list config files @@ -325,15 +356,18 @@ tar -tzf crowdsec-export.tar.gz **Objective:** Import a CrowdSec configuration package **Prerequisites:** + - Export file from TC-8 or test config archive **Steps:** + 1. Navigate to `/security/crowdsec` 2. Select file for import 3. Click "Import" button 4. Verify backup created and config applied **Curl Command:** + ```bash curl -X POST -b "$COOKIE_FILE" \ -F "file=@crowdsec-export.tar.gz" \ @@ -341,6 +375,7 @@ curl -X POST -b "$COOKIE_FILE" \ ``` **Expected Response:** + ```json {"status": "imported", "backup": "data/crowdsec.backup.YYYYMMDD-HHMMSS"} ``` @@ -352,17 +387,20 @@ curl -X POST -b "$COOKIE_FILE" \ **Objective:** Verify LAPI connectivity status **Curl Command:** + ```bash curl -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/lapi/health ``` **Expected Response (healthy):** + ```json {"healthy": true, "lapi_url": "http://127.0.0.1:8085", "status": 200} ``` **Expected Response (unhealthy):** + ```json {"healthy": false, "error": "LAPI unreachable", "lapi_url": "http://127.0.0.1:8085"} ``` @@ -374,21 +412,25 @@ curl -b "$COOKIE_FILE" \ **Objective:** Verify CrowdSec can be stopped **Steps:** + 1. With CrowdSec running, click "Stop" button 2. Verify status changes to "Stopped" **Curl Command:** + ```bash curl -X POST -b "$COOKIE_FILE" \ http://localhost:8080/api/v1/admin/crowdsec/stop ``` **Expected Response:** + ```json {"status": "stopped"} ``` **Validation:** + - PID file removed from `data/crowdsec/` - Status endpoint returns `{"running": false, "pid": 0}` @@ -397,6 +439,7 @@ curl -X POST -b "$COOKIE_FILE" \ ## Integration Test Script Requirements ### Script Location + `scripts/crowdsec_decision_integration.sh` ### Script Outline @@ -668,41 +711,50 @@ func TestCrowdsecDecisionsIntegration(t *testing.T) { ## Error Scenarios ### Invalid IP Format + ```bash curl -X POST -b "$COOKIE_FILE" \ -H "Content-Type: application/json" \ -d '{"ip": "invalid-ip"}' \ http://localhost:8080/api/v1/admin/crowdsec/ban ``` + **Expected:** HTTP 400 or underlying cscli error ### Missing IP Parameter + ```bash curl -X POST -b "$COOKIE_FILE" \ -H "Content-Type: application/json" \ -d '{"duration": "1h"}' \ http://localhost:8080/api/v1/admin/crowdsec/ban ``` + **Expected:** HTTP 400 `{"error": "ip is required"}` ### Empty IP String + ```bash curl -X POST -b "$COOKIE_FILE" \ -H "Content-Type: application/json" \ -d '{"ip": " "}' \ http://localhost:8080/api/v1/admin/crowdsec/ban ``` + **Expected:** HTTP 400 `{"error": "ip cannot be empty"}` ### CrowdSec Not Available + When `cscli` is not in PATH: **Expected:** HTTP 200 with `{"decisions": [], "error": "cscli not available or failed"}` ### Export When No Config + ```bash # When data/crowdsec doesn't exist curl -b "$COOKIE_FILE" http://localhost:8080/api/v1/admin/crowdsec/export ``` + **Expected:** HTTP 404 `{"error": "crowdsec config not found"}` --- diff --git a/docs/plans/current_spec.md b/docs/plans/current_spec.md index 2004e710..3cb626cb 100644 --- a/docs/plans/current_spec.md +++ b/docs/plans/current_spec.md @@ -15,6 +15,7 @@ Trivy has identified CVE-2025-62408 in c-ares 1.34.5-r0. The fix requires rebuil **No Dockerfile changes required** - the existing `apk upgrade` command will automatically pull the patched version on the next build. See the full remediation plan for: + - Root cause analysis - CVE details and impact assessment - Step-by-step implementation guide diff --git a/docs/plans/docs_to_issues_workflow.md b/docs/plans/docs_to_issues_workflow.md index 3b3a96c4..edeaad44 100644 --- a/docs/plans/docs_to_issues_workflow.md +++ b/docs/plans/docs_to_issues_workflow.md @@ -672,6 +672,7 @@ docs/ ### 7.2 Issue Tracking Each created issue includes footer: + ```markdown --- *Auto-created from [filename.md](link-to-source-commit)* @@ -746,17 +747,20 @@ console.log(JSON.stringify(result.data, null, 2)); ## 10. Implementation Phases ### Phase 1: Setup (15 min) + 1. Create `.github/workflows/docs-to-issues.yml` 2. Create `docs/issues/created/.gitkeep` 3. Create `docs/issues/_TEMPLATE.md` 4. Create `docs/issues/README.md` ### Phase 2: File Migration (30 min) + 1. Add frontmatter to existing files (in order of priority) 2. Test with dry_run mode 3. Create one test issue to verify ### Phase 3: Validation (15 min) + 1. Verify issue creation 2. Verify label creation 3. Verify project board integration diff --git a/docs/plans/ui_ux_bugfixes_spec.md b/docs/plans/ui_ux_bugfixes_spec.md index 9b16717e..2205a429 100644 --- a/docs/plans/ui_ux_bugfixes_spec.md +++ b/docs/plans/ui_ux_bugfixes_spec.md @@ -306,7 +306,7 @@ if (!status) return
No security s } ``` -2. **App.tsx** - Update routes: +1. **App.tsx** - Update routes: ```tsx // Remove: } /> diff --git a/docs/plans/waf_testing_plan.md b/docs/plans/waf_testing_plan.md index fe55c965..58656d2d 100644 --- a/docs/plans/waf_testing_plan.md +++ b/docs/plans/waf_testing_plan.md @@ -132,6 +132,7 @@ The hash is derived from content to ensure Caddy reloads when rules change. ### 2.3 Existing Integration Test Analysis The existing `coraza_integration.sh` tests: + - ✅ XSS payload blocking (``) - ✅ BLOCK mode (expects HTTP 403) - ✅ MONITOR mode switching (expects HTTP 200 after mode change) @@ -234,6 +235,7 @@ curl -s -X POST -H "Content-Type: application/json" \ **Objective:** Create a ruleset that blocks SQL injection patterns **Curl Command:** + ```bash echo "=== TC-1: Create SQLi Ruleset ===" @@ -252,6 +254,7 @@ echo "$RESP" | jq . ``` **Expected Response:** + ```json { "ruleset": { @@ -271,6 +274,7 @@ echo "$RESP" | jq . **Objective:** Create a ruleset that blocks XSS patterns **Curl Command:** + ```bash echo "=== TC-2: Create XSS Ruleset ===" @@ -294,6 +298,7 @@ echo "$RESP" | jq . **Objective:** Set WAF mode to blocking with a specific ruleset **Curl Command:** + ```bash echo "=== TC-3: Enable WAF (Block Mode) ===" @@ -317,6 +322,7 @@ sleep 5 ``` **Verification:** + ```bash # Check WAF status curl -s -b ${TMP_COOKIE} http://localhost:8080/api/v1/security/status | jq '.waf' @@ -362,6 +368,7 @@ echo "SQLi POST body: HTTP $RESP (expect 403)" ``` **Expected Results:** + - All requests return HTTP 403 --- @@ -371,6 +378,7 @@ echo "SQLi POST body: HTTP $RESP (expect 403)" **Objective:** Verify XSS patterns are blocked with HTTP 403 **Curl Commands:** + ```bash echo "=== TC-5: XSS Blocking ===" @@ -404,6 +412,7 @@ echo "XSS script tag (JSON): HTTP $RESP (expect 403)" ``` **Expected Results:** + - All requests return HTTP 403 --- @@ -413,6 +422,7 @@ echo "XSS script tag (JSON): HTTP $RESP (expect 403)" **Objective:** Verify requests pass but are logged in monitor mode **Curl Commands:** + ```bash echo "=== TC-6: Detection Mode ===" @@ -440,6 +450,7 @@ docker exec charon-waf-test sh -c 'tail -50 /var/log/caddy/access.log 2>/dev/nul ``` **Expected Results:** + - HTTP 200 response (request passes through) - WAF detection logged (in Caddy access logs or Coraza logs) @@ -450,6 +461,7 @@ docker exec charon-waf-test sh -c 'tail -50 /var/log/caddy/access.log 2>/dev/nul **Objective:** Verify both SQLi and XSS rules can be combined **Curl Commands:** + ```bash echo "=== TC-7: Multiple Rulesets (Combined) ===" @@ -498,6 +510,7 @@ echo "Combined - Legitimate: HTTP $RESP (expect 200)" **Objective:** Verify all rulesets are listed correctly **Curl Command:** + ```bash echo "=== TC-8: List Rulesets ===" @@ -506,6 +519,7 @@ echo "$RESP" | jq '.rulesets[] | {name, mode, last_updated}' ``` **Expected Response:** + ```json [ {"name": "sqli-protection", "mode": "", "last_updated": "..."}, @@ -521,6 +535,7 @@ echo "$RESP" | jq '.rulesets[] | {name, mode, last_updated}' **Objective:** Add and remove WAF rule exclusions for false positives **Curl Commands:** + ```bash echo "=== TC-9: WAF Rule Exclusions ===" @@ -548,6 +563,7 @@ echo "Delete exclusion: $RESP" **Objective:** Confirm WAF handler is present in running Caddy config **Curl Command:** + ```bash echo "=== TC-10: Verify Caddy Config ===" @@ -585,6 +601,7 @@ fi **Objective:** Verify ruleset can be deleted **Curl Commands:** + ```bash echo "=== TC-11: Delete Ruleset ===" @@ -793,33 +810,33 @@ Location: `backend/integration/waf_integration_test.go` package integration import ( - "context" - "os/exec" - "strings" - "testing" - "time" + "context" + "os/exec" + "strings" + "testing" + "time" ) // TestWAFIntegration runs the scripts/waf_integration.sh and ensures it completes successfully. func TestWAFIntegration(t *testing.T) { - t.Parallel() + t.Parallel() - ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute) - defer cancel() + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute) + defer cancel() - cmd := exec.CommandContext(ctx, "bash", "./scripts/waf_integration.sh") - cmd.Dir = "../.." + cmd := exec.CommandContext(ctx, "bash", "./scripts/waf_integration.sh") + cmd.Dir = "../.." - out, err := cmd.CombinedOutput() - t.Logf("waf_integration script output:\n%s", string(out)) + out, err := cmd.CombinedOutput() + t.Logf("waf_integration script output:\n%s", string(out)) - if err != nil { - t.Fatalf("waf integration failed: %v", err) - } + if err != nil { + t.Fatalf("waf integration failed: %v", err) + } - if !strings.Contains(string(out), "All WAF tests passed") { - t.Fatalf("unexpected script output, expected pass assertion not found") - } + if !strings.Contains(string(out), "All WAF tests passed") { + t.Fatalf("unexpected script output, expected pass assertion not found") + } } ``` diff --git a/docs/reports/qa_crowdsec_implementation.md b/docs/reports/qa_crowdsec_implementation.md index 06c73483..5bd58a1a 100644 --- a/docs/reports/qa_crowdsec_implementation.md +++ b/docs/reports/qa_crowdsec_implementation.md @@ -21,6 +21,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `.venv/bin/pre-commit run --all-files` - All hooks passed including: - Go Vet @@ -39,6 +40,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `cd backend && go build ./...` - No compilation errors @@ -49,6 +51,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `cd backend && go test ./...` - All test packages passed: - `internal/api/handlers` - 21.2s @@ -65,6 +68,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `cd frontend && npm run type-check` - TypeScript compilation: No errors @@ -75,6 +79,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `cd frontend && npm run test` - Results: - Test Files: **84 passed** @@ -110,6 +115,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `docker build --build-arg VCS_REF=$(git rev-parse HEAD) -t charon:local .` - Image built successfully: `sha256:ee53c99130393bdd8a09f1d06bd55e31f82676ecb61bd03842cbbafb48eeea01` - Frontend build: ✓ built in 6.77s @@ -122,6 +128,7 @@ All mandatory checks passed successfully. Several linting issues were found and **Status:** ✅ PASS **Details:** + - Ran: `bash scripts/crowdsec_startup_test.sh` - All 6 checks passed: @@ -135,6 +142,7 @@ All mandatory checks passed successfully. Several linting issues were found and | 6 | CrowdSec process running | ✅ PASS | **CrowdSec Components Verified:** + - LAPI: `{"status":"up"}` - Acquisition: Configured for Caddy logs at `/var/log/caddy/access.log` - Parsers: crowdsecurity/caddy-logs, geoip-enrich, http-logs, syslog-logs diff --git a/docs/reports/qa_uiux_testing_report.md b/docs/reports/qa_uiux_testing_report.md index 633d5117..3eaba8a9 100644 --- a/docs/reports/qa_uiux_testing_report.md +++ b/docs/reports/qa_uiux_testing_report.md @@ -26,11 +26,13 @@ **Command**: `npm run test` ### Results + - **Test Files**: 87 passed (87) - **Tests**: 799 passed, 2 skipped (801) - **Duration**: ~58 seconds ### Test Categories + | Category | Test Files | Description | |----------|------------|-------------| | Security Page | 6 files | Dashboard, loading overlays, error handling, spec tests | @@ -41,6 +43,7 @@ | Utils | 6 files | Utility function tests | ### Notable Test Suites + - **Security.loading.test.tsx**: 12 tests verifying loading overlay behavior - **Security.dashboard.test.tsx**: 18 tests for security dashboard card status - **Security.errors.test.tsx**: 13 tests for error handling and toast notifications @@ -54,6 +57,7 @@ **Command**: `npm run type-check` ### Results + - **Status**: ✅ Passed - **Errors**: 0 - **Compiler**: `tsc --noEmit` @@ -87,6 +91,7 @@ All TypeScript types are valid and properly defined across the frontend codebase | data/ | 93.33% | 100% | 80% | 95.83% | ### High Coverage Files (100%) + - `api/accessLists.ts` - `api/backups.ts` - `api/certificates.ts` @@ -105,6 +110,7 @@ All TypeScript types are valid and properly defined across the frontend codebase **Command**: `pre-commit run --all-files` ### Results + | Hook | Status | |------|--------| | Go Vet | ✅ Passed | @@ -117,6 +123,7 @@ All TypeScript types are valid and properly defined across the frontend codebase | Frontend Lint (Fix) | ✅ Passed | ### Backend Coverage + - **Backend Coverage**: 85.2% (minimum required: 85%) - **Status**: ✅ Coverage requirement met @@ -127,6 +134,7 @@ All TypeScript types are valid and properly defined across the frontend codebase **Command**: `npx markdownlint-cli2 "docs/**/*.md" "*.md"` ### Results + - **Status**: ✅ Passed - **Errors**: 0 in project files - **Note**: External pip package files (in `.venv/lib/`) showed 4 warnings which are expected and not part of the project codebase @@ -138,6 +146,7 @@ All TypeScript types are valid and properly defined across the frontend codebase **Command**: `npm run lint` ### Results + - **Errors**: 0 - **Warnings**: 6 @@ -148,7 +157,7 @@ All TypeScript types are valid and properly defined across the frontend codebase | e2e/tests/security-mobile.spec.ts | 289 | @typescript-eslint/no-unused-vars | 'onclick' assigned but never used | | src/pages/CrowdSecConfig.tsx | 212 | react-hooks/exhaustive-deps | Missing dependencies in useEffect | | src/pages/CrowdSecConfig.tsx | 715 | @typescript-eslint/no-explicit-any | Unexpected any type | -| src/pages/__tests__/CrowdSecConfig.spec.tsx | 258, 284, 317 | @typescript-eslint/no-explicit-any | Unexpected any type (test file) | +| src/pages/**tests**/CrowdSecConfig.spec.tsx | 258, 284, 317 | @typescript-eslint/no-explicit-any | Unexpected any type (test file) | **Note**: These warnings are non-critical and relate to existing code patterns. The `any` types in test files are acceptable for mocking purposes. The missing dependencies warning is a common pattern for intentional effect behavior. @@ -159,6 +168,7 @@ All TypeScript types are valid and properly defined across the frontend codebase ### No Critical Issues All primary QA checks passed. The project maintains: + - ✅ High test coverage (89.45% frontend, 85.2% backend) - ✅ Type safety with zero TypeScript errors - ✅ Code quality standards enforced via pre-commit diff --git a/docs/reports/rate_limit_fix_summary.md b/docs/reports/rate_limit_fix_summary.md index c6b0e262..8c30ce4d 100644 --- a/docs/reports/rate_limit_fix_summary.md +++ b/docs/reports/rate_limit_fix_summary.md @@ -7,81 +7,98 @@ ## Issues Identified and Fixed ### 1. **Caddy Admin API Not Accessible from Host** + **Problem:** The Caddy admin API was binding to `localhost:2019` inside the container, making it inaccessible from the host machine for monitoring and verification. **Root Cause:** Default Caddy admin API binding is `127.0.0.1:2019` for security. **Fix:** + - Added `AdminConfig` struct to `backend/internal/caddy/types.go` - Modified `GenerateConfig` in `backend/internal/caddy/config.go` to set admin listen address to `0.0.0.0:2019` - Updated `docker-entrypoint.sh` to include admin config in initial Caddy JSON **Files Modified:** + - `backend/internal/caddy/types.go` - Added `AdminConfig` type - `backend/internal/caddy/config.go` - Set `Admin.Listen = "0.0.0.0:2019"` - `docker-entrypoint.sh` - Initial config includes admin binding ### 2. **Missing RateLimitMode Field in SecurityConfig Model** + **Problem:** The runtime checks expected `RateLimitMode` (string) field but the model only had `RateLimitEnable` (bool). **Root Cause:** Inconsistency between field naming conventions - other security features use `*Mode` pattern (WAFMode, CrowdSecMode). **Fix:** + - Added `RateLimitMode` field to `SecurityConfig` model in `backend/internal/models/security_config.go` - Updated `UpdateConfig` handler to sync `RateLimitMode` with `RateLimitEnable` for backward compatibility **Files Modified:** + - `backend/internal/models/security_config.go` - Added `RateLimitMode string` - `backend/internal/api/handlers/security_handler.go` - Syncs mode field on config update ### 3. **GetStatus Handler Not Reading from Database** + **Problem:** The `GetStatus` API endpoint was reading from static environment config instead of the persisted `SecurityConfig` in the database. **Root Cause:** Handler was using `h.cfg` (static config from environment) with only partial overrides from `settings` table, not checking `security_configs` table. **Fix:** + - Completely rewrote `GetStatus` to prioritize database `SecurityConfig` over static config - Added proper fallback chain: DB SecurityConfig → Settings table overrides → Static config defaults - Ensures UI and API reflect actual runtime configuration **Files Modified:** + - `backend/internal/api/handlers/security_handler.go` - Rewrote `GetStatus` method ### 4. **computeEffectiveFlags Not Using Database SecurityConfig** + **Problem:** The `computeEffectiveFlags` method in caddy manager was reading from static config (`m.securityCfg`) instead of database `SecurityConfig`. **Root Cause:** Function started with static config values, then only applied `settings` table overrides, ignoring the primary `security_configs` table. **Fix:** + - Rewrote `computeEffectiveFlags` to read from `SecurityConfig` table first - Maintained fallback to static config and settings table overrides - Ensures Caddy config generation uses actual persisted security configuration **Files Modified:** + - `backend/internal/caddy/manager.go` - Rewrote `computeEffectiveFlags` method ### 5. **Invalid burst Field in Rate Limit Handler** + **Problem:** The generated Caddy config included a `burst` field that the `caddy-ratelimit` plugin doesn't support. **Root Cause:** Incorrect assumption about caddy-ratelimit plugin schema. **Error Message:** + ``` loading module 'rate_limit': decoding module config: http.handlers.rate_limit: json: unknown field "burst" ``` **Fix:** + - Removed `burst` field from rate limit handler configuration - Removed unused burst calculation logic - Added comment documenting that caddy-ratelimit uses sliding window algorithm without separate burst parameter **Files Modified:** + - `backend/internal/caddy/config.go` - Removed `burst` from `buildRateLimitHandler` ## Testing Results ### Before Fixes + ``` ✗ Caddy admin API not responding ✗ SecurityStatus showing rate_limit.enabled: false despite config @@ -90,6 +107,7 @@ http.handlers.rate_limit: json: unknown field "burst" ``` ### After Fixes + ``` ✓ Caddy admin API accessible at localhost:2119 ✓ SecurityStatus correctly shows rate_limit.enabled: true @@ -101,6 +119,7 @@ http.handlers.rate_limit: json: unknown field "burst" ``` ## Integration Test Command + ```bash bash ./scripts/rate_limit_integration.sh ``` @@ -108,6 +127,7 @@ bash ./scripts/rate_limit_integration.sh ## Architecture Improvements ### Configuration Priority Chain + The fixes established a clear configuration priority chain: 1. **Database SecurityConfig** (highest priority) @@ -123,6 +143,7 @@ The fixes established a clear configuration priority chain: - Provides defaults for fresh installations ### Consistency Between Components + - **GetStatus API**: Now reads from DB SecurityConfig first - **computeEffectiveFlags**: Now reads from DB SecurityConfig first - **UpdateConfig API**: Syncs RateLimitMode with RateLimitEnable @@ -131,17 +152,21 @@ The fixes established a clear configuration priority chain: ## Migration Considerations ### Backward Compatibility + - `RateLimitEnable` (bool) field maintained for backward compatibility - `UpdateConfig` automatically syncs `RateLimitMode` from `RateLimitEnable` - Existing SecurityConfig records work without migration ### Database Schema + No migration required - new field has appropriate defaults: + ```go RateLimitMode string `json:"rate_limit_mode"` // "disabled", "enabled" ``` ## Related Documentation + - [Rate Limiter Testing Plan](../plans/rate_limiter_testing_plan.md) - [Cerberus Security Documentation](../cerberus.md) - [API Documentation](../api.md#security-endpoints) @@ -151,27 +176,34 @@ RateLimitMode string `json:"rate_limit_mode"` // "disabled", "enabled" To verify rate limiting is working: 1. **Check Security Status:** + ```bash curl -s http://localhost:8080/api/v1/security/status | jq '.rate_limit' ``` + Should show: `{"enabled": true, "mode": "enabled"}` 2. **Check Caddy Config:** + ```bash curl -s http://localhost:2019/config/ | jq '.apps.http.servers.charon_server.routes[0].handle' | grep rate_limit ``` + Should find rate_limit handler in proxy route 3. **Test Enforcement:** + ```bash # Send requests exceeding limit for i in {1..5}; do curl -H "Host: your-domain.local" http://localhost/; done ``` + Should see HTTP 429 on requests exceeding limit ## Conclusion All rate limiting integration test issues have been resolved. The system now correctly: + - Reads SecurityConfig from database - Applies rate limiting when enabled in SecurityConfig - Generates valid Caddy configuration diff --git a/docs/reports/rate_limit_test_status.md b/docs/reports/rate_limit_test_status.md index f05e21b4..9173e3e4 100644 --- a/docs/reports/rate_limit_test_status.md +++ b/docs/reports/rate_limit_test_status.md @@ -10,26 +10,31 @@ Successfully fixed all rate limit integration test failures. The integration tes ## Root Causes Fixed ### 1. Caddy Admin API Binding (Infrastructure) + - **Issue**: Admin API bound to 127.0.0.1:2019 inside container, inaccessible from host - **Fix**: Changed binding to 0.0.0.0:2019 in `config.go` and `docker-entrypoint.sh` - **Files**: `backend/internal/caddy/config.go`, `docker-entrypoint.sh`, `backend/internal/caddy/types.go` ### 2. Missing RateLimitMode Field (Data Model) + - **Issue**: SecurityConfig model lacked RateLimitMode field - **Fix**: Added `RateLimitMode string` field to SecurityConfig model - **Files**: `backend/internal/models/security_config.go` ### 3. GetStatus Reading Wrong Source (Handler Logic) + - **Issue**: GetStatus read static config instead of database SecurityConfig - **Fix**: Rewrote GetStatus to prioritize DB SecurityConfig over static config - **Files**: `backend/internal/api/handlers/security_handler.go` ### 4. Configuration Priority Chain (Runtime Logic) + - **Issue**: `computeEffectiveFlags` read static config first, ignoring DB overrides - **Fix**: Completely rewrote priority chain: DB SecurityConfig → Settings table → Static config - **Files**: `backend/internal/caddy/manager.go` ### 5. Unsupported burst Field (Caddy Config) + - **Issue**: `caddy-ratelimit` plugin doesn't support `burst` parameter (sliding window only) - **Fix**: Removed burst field from rate_limit handler configuration - **Files**: `backend/internal/caddy/config.go`, `backend/internal/caddy/config_test.go` @@ -37,6 +42,7 @@ Successfully fixed all rate limit integration test failures. The integration tes ## Test Results ### ✅ Integration Test: PASSING + ``` === ALL RATE LIMIT TESTS PASSED === ✓ Request blocked with HTTP 429 as expected @@ -44,12 +50,15 @@ Successfully fixed all rate limit integration test failures. The integration tes ``` ### ✅ Unit Tests (Rate Limit Config): PASSING + - `TestBuildRateLimitHandler_UsesBurst` - Updated to verify burst NOT present - `TestBuildRateLimitHandler_DefaultBurst` - Updated to verify burst NOT present - All 11 rate limit handler tests passing ### ⚠️ Unrelated Test Failures + The following tests fail due to expecting old behavior (Settings table overrides everything): + - `TestSecurityHandler_GetStatus_RespectsSettingsTable` - `TestSecurityHandler_GetStatus_WAFModeFromSettings` - `TestSecurityHandler_GetStatus_RateLimitModeFromSettings` @@ -61,6 +70,7 @@ The following tests fail due to expecting old behavior (Settings table overrides ## Configuration Priority Chain (Correct Behavior) ### Highest Priority → Lowest Priority + 1. **Database SecurityConfig** (`security_configs` table, `name='default'`) - WAFMode, RateLimitMode, CrowdSecMode - Persisted via UpdateConfig API endpoint @@ -74,6 +84,7 @@ The following tests fail due to expecting old behavior (Settings table overrides ## Files Modified ### Core Implementation (8 files) + 1. `backend/internal/models/security_config.go` - Added RateLimitMode field 2. `backend/internal/caddy/manager.go` - Rewrote computeEffectiveFlags priority chain 3. `backend/internal/caddy/config.go` - Fixed admin binding, removed burst field @@ -84,17 +95,20 @@ The following tests fail due to expecting old behavior (Settings table overrides 8. `backend/internal/caddy/config_test.go` - Updated 3 tests to remove burst assertions ### Test Updates (1 file) + 9. `backend/internal/api/handlers/security_handler_audit_test.go` - Fixed TestSecurityHandler_GetStatus_SettingsOverride ## Next Steps ### Required Follow-up + 1. Update the 5 failing settings tests in `security_handler_settings_test.go` to test correct priority: - Tests should create DB SecurityConfig with `name='default'` - Tests should verify DB config takes precedence over Settings - Tests should verify Settings still work when no DB config exists ### Optional Enhancements + 1. Add integration tests for configuration priority chain 2. Document the priority chain in `docs/security.md` 3. Add API endpoint to view effective security config (showing which source is used) @@ -115,12 +129,14 @@ cd backend && go test ./... ## Technical Details ### caddy-ratelimit Plugin Behavior + - Uses **sliding window** algorithm (not token bucket) - Parameters: `key`, `window`, `max_events` - Does NOT support `burst` parameter - Returns HTTP 429 with `Retry-After` header when limit exceeded ### SecurityConfig Model Fields (Relevant) + ```go type SecurityConfig struct { Enabled bool `json:"enabled"` @@ -133,6 +149,7 @@ type SecurityConfig struct { ``` ### GetStatus Response Structure + ```json { "cerberus": {"enabled": true},