chore: repo cleanup by archiving plans / reports

2026-02-19 07:56:42 +00:00
parent 745d3afab5
commit f4ef79def3
412 changed files with 2605 additions and 1864 deletions
--- a/docs/plans/archive/crowdsec_lapi_auth_fix.md
+++ b/docs/plans/archive/crowdsec_lapi_auth_fix.md
@@ -0,0 +1,439 @@
+# CrowdSec LAPI Authentication Fix
+
+**Issue Reference**: Related to Issue #585 (CrowdSec Web Console Enrollment)
+**Status**: Ready for Implementation
+**Priority**: P1 (Blocking CrowdSec functionality)
+**Created**: 2026-02-03
+**Estimated Effort**: 2-4 hours
+
+## Executive Summary
+
+After a container rebuild, the CrowdSec integration fails with "access forbidden" when attempting to connect to the LAPI. This blocks all CrowdSec functionality including IP banning and web console enrollment testing.
+
+**Error Observed**:
+```json
+{
+  "level": "error",
+  "ts": 1770143945.8009417,
+  "logger": "crowdsec",
+  "msg": "failed to connect to LAPI, retrying in 10s: API error: access forbidden",
+  "instance_id": "99c91cc1",
+  "address": "http://127.0.0.1:8085"
+}
+```
+
+---
+
+## Root Cause Analysis
+
+### Finding 1: Invalid Static API Key
+
+**Problem**: User configured `CHARON_SECURITY_CROWDSEC_API_KEY=charonbouncerkey2024` in docker-compose.yml.
+
+**Why This Fails**: CrowdSec bouncer API keys must be **generated** by CrowdSec via `cscli bouncers add <name>`. The manually-specified key `charonbouncerkey2024` was never registered with CrowdSec LAPI, so the LAPI rejects authentication with "access forbidden".
+
+**Evidence**:
+- [registration.go](../../backend/internal/crowdsec/registration.go#L96-L106): `EnsureBouncerRegistered()` first checks for env var API key, returns it if present (without validation)
+- [register_bouncer.sh](../../configs/crowdsec/register_bouncer.sh): Script generates real API key via `cscli bouncers add`
+
+### Finding 2: Bouncer Not Auto-Registered on Start
+
+**Problem**: When CrowdSec agent starts, the bouncer is never registered automatically.
+
+**Evidence**:
+- [docker-entrypoint.sh](../../.docker/docker-entrypoint.sh#L223): Only **machine** registration occurs: `cscli machines add -a --force`
+- [crowdsec_handler.go#L191-L295](../../backend/internal/api/handlers/crowdsec_handler.go#L191-L295): `Start()` handler starts CrowdSec process but does NOT call bouncer registration
+- [crowdsec_handler.go#L1432-1478](../../backend/internal/api/handlers/crowdsec_handler.go#L1432-1478): `RegisterBouncer()` is a **separate** API endpoint (`POST /api/v1/admin/crowdsec/bouncer/register`) that must be called manually
+
+### Finding 3: Incorrect API URL Configuration (Minor)
+
+**Problem**: User configured `CHARON_SECURITY_CROWDSEC_API_URL=http://localhost:8080`.
+
+**Why This Is Wrong**:
+- CrowdSec LAPI listens on port **8085** (configured in entrypoint via sed)
+- Port 8080 is used by Charon management API
+- Code default is correct: `http://127.0.0.1:8085` (see [config.go#L60-64](../../backend/internal/caddy/config.go#L60-64))
+
+**Evidence from entrypoint**:
+```bash
+# Configure CrowdSec LAPI to use port 8085 to avoid conflict with Charon (port 8080)
+sed -i 's|listen_uri: 127.0.0.1:8080|listen_uri: 127.0.0.1:8085|g' /etc/crowdsec/config.yaml
+```
+
+### Data Flow Diagram
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                         Current (Broken) Flow                                │
+├─────────────────────────────────────────────────────────────────────────────┤
+│                                                                              │
+│  1. Container starts                                                         │
+│     ├─► docker-entrypoint.sh                                                 │
+│     │   ├─► mkdir /app/data/crowdsec                                        │
+│     │   ├─► Copy config to persistent storage                               │
+│     │   ├─► cscli machines add -a --force  ✅ Machine registered            │
+│     │   └─► ❌ NO bouncer registration                                       │
+│                                                                              │
+│  2. User enables CrowdSec via GUI                                            │
+│     ├─► POST /api/v1/admin/crowdsec/start                                   │
+│     │   ├─► SecurityConfig.CrowdSecMode = "local"                           │
+│     │   ├─► Start crowdsec process                                          │
+│     │   ├─► Wait for LAPI ready                                             │
+│     │   └─► ❌ NO bouncer registration                                       │
+│                                                                              │
+│  3. Caddy loads CrowdSec bouncer config                                      │
+│     ├─► GenerateConfig() in caddy/config.go                                 │
+│     │   ├─► apiKey := getCrowdSecAPIKey()                                   │
+│     │   │   └─► Returns "charonbouncerkey2024" from CHARON_SECURITY_...     │
+│     │   └─► CrowdSecApp{APIKey: "charonbouncerkey2024", ...}                │
+│                                                                              │
+│  4. Caddy CrowdSec bouncer tries to connect                                  │
+│     ├─► Connect to http://127.0.0.1:8085                                    │
+│     ├─► Send API key "charonbouncerkey2024"                                 │
+│     └─► ❌ LAPI responds: "access forbidden" (key not registered)           │
+│                                                                              │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## Requirements (EARS Notation)
+
+### REQ-1: API Key Validation
+**WHEN** a CrowdSec API key is provided via environment variable, **THE SYSTEM SHALL** validate that the key is actually registered with the LAPI before using it.
+
+**Acceptance Criteria**:
+- Validation occurs during CrowdSec start
+- Invalid keys trigger a warning log
+- System attempts auto-registration if validation fails
+
+### REQ-2: Auto-Registration on Start
+**WHEN** CrowdSec starts and no valid bouncer is registered, **THE SYSTEM SHALL** automatically register a bouncer and store the generated API key.
+
+**Acceptance Criteria**:
+- Bouncer registration happens automatically after LAPI is ready
+- Generated API key is persisted to `/etc/crowdsec/bouncers/caddy-bouncer.key`
+- Caddy config is regenerated with the new API key
+
+### REQ-3: Caddy Config Regeneration
+**WHEN** a new bouncer API key is generated, **THE SYSTEM SHALL** regenerate the Caddy configuration with the updated key.
+
+**Acceptance Criteria**:
+- Caddy config uses the newly generated API key
+- No container restart required
+- Bouncer connects successfully to LAPI
+
+### REQ-4: Upgrade Scenario Handling
+**WHEN** upgrading from a broken state (invalid static key), **THE SYSTEM SHALL** heal automatically without manual intervention.
+
+**Acceptance Criteria**:
+- Works for both fresh installs and upgrades
+- Volume-mounted data is preserved
+- No manual cleanup required
+
+---
+
+## Technical Design
+
+### Solution Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                         Fixed Flow                                           │
+├─────────────────────────────────────────────────────────────────────────────┤
+│                                                                              │
+│  1. Container starts (unchanged)                                             │
+│     └─► docker-entrypoint.sh                                                 │
+│         └─► cscli machines add -a --force                                   │
+│                                                                              │
+│  2. User enables CrowdSec via GUI                                            │
+│     └─► POST /api/v1/admin/crowdsec/start                                   │
+│         ├─► Start crowdsec process                                          │
+│         ├─► Wait for LAPI ready                                             │
+│         ├─► 🆕 EnsureBouncerRegistered()                                     │
+│         │   ├─► Check if env key is valid                                   │
+│         │   ├─► If invalid, run register_bouncer.sh                         │
+│         │   └─► Store key in database/file                                  │
+│         ├─► 🆕 Regenerate Caddy config with new API key                     │
+│         └─► Return success                                                   │
+│                                                                              │
+│  3. Caddy loads CrowdSec bouncer config                                      │
+│     ├─► getCrowdSecAPIKey()                                                 │
+│     │   ├─► Check env var first                                             │
+│     │   ├─► 🆕 Check file /etc/crowdsec/bouncers/caddy-bouncer.key         │
+│     │   └─► 🆕 Check database settings table                                │
+│     └─► CrowdSecApp{APIKey: <valid-key>, ...}                               │
+│                                                                              │
+│  4. Caddy CrowdSec bouncer connects                                          │
+│     └─► ✅ LAPI accepts valid key                                            │
+│                                                                              │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+### Implementation Changes
+
+#### Change 1: Update `Start()` Handler to Register Bouncer
+**File**: `backend/internal/api/handlers/crowdsec_handler.go`
+
+**Location**: After LAPI readiness check (line ~290)
+
+```go
+// After confirming LAPI is ready, ensure bouncer is registered
+if lapiReady {
+    // Register bouncer if needed (idempotent)
+    apiKey, regErr := h.ensureBouncerRegistration(ctx)
+    if regErr != nil {
+        logger.Log().WithError(regErr).Warn("Failed to register bouncer, CrowdSec may not enforce decisions")
+    } else if apiKey != "" {
+        // Store the API key for Caddy config generation
+        h.storeAPIKey(ctx, apiKey)
+
+        // Regenerate Caddy config with new API key
+        if h.CaddyManager != nil {
+            if err := h.CaddyManager.ReloadConfig(ctx); err != nil {
+                logger.Log().WithError(err).Warn("Failed to reload Caddy config with new bouncer key")
+            }
+        }
+    }
+}
+```
+
+#### Change 2: Add `ensureBouncerRegistration()` Method
+**File**: `backend/internal/api/handlers/crowdsec_handler.go`
+
+```go
+// ensureBouncerRegistration checks if bouncer is registered and registers if needed.
+// Returns the API key (empty string if already registered via env var).
+func (h *CrowdsecHandler) ensureBouncerRegistration(ctx context.Context) (string, error) {
+    // First check if env var key is actually valid
+    envKey := getLAPIKey()
+    if envKey != "" {
+        // Validate the key is actually registered
+        if h.validateBouncerKey(ctx, envKey) {
+            return "", nil // Key is valid, nothing to do
+        }
+        logger.Log().Warn("Env-provided CrowdSec API key is invalid, will register new bouncer")
+    }
+
+    // Check if key file already exists
+    keyFile := "/etc/crowdsec/bouncers/caddy-bouncer.key"
+    if data, err := os.ReadFile(keyFile); err == nil {
+        key := strings.TrimSpace(string(data))
+        if key != "" && h.validateBouncerKey(ctx, key) {
+            return key, nil // Key file is valid
+        }
+    }
+
+    // Register new bouncer using script
+    scriptPath := "/usr/local/bin/register_bouncer.sh"
+    output, err := h.CmdExec.Execute(ctx, "bash", scriptPath)
+    if err != nil {
+        return "", fmt.Errorf("bouncer registration failed: %w: %s", err, string(output))
+    }
+
+    // Extract API key from output (last non-empty line)
+    lines := strings.Split(strings.TrimSpace(string(output)), "\n")
+    apiKey := ""
+    for i := len(lines) - 1; i >= 0; i-- {
+        line := strings.TrimSpace(lines[i])
+        if len(line) >= 32 && !strings.Contains(line, " ") {
+            apiKey = line
+            break
+        }
+    }
+
+    if apiKey == "" {
+        return "", fmt.Errorf("bouncer registration output did not contain API key")
+    }
+
+    return apiKey, nil
+}
+
+// validateBouncerKey checks if an API key is actually registered with CrowdSec.
+func (h *CrowdsecHandler) validateBouncerKey(ctx context.Context, key string) bool {
+    // Use cscli bouncers list to check if 'caddy-bouncer' exists
+    checkCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
+    defer cancel()
+
+    output, err := h.CmdExec.Execute(checkCtx, "cscli", "bouncers", "list", "-o", "json")
+    if err != nil {
+        return false
+    }
+
+    // Parse JSON and check for caddy-bouncer
+    var bouncers []struct {
+        Name   string `json:"name"`
+        APIKey string `json:"api_key"`
+    }
+    if err := json.Unmarshal(output, &bouncers); err != nil {
+        return false
+    }
+
+    for _, b := range bouncers {
+        if b.Name == "caddy-bouncer" {
+            // Note: cscli doesn't return the full API key, just that it exists
+            // We trust it's valid if the bouncer is registered
+            return true
+        }
+    }
+
+    return false
+}
+```
+
+#### Change 3: Update `getCrowdSecAPIKey()` to Check Key File
+**File**: `backend/internal/caddy/config.go`
+
+**Location**: `getCrowdSecAPIKey()` function (line ~1129)
+
+```go
+// getCrowdSecAPIKey retrieves the CrowdSec bouncer API key.
+// Priority: environment variable > key file > empty
+func getCrowdSecAPIKey() string {
+    // Check environment variables first
+    envVars := []string{
+        "CROWDSEC_API_KEY",
+        "CROWDSEC_BOUNCER_API_KEY",
+        "CERBERUS_SECURITY_CROWDSEC_API_KEY",
+        "CHARON_SECURITY_CROWDSEC_API_KEY",
+        "CPM_SECURITY_CROWDSEC_API_KEY",
+    }
+
+    for _, key := range envVars {
+        if val := os.Getenv(key); val != "" {
+            return val
+        }
+    }
+
+    // Check key file (generated by register_bouncer.sh)
+    keyFile := "/etc/crowdsec/bouncers/caddy-bouncer.key"
+    if data, err := os.ReadFile(keyFile); err == nil {
+        key := strings.TrimSpace(string(data))
+        if key != "" {
+            return key
+        }
+    }
+
+    return ""
+}
+```
+
+#### Change 4: Add CaddyManager to CrowdsecHandler
+**File**: `backend/internal/api/handlers/crowdsec_handler.go`
+
+This may already exist, but ensure the handler can trigger Caddy config reload:
+
+```go
+type CrowdsecHandler struct {
+    DB           *gorm.DB
+    BinPath      string
+    DataDir      string
+    Executor     CrowdsecExecutor
+    CmdExec      CommandExecutor
+    LAPIMaxWait  time.Duration
+    LAPIPollInterval time.Duration
+    CaddyManager *caddy.Manager  // ADD: For config reload
+}
+```
+
+---
+
+## Immediate Workaround (User Action)
+
+While the fix is being implemented, the user can:
+
+1. **Remove the static API key** from docker-compose.yml:
+   ```yaml
+   # REMOVE or comment out this line:
+   # - CHARON_SECURITY_CROWDSEC_API_KEY=charonbouncerkey2024
+   ```
+
+2. **Fix the API URL**:
+   ```yaml
+   # Change from:
+   - CHARON_SECURITY_CROWDSEC_API_URL=http://localhost:8080
+   # To:
+   - CHARON_SECURITY_CROWDSEC_API_URL=http://localhost:8085
+   ```
+
+3. **Manually register bouncer** after container starts:
+   ```bash
+   docker exec -it charon /usr/local/bin/register_bouncer.sh
+   ```
+
+4. **Restart container** to pick up the new key:
+   ```bash
+   docker compose restart charon
+   ```
+
+---
+
+## Test Scenarios
+
+### Scenario 1: Fresh Install
+1. Start container with no CrowdSec env vars
+2. Enable CrowdSec via GUI
+3. **Expected**: Bouncer auto-registers, no errors in logs
+
+### Scenario 2: Upgrade from Invalid Key
+1. Start container with `CHARON_SECURITY_CROWDSEC_API_KEY=invalid`
+2. Enable CrowdSec via GUI
+3. **Expected**: System detects invalid key, registers new bouncer, logs warning
+
+### Scenario 3: Upgrade with Valid Key File
+1. Container has `/etc/crowdsec/bouncers/caddy-bouncer.key` from previous run
+2. Restart container, enable CrowdSec
+3. **Expected**: Uses existing key file, no re-registration
+
+### Scenario 4: API URL Misconfiguration
+1. Set `CHARON_SECURITY_CROWDSEC_API_URL=http://localhost:8080` (wrong port)
+2. Enable CrowdSec
+3. **Expected**: Uses default 8085 port, logs warning about ignored URL
+
+---
+
+## Implementation Checklist
+
+- [ ] **Task 1**: Add `validateBouncerKey()` method to crowdsec_handler.go
+- [ ] **Task 2**: Add `ensureBouncerRegistration()` method
+- [ ] **Task 3**: Update `Start()` to call bouncer registration after LAPI ready
+- [ ] **Task 4**: Update `getCrowdSecAPIKey()` in caddy/config.go to read from key file
+- [ ] **Task 5**: Add integration test for bouncer auto-registration
+- [ ] **Task 6**: Update documentation to clarify API key generation
+
+---
+
+## Files to Modify
+
+| File | Change |
+|------|--------|
+| `backend/internal/api/handlers/crowdsec_handler.go` | Add validation and auto-registration |
+| `backend/internal/caddy/config.go` | Update `getCrowdSecAPIKey()` |
+| `docs/docker-compose.yml` (examples) | Remove/update API key examples |
+| `README.md` or `SECURITY.md` | Clarify CrowdSec setup |
+
+---
+
+## Risk Assessment
+
+| Risk | Likelihood | Impact | Mitigation |
+|------|------------|--------|------------|
+| Breaking existing valid keys | Low | Medium | Only re-register if validation fails |
+| register_bouncer.sh not present | Low | High | Check script existence before calling |
+| Caddy reload fails | Low | Medium | Continue without bouncer, log warning |
+| Race condition on startup | Low | Low | CrowdSec must finish starting first |
+
+---
+
+## References
+
+- [CrowdSec bouncer registration](https://doc.crowdsec.net/docs/bouncers/intro)
+- [Caddy CrowdSec Plugin](https://github.com/hslatman/caddy-crowdsec-bouncer)
+- [Issue #585 - CrowdSec Web Console Enrollment](https://github.com/Wikid82/Charon/issues/585)
+- [register_bouncer.sh](../../configs/crowdsec/register_bouncer.sh)
+- [docker-entrypoint.sh](../../.docker/docker-entrypoint.sh)
+
+---
+
+**Last Updated**: 2026-02-03
+**Owner**: TBD