Files

GitHub Actions 93894c517b fix(security): resolve API key logging vulnerability and enhance import validation

Critical security fix addressing CWE-312/315/359 (Cleartext Storage/Cookie
Storage/Privacy Exposure) where CrowdSec bouncer API keys were logged in cleartext.
Implemented maskAPIKey() utility to show only first 4 and last 4 characters,
protecting sensitive credentials in production logs.

Enhanced CrowdSec configuration import validation with:
- Zip bomb protection via 100x compression ratio limit
- Format validation rejecting zip archives (only tar.gz allowed)
- CrowdSec-specific YAML structure validation
- Rollback mechanism on validation failures

UX improvement: moved CrowdSec API key display from Security Dashboard to
CrowdSec Config page for better logical organization.

Comprehensive E2E test coverage:
- Created 10 test scenarios including valid import, missing files, invalid YAML,
  zip bombs, wrong formats, and corrupted archives
- 87/108 E2E tests passing (81% pass rate, 0 regressions)

Security validation:
- CodeQL: 0 CWE-312/315/359 findings (vulnerability fully resolved)
- Docker Image: 7 HIGH base image CVEs documented (non-blocking, Debian upstream)
- Pre-commit hooks: 13/13 passing (fixed 23 total linting issues)

Backend coverage: 82.2% (+1.1%)
Frontend coverage: 84.19% (+0.3%)

2026-02-04 00:12:13 +00:00

60 KiB

Raw Blame History

CrowdSec API Enhancement Plan

Date: 2026-02-03 Author: GitHub Copilot Planning Agent Status: Draft Priority: CRITICAL (P0 Security Issue) Issues: #586, QA Report Failures (crowdsec-diagnostics.spec.ts), CodeQL CWE-312/315/359

🚨 CRITICAL SECURITY ALERT 🚨

CodeQL has identified a CRITICAL security vulnerability that MUST be fixed BEFORE any other work begins.

Vulnerability Details:

Location: backend/internal/api/handlers/crowdsec_handler.go:1378
Function: logBouncerKeyBanner()
Issue: API keys logged in cleartext to application logs
CVEs:
- CWE-312: Cleartext Storage of Sensitive Information
- CWE-315: Cleartext Storage in Cookie (potential)
- CWE-359: Exposure of Private Personal Information
Severity: CRITICAL
Priority: P0 (MUST FIX FIRST)

Security Impact:

✅ API keys stored in plaintext in log files
✅ Logs may be shipped to external services (CloudWatch, Splunk, etc.)
✅ Logs accessible to unauthorized users with file system access
✅ Logs often stored unencrypted on disk
✅ Potential compliance violations (GDPR, PCI-DSS, SOC 2)

Proof of Vulnerable Code:

// Line 1366-1378
func (h *CrowdsecHandler) logBouncerKeyBanner(apiKey string) {
	banner := `
════════════════════════════════════════════════════════════════════
🔐 CrowdSec Bouncer Registered Successfully
────────────────────────────────────────────────────────────────────
Bouncer Name: %s
API Key:      %s  // ❌ EXPOSES FULL API KEY IN LOGS
Saved To:     %s
────────────────────────────────────────────────────────────────────
...
	logger.Log().Infof(banner, bouncerName, apiKey, bouncerKeyFile)  // ❌ LOGS SECRET
}

Executive Summary

This plan addresses ONE CRITICAL SECURITY ISSUE and three interconnected CrowdSec API issues identified in QA testing and CodeQL scanning:

Key Findings

SECURITY ISSUE (P0 - MUST FIX FIRST): 0. CodeQL API Key Exposure: CRITICAL vulnerability in logging

❌ API keys logged in cleartext at line 1378
❌ Violates CWE-312 (Cleartext Storage)
❌ Violates CWE-315 (Cookie Storage Risk)
❌ Violates CWE-359 (Privacy Exposure)
Fix Required: Implement secure masking for API keys in logs
Estimated Time: 2 hours

API/TEST ISSUES:

Issue 1 (Files API Split): Backend already has correct separation
- ✅ GET /admin/crowdsec/files returns list
- ✅ GET /admin/crowdsec/file?path=... returns content
- ❌ Test calls /files?path=... (wrong endpoint)
- Fix Required: 1-line test correction
Issue 2 (Config Retrieval): Feature already implemented
- Backend ReadFile handler exists and works
- Frontend correctly uses /admin/crowdsec/file?path=...
- Test failure caused by Issue 1
- Fix Required: Same test correction as Issue 1
Issue 3 (Import Validation): Enhancement opportunity
- Import functionality exists and works
- Missing: Comprehensive validation pre-import
- Missing: Better error messages
- Fix Required: Add validation layer

Revised Implementation Scope

Issue	Original Estimate	Revised Estimate	Change Reason
Security Issue	N/A	2 hours (CRITICAL)	CodeQL finding - P0
Issue 1	3-4 hours (API split)	30 min (test fix)	No API changes needed
Issue 2	2-3 hours (implement feature)	0 hours (already done)	Already fully implemented
Issue 3	4-5 hours (import fixes)	4-5 hours (validation)	Scope unchanged
Issue 4 (UX)	N/A	2-3 hours (NEW)	Move API key to config page
Total	9-12 hours	8.5-10.5 hours	Includes security + UX

Impact Assessment

Issue	Severity	User Impact	Test Impact	Security Risk
API Key Logging	CRITICAL	Secrets exposed	N/A	HIGH
Files API Test Bug	LOW	None (API works)	1 E2E test failing	NONE
Config Retrieval	NONE	Feature works	Dependent on Issue 1 fix	NONE
Import Validation	MEDIUM	Poor error UX	Tests passing but coverage gaps	LOW
API Key Location	LOW	Poor UX	None	NONE

Recommendation:

Sprint 0 (P0): Fix API key logging vulnerability IMMEDIATELY
Sprint 1: Fix test bug (unblocks QA)
Sprint 2: Implement import validation
Sprint 3: Move CrowdSec API key to config page (UX improvement)

Research Findings

Current Implementation Analysis

Backend Files

File Path	Purpose	Lines	Status
`backend/internal/api/handlers/crowdsec_handler.go`	Main CrowdSec handler	2057	✅ Complete
`backend/internal/api/routes/routes.go`	Route registration	650	✅ Complete

API Endpoint Architecture (Already Correct)

// Current route registration (lines 2023-2052)
rg.GET("/admin/crowdsec/files", h.ListFiles)        // Returns {files: [...]}
rg.GET("/admin/crowdsec/file", h.ReadFile)          // Returns {content: string}
rg.POST("/admin/crowdsec/file", h.WriteFile)        // Updates config
rg.POST("/admin/crowdsec/import", h.ImportConfig)   // Imports tar.gz/zip
rg.GET("/admin/crowdsec/export", h.ExportConfig)    // Exports to tar.gz

Analysis: Two separate endpoints already exist with clear separation of concerns. This follows REST principles.

Handler Implementation (Already Correct)

ListFiles Handler (Line 525-545):

func (h *CrowdsecHandler) ListFiles(c *gin.Context) {
    var files []string
    // Walks DataDir and collects file paths
    c.JSON(http.StatusOK, gin.H{"files": files})  // ✅ Returns array
}

ReadFile Handler (Line 547-574):

func (h *CrowdsecHandler) ReadFile(c *gin.Context) {
    rel := c.Query("path")  // Gets ?path= param
    if rel == "" {
        c.JSON(http.StatusBadRequest, gin.H{"error": "path required"})
        return
    }
    // Reads file content
    c.JSON(http.StatusOK, gin.H{"content": string(data)})  // ✅ Returns content
}

Status: ✅ Implementation is correct and follows REST principles.

Frontend Integration (Already Correct)

File: frontend/src/api/crowdsec.ts (Lines 91-94)

export async function readCrowdsecFile(path: string) {
  const resp = await client.get<{ content: string }>(
    `/admin/crowdsec/file?path=${encodeURIComponent(path)}`  // ✅ Correct endpoint
  )
  return resp.data
}

Status: ✅ Frontend correctly calls /admin/crowdsec/file (singular).

Test Failure Root Cause

File: tests/security/crowdsec-diagnostics.spec.ts (Lines 323-355)

// Step 1: Get file list - ✅ CORRECT
const listResponse = await request.get('/api/v1/admin/crowdsec/files');
const fileList = files.files as string[];
const configPath = fileList.find((f) => f.includes('config.yaml'));

// Step 2: Retrieve file content - ❌ WRONG ENDPOINT
const contentResponse = await request.get(
  `/api/v1/admin/crowdsec/files?path=${encodeURIComponent(configPath)}`  // ❌ Should be /file
);

expect(contentResponse.ok()).toBeTruthy();
const content = await contentResponse.json();
expect(content).toHaveProperty('content');  // ❌ FAILS - Gets {files: [...]}

Root Cause: Test uses /files?path=... (plural) instead of /file?path=... (singular).

Status: ❌ Test bug, not API bug

Proposed Solution

Sprint 0: Critical Security Fix (P0 - BLOCK ALL OTHER WORK)

Duration: 2 hours Priority: P0 (CRITICAL - No other work can proceed) Blocker: YES - Must be completed before Supervisor Review

Security Vulnerability Details

File: backend/internal/api/handlers/crowdsec_handler.go Function: logBouncerKeyBanner() (Lines 1366-1378) Vulnerable Line: 1378

Current Vulnerable Implementation:

func (h *CrowdsecHandler) logBouncerKeyBanner(apiKey string) {
	banner := `
════════════════════════════════════════════════════════════════════
🔐 CrowdSec Bouncer Registered Successfully
────────────────────────────────────────────────────────────────────
Bouncer Name: %s
API Key:      %s  // ❌ EXPOSES FULL SECRET
Saved To:     %s
────────────────────────────────────────────────────────────────────
💡 TIP: If connecting to an EXTERNAL CrowdSec instance, copy this
   key to your docker-compose.yml as CHARON_SECURITY_CROWDSEC_API_KEY
════════════════════════════════════════════════════════════════════`
	logger.Log().Infof(banner, bouncerName, apiKey, bouncerKeyFile)  // ❌ LOGS SECRET
}

Secure Fix Implementation

Step 1: Create Secure Masking Utility

Add to backend/internal/api/handlers/crowdsec_handler.go (after line 2057):

// maskAPIKey redacts an API key for safe logging, showing only prefix/suffix.
// Format: "abc1...xyz9" (first 4 + last 4 chars, or less if key is short)
func maskAPIKey(key string) string {
	if key == "" {
		return "[empty]"
	}

	// For very short keys (< 16 chars), mask completely
	if len(key) < 16 {
		return "[REDACTED]"
	}

	// Show first 4 and last 4 characters only
	// Example: "abc123def456" -> "abc1...f456"
	return fmt.Sprintf("%s...%s", key[:4], key[len(key)-4:])
}

// validateAPIKeyFormat performs basic validation on API key format.
// Returns true if the key looks valid (length, charset).
func validateAPIKeyFormat(key string) bool {
	if len(key) < 16 || len(key) > 128 {
		return false
	}

	// API keys should be alphanumeric (base64-like)
	for _, r := range key {
		if !((r >= 'a' && r <= 'z') ||
		     (r >= 'A' && r <= 'Z') ||
		     (r >= '0' && r <= '9') ||
		     r == '-' || r == '_' || r == '+' || r == '/') {
			return false
		}
	}

	return true
}

Step 2: Update Logging Function

Replace logBouncerKeyBanner() (Lines 1366-1378):

// logBouncerKeyBanner logs bouncer registration with MASKED API key.
func (h *CrowdsecHandler) logBouncerKeyBanner(apiKey string) {
	// Security: NEVER log full API keys - mask for safe display
	maskedKey := maskAPIKey(apiKey)

	// Validate key format for integrity check
	validFormat := validateAPIKeyFormat(apiKey)
	if !validFormat {
		logger.Log().Warn("Bouncer API key has unexpected format - may be invalid")
	}

	banner := `
════════════════════════════════════════════════════════════════════
🔐 CrowdSec Bouncer Registered Successfully
────────────────────────────────────────────────────────────────────
Bouncer Name: %s
API Key:      %s  ✅ (Key is saved securely to file)
Saved To:     %s
────────────────────────────────────────────────────────────────────
💡 TIP: If connecting to an EXTERNAL CrowdSec instance, copy this
   key from %s to your docker-compose.yml

⚠️  SECURITY: API keys are sensitive credentials. The full key is
   saved to the file above and will NOT be displayed again.
════════════════════════════════════════════════════════════════════`

	logger.Log().Infof(banner, bouncerName, maskedKey, bouncerKeyFile, bouncerKeyFile)
}

Step 3: Audit All API Key Usage

Check these locations for additional exposures:

HTTP Responses (VERIFIED SAFE):
- No c.JSON() calls return API keys
- Start/Stop/Status endpoints do not expose keys ✅
Error Messages:
- Ensure no error messages inadvertently log API keys
- Use generic error messages for auth failures
Environment Variables:
- Document proper secret handling in README
- Never log environment variable contents

Unit Tests

File: backend/internal/api/handlers/crowdsec_handler_test.go

func TestMaskAPIKey(t *testing.T) {
	tests := []struct {
		name     string
		input    string
		expected string
	}{
		{
			name:     "normal key",
			input:    "abc123def456ghi789",
			expected: "abc1...h789",
		},
		{
			name:     "short key (masked completely)",
			input:    "shortkey123",
			expected: "[REDACTED]",
		},
		{
			name:     "empty key",
			input:    "",
			expected: "[empty]",
		},
		{
			name:     "minimum length key",
			input:    "abcd1234efgh5678",
			expected: "abcd...5678",
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			result := maskAPIKey(tt.input)
			assert.Equal(t, tt.expected, result)

			// Security check: masked value must not contain full key
			if len(tt.input) >= 16 {
				assert.NotContains(t, result, tt.input[4:len(tt.input)-4])
			}
		})
	}
}

func TestValidateAPIKeyFormat(t *testing.T) {
	tests := []struct {
		name  string
		key   string
		valid bool
	}{
		{"valid base64-like", "abc123DEF456ghi789XYZ", true},
		{"too short", "short", false},
		{"too long", strings.Repeat("a", 130), false},
		{"invalid chars", "key-with-special-#$%", false},
		{"valid with dash", "abc-123-def-456-ghi", true},
		{"valid with underscore", "abc_123_def_456_ghi", true},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			result := validateAPIKeyFormat(tt.key)
			assert.Equal(t, tt.valid, result)
		})
	}
}

func TestLogBouncerKeyBanner_NoSecretExposure(t *testing.T) {
	// Capture log output
	var logOutput bytes.Buffer
	logger.SetOutput(&logOutput)
	defer logger.SetOutput(os.Stderr)

	handler := &CrowdsecHandler{}
	testKey := "super-secret-api-key-12345678"

	handler.logBouncerKeyBanner(testKey)

	logText := logOutput.String()

	// Security assertions: Full key must NOT appear in logs
	assert.NotContains(t, logText, testKey, "Full API key must not appear in logs")
	assert.Contains(t, logText, "supe...5678", "Masked key should appear")
	assert.Contains(t, logText, "[SECURITY]", "Security warning should be present")
}

Security Validation

Manual Testing:

# 1. Run backend with test key
cd backend
CROWDSEC_API_KEY="test-secret-key-123456789" go run cmd/main.go

# 2. Trigger bouncer registration
curl -X POST http://localhost:8080/api/v1/admin/crowdsec/start

# 3. Check logs - API key should be masked
grep "API Key:" /var/log/charon/app.log
# Expected: "API Key: test...6789" NOT "test-secret-key-123456789"

# 4. Run unit tests
go test ./backend/internal/api/handlers -run TestMaskAPIKey -v
go test ./backend/internal/api/handlers -run TestLogBouncerKeyBanner -v

CodeQL Re-scan:

# After fix, re-run CodeQL scan
.github/skills/scripts/skill-runner.sh security-codeql-scan

# Expected: CWE-312/315/359 findings should be RESOLVED

Acceptance Criteria

maskAPIKey() utility function implemented
validateAPIKeyFormat() validation function implemented
logBouncerKeyBanner() updated to use masked keys
Unit tests for masking utility (100% coverage)
Unit tests verify no full key exposure in logs
Manual testing confirms masked output
CodeQL scan passes with 0 CWE-312/315/359 findings
All existing tests still pass
Documentation updated with security best practices

Documentation Updates

File: docs/security/api-key-handling.md (Create new)

# API Key Security Guidelines

## Logging Best Practices

**NEVER** log sensitive credentials in plaintext. Always mask API keys, tokens, and passwords.

### Masking Implementation

```go
// ✅ GOOD: Masked key
logger.Infof("API Key: %s", maskAPIKey(apiKey))

// ❌ BAD: Full key exposure
logger.Infof("API Key: %s", apiKey)

Key Storage

Store keys in secure files with restricted permissions (0600)
Use environment variables for secrets
Never commit keys to version control
Rotate keys regularly
Use separate keys per environment (dev/staging/prod)

Compliance

This implementation addresses:

CWE-312: Cleartext Storage of Sensitive Information
CWE-315: Cleartext Storage in Cookie
CWE-359: Exposure of Private Personal Information
OWASP A02:2021 - Cryptographic Failures


**Update**: `README.md` - Add security section

```markdown
## Security Considerations

### API Key Management

CrowdSec API keys are sensitive credentials. Charon implements secure key handling:

- ✅ Keys are masked in logs (show first 4 + last 4 chars only)
- ✅ Keys are stored in files with restricted permissions
- ✅ Keys are never sent in HTTP responses
- ✅ Keys are never stored in cookies

**Best Practices:**
1. Use environment variables for production deployments
2. Rotate keys regularly
3. Monitor access logs for unauthorized attempts
4. Use different keys per environment

Phase 1: Test Bug Fix (Issue 1 & 2)

Duration: 30 minutes Priority: P1 (Blocked by Sprint 0) Depends On: Sprint 0 must complete first

E2E Test Fix

File: tests/security/crowdsec-diagnostics.spec.ts (Lines 320-360)

Change Required:

- const contentResponse = await request.get(
-   `/api/v1/admin/crowdsec/files?path=${encodeURIComponent(configPath)}`
- );
+ const contentResponse = await request.get(
+   `/api/v1/admin/crowdsec/file?path=${encodeURIComponent(configPath)}`
+ );

Explanation: Change plural /files to singular /file to match API design.

Acceptance Criteria:

Test uses correct endpoint /admin/crowdsec/file?path=...
Response contains {content: string, path: string}
Test passes on all browsers (Chromium, Firefox, WebKit)

Validation Command:

# Test against Docker environment
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e
npx playwright test tests/security/crowdsec-diagnostics.spec.ts --project=chromium

# Expected: Test passes

Sprint 2: Import Validation Enhancement (Issue 3)

Duration: 5-6 hours Priority: P1 (Blocked by Sprint 0) Depends On: Sprint 0 must complete first Supervisor Enhancement: Added zip bomb protection

Enhanced Validation Architecture

Problem: Current ImportConfig handler (lines 378-457) lacks:

Archive format validation (accepts any file)
File size limits (no protection against zip bombs)
Required file validation (doesn't check for config.yaml)
YAML syntax validation (imports broken configs)
Rollback mechanism on validation failures

Validation Strategy

New Architecture:

Upload → Format Check → Size Check → Extract → Structure Validation → YAML Validation → Commit
         ↓ fail         ↓ fail       ↓ fail    ↓ fail                ↓ fail
         Reject 422     Reject 413   Rollback  Rollback             Rollback

Implementation Details

File: backend/internal/api/handlers/crowdsec_handler.go (Add at line ~2060)

// Configuration validator
type ConfigArchiveValidator struct {
    MaxSize             int64    // 50MB compressed default
    MaxUncompressed     int64    // 500MB uncompressed default
    MaxCompressionRatio float64  // 100x default
    RequiredFiles       []string // config.yaml minimum
}

func (v *ConfigArchiveValidator) Validate(archivePath string) error {
    // 1. Check file size
    info, err := os.Stat(archivePath)
    if err != nil {
        return fmt.Errorf("stat archive: %w", err)
    }
    if info.Size() > v.MaxSize {
        return fmt.Errorf("archive too large: %d bytes (max %d)", info.Size(), v.MaxSize)
    }

    // 2. Detect format (tar.gz or zip only)
    format, err := detectArchiveFormat(archivePath)
    if err != nil {
        return fmt.Errorf("detect format: %w", err)
    }
    if format != "tar.gz" && format != "zip" {
        return fmt.Errorf("unsupported format: %s (expected tar.gz or zip)", format)
    }

    // 3. Validate contents
    files, err := listArchiveContents(archivePath, format)
    if err != nil {
        return fmt.Errorf("list contents: %w", err)
    }

    // 4. Check for required config files
    missing := []string{}
    for _, required := range v.RequiredFiles {
        found := false
        for _, file := range files {
            if strings.HasSuffix(file, required) {
                found = true
                break
            }
        }
        if !found {
            missing = append(missing, required)
        }
    }
    if len(missing) > 0 {
        return fmt.Errorf("missing required files: %v", missing)
    }

    return nil
}

// Format detector
func detectArchiveFormat(path string) (string, error) {
    f, err := os.Open(path)
    if err != nil {
        return "", err
    }
    defer f.Close()

    // Read magic bytes
    buf := make([]byte, 512)
    n, err := f.Read(buf)
    if err != nil && err != io.EOF {
        return "", err
    }

    // Check for gzip magic bytes (1f 8b)
    if n >= 2 && buf[0] == 0x1f && buf[1] == 0x8b {
        return "tar.gz", nil
    }

    // Check for zip magic bytes (50 4b)
    if n >= 4 && buf[0] == 0x50 && buf[1] == 0x4b {
        return "zip", nil
    }

    return "", fmt.Errorf("unknown format")
}

// Enhanced ImportConfig with validation
func (h *CrowdsecHandler) ImportConfig(c *gin.Context) {
    file, err := c.FormFile("file")
    if err != nil {
        c.JSON(http.StatusBadRequest, gin.H{
            "error": "file required",
            "details": "multipart form field 'file' is missing",
        })
        return
    }

    // Save to temp location
    tmpDir := os.TempDir()
    tmpPath := filepath.Join(tmpDir, fmt.Sprintf("crowdsec-import-%d", time.Now().UnixNano()))
    if err := os.MkdirAll(tmpPath, 0o750); err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{
            "error": "failed to create temp dir",
            "details": err.Error(),
        })
        return
    }
    defer os.RemoveAll(tmpPath)

    dst := filepath.Join(tmpPath, file.Filename)
    if err := c.SaveUploadedFile(file, dst); err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{
            "error": "failed to save upload",
            "details": err.Error(),
        })
        return
    }

    // ✨ NEW: Validate archive
    validator := &ConfigArchiveValidator{
        MaxSize: 50 * 1024 * 1024, // 50MB
        RequiredFiles: []string{"config.yaml"},
    }
    if err := validator.Validate(dst); err != nil {
        c.JSON(http.StatusUnprocessableEntity, gin.H{
            "error": "invalid config archive",
            "details": err.Error(),
        })
        return
    }

    // Create backup before import
    backupDir := h.DataDir + ".backup." + time.Now().Format("20060102-150405")
    if _, err := os.Stat(h.DataDir); err == nil {
        if err := os.Rename(h.DataDir, backupDir); err != nil {
            c.JSON(http.StatusInternalServerError, gin.H{
                "error": "failed to create backup",
                "details": err.Error(),
            })
            return
        }
    }

    // Extract archive
    if err := extractArchive(dst, h.DataDir); err != nil {
        // ✨ NEW: Restore backup on extraction failure
        if backupDir != "" {
            _ = os.RemoveAll(h.DataDir)
            _ = os.Rename(backupDir, h.DataDir)
        }
        c.JSON(http.StatusInternalServerError, gin.H{
            "error": "failed to extract archive",
            "details": err.Error(),
            "backup_restored": backupDir != "",
        })
        return
    }

    // ✨ NEW: Validate extracted config
    configPath := filepath.Join(h.DataDir, "config.yaml")
    if _, err := os.Stat(configPath); os.IsNotExist(err) {
        // Try subdirectory
        configPath = filepath.Join(h.DataDir, "config", "config.yaml")
        if _, err := os.Stat(configPath); os.IsNotExist(err) {
            // Rollback
            _ = os.RemoveAll(h.DataDir)
            _ = os.Rename(backupDir, h.DataDir)
            c.JSON(http.StatusUnprocessableEntity, gin.H{
                "error": "invalid config structure",
                "details": "config.yaml not found in expected locations",
                "backup_restored": true,
            })
            return
        }
    }

    // ✨ NEW: Validate YAML syntax
    if err := validateYAMLFile(configPath); err != nil {
        // Rollback
        _ = os.RemoveAll(h.DataDir)
        _ = os.Rename(backupDir, h.DataDir)
        c.JSON(http.StatusUnprocessableEntity, gin.H{
            "error": "invalid config syntax",
            "file": "config.yaml",
            "details": err.Error(),
            "backup_restored": true,
        })
        return
    }

    c.JSON(http.StatusOK, gin.H{
        "status": "imported",
        "backup": backupDir,
        "files_extracted": countFiles(h.DataDir),
        "reload_hint": true,
    })
}

func validateYAMLFile(path string) error {
    data, err := os.ReadFile(path)
    if err != nil {
        return err
    }

    var config map[string]interface{}
    if err := yaml.Unmarshal(data, &config); err != nil {
        return fmt.Errorf("YAML syntax error: %w", err)
    }

    // Basic structure validation
    if _, ok := config["api"]; !ok {
        return fmt.Errorf("missing required field: api")
    }

    return nil
}

func extractArchive(src, dst string) error {
    format, err := detectArchiveFormat(src)
    if err != nil {
        return err
    }

    if format == "tar.gz" {
        return extractTarGz(src, dst)
    }
    return extractZip(src, dst)
}

func extractTarGz(src, dst string) error {
    f, err := os.Open(src)
    if err != nil {
        return err
    }
    defer f.Close()

    gzr, err := gzip.NewReader(f)
    if err != nil {
        return err
    }
    defer gzr.Close()

    tr := tar.NewReader(gzr)
    for {
        header, err := tr.Next()
        if err == io.EOF {
            break
        }
        if err != nil {
            return err
        }

        target := filepath.Join(dst, header.Name)

        // Security: prevent path traversal
        if !strings.HasPrefix(target, filepath.Clean(dst)+string(os.PathSeparator)) {
            return fmt.Errorf("invalid file path: %s", header.Name)
        }

        switch header.Typeflag {
        case tar.TypeDir:
            if err := os.MkdirAll(target, 0750); err != nil {
                return err
            }
        case tar.TypeReg:
            os.MkdirAll(filepath.Dir(target), 0750)
            outFile, err := os.Create(target)
            if err != nil {
                return err
            }
            if _, err := io.Copy(outFile, tr); err != nil {
                outFile.Close()
                return err
            }
            outFile.Close()
        }
    }
    return nil
}

Unit Tests

File: backend/internal/api/handlers/crowdsec_handler_test.go (Add new tests)

func TestImportConfig_Validation(t *testing.T) {
    tests := []struct {
        name       string
        archive    func() io.Reader
        wantStatus int
        wantError  string
    }{
        {
            name: "valid archive",
            archive: func() io.Reader {
                return createTestArchive(map[string]string{
                    "config.yaml": "api:\n  server:\n    listen_uri: test",
                })
            },
            wantStatus: 200,
        },
        {
            name: "missing config.yaml",
            archive: func() io.Reader {
                return createTestArchive(map[string]string{
                    "acquis.yaml": "filenames:\n  - /var/log/test.log",
                })
            },
            wantStatus: 422,
            wantError:  "missing required files",
        },
        {
            name: "invalid YAML syntax",
            archive: func() io.Reader {
                return createTestArchive(map[string]string{
                    "config.yaml": "invalid: yaml: syntax: [[ unclosed",
                })
            },
            wantStatus: 422,
            wantError:  "invalid config syntax",
        },
        {
            name: "invalid format",
            archive: func() io.Reader {
                return strings.NewReader("not a valid archive")
            },
            wantStatus: 422,
            wantError:  "unsupported format",
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            // Setup
            dataDir := t.TempDir()
            handler := &CrowdsecHandler{DataDir: dataDir}
            router := gin.Default()
            router.POST("/admin/crowdsec/import", handler.ImportConfig)

            // Create multipart request
            body := &bytes.Buffer{}
            writer := multipart.NewWriter(body)
            part, _ := writer.CreateFormFile("file", "test.tar.gz")
            io.Copy(part, tt.archive())
            writer.Close()

            req := httptest.NewRequest("POST", "/admin/crowdsec/import", body)
            req.Header.Set("Content-Type", writer.FormDataContentType())
            w := httptest.NewRecorder()

            // Execute
            router.ServeHTTP(w, req)

            // Assert
            assert.Equal(t, tt.wantStatus, w.Code)

            if tt.wantError != "" {
                var resp map[string]interface{}
                json.Unmarshal(w.Body.Bytes(), &resp)
                assert.Contains(t, resp["error"], tt.wantError)
            }
        })
    }
}

E2E Tests

File: tests/security/crowdsec-import.spec.ts (Add new tests)

test.describe('CrowdSec Config Import - Validation', () => {
  test('should reject archive without config.yaml', async ({ request }) => {
    const mockArchive = createTarGz({
      'acquis.yaml': 'filenames:\n  - /var/log/test.log'
    });

    const formData = new FormData();
    formData.append('file', new Blob([mockArchive]), 'incomplete.tar.gz');

    const response = await request.post('/api/v1/admin/crowdsec/import', {
      data: formData,
    });

    expect(response.status()).toBe(422);
    const error = await response.json();
    expect(error.error).toContain('invalid config archive');
    expect(error.details).toContain('config.yaml');
  });

  test('should reject invalid YAML syntax', async ({ request }) => {
    const mockArchive = createTarGz({
      'config.yaml': 'invalid: yaml: syntax: [[ unclosed'
    });

    const formData = new FormData();
    formData.append('file', new Blob([mockArchive]), 'invalid.tar.gz');

    const response = await request.post('/api/v1/admin/crowdsec/import', {
      data: formData,
    });

    expect(response.status()).toBe(422);
    const error = await response.json();
    expect(error.error).toContain('invalid config syntax');
    expect(error.backup_restored).toBe(true);
  });

  test('should rollback on extraction failure', async ({ request }) => {
    const corruptedArchive = Buffer.from('not a valid archive');

    const formData = new FormData();
    formData.append('file', new Blob([corruptedArchive]), 'corrupt.tar.gz');

    const response = await request.post('/api/v1/admin/crowdsec/import', {
      data: formData,
    });

    expect(response.status()).toBe(500);
    const error = await response.json();
    expect(error.backup_restored).toBe(true);
  });
});

Acceptance Criteria:

Archive format validated (tar.gz, zip only)
File size limits enforced (50MB max)
Required file presence checked (config.yaml)
YAML syntax validation
Automatic rollback on validation failures
Backup created before every import
Path traversal attacks blocked during extraction
E2E tests for all error scenarios
Unit test coverage ≥ 85%

Implementation Plan

Sprint 0: Critical Security Fix (P0 - Day 0, 2 hours)

🚨 BLOCKER: This MUST be completed before ANY other work, including Supervisor Review

Task 0.1: Implement Secure API Key Masking

Assignee: TBD Priority: P0 (CRITICAL BLOCKER) Files: backend/internal/api/handlers/crowdsec_handler.go Estimated Time: 1 hour

Steps:

Add maskAPIKey() utility function (after line 2057)
Add validateAPIKeyFormat() validation function
Update logBouncerKeyBanner() to use masked keys (line 1366-1378)
Add security warning message to banner
Document security rationale in comments

Validation:

# Build and verify function exists
cd backend
go build ./...

# Expected: Build succeeds with no errors

Task 0.2: Add Unit Tests for Security Fix

Assignee: TBD Priority: P0 (CRITICAL BLOCKER) Files: backend/internal/api/handlers/crowdsec_handler_test.go Estimated Time: 30 minutes

Steps:

Add TestMaskAPIKey() with multiple test cases
Add TestValidateAPIKeyFormat() for validation logic
Add TestLogBouncerKeyBanner_NoSecretExposure() integration test
Ensure 100% coverage of new security functions

Validation:

# Run security-focused unit tests
go test ./backend/internal/api/handlers -run TestMaskAPIKey -v
go test ./backend/internal/api/handlers -run TestLogBouncerKeyBanner -v

# Check coverage
go test ./backend/internal/api/handlers -coverprofile=coverage.out
go tool cover -func=coverage.out | grep -E "(maskAPIKey|validateAPIKeyFormat|logBouncerKeyBanner)"

# Expected: 100% coverage for security functions

Task 0.3: Manual Security Validation

Assignee: TBD Priority: P0 (CRITICAL BLOCKER) Estimated Time: 20 minutes

Steps:

Start backend with test API key
Trigger bouncer registration (POST /admin/crowdsec/start)
Verify logs contain masked key, not full key
Check no API keys in HTTP responses
Verify file permissions on saved key file (should be 0600)

Validation:

# Start backend with test key
cd backend
CROWDSEC_API_KEY="test-secret-key-123456789" go run cmd/main.go &

# Trigger registration
curl -X POST http://localhost:8080/api/v1/admin/crowdsec/start

# Check logs (should show masked key)
grep "API Key:" /var/log/charon/app.log | grep -v "test-secret-key-123456789"
# Expected: Log lines found with masked key "test...6789"

grep "API Key:" /var/log/charon/app.log | grep "test-secret-key-123456789"
# Expected: No matches (full key not logged)

Task 0.4: CodeQL Security Re-scan

Assignee: TBD Priority: P0 (CRITICAL BLOCKER) Estimated Time: 10 minutes

Steps:

Run CodeQL scan on modified code
Verify CWE-312/315/359 findings are resolved
Check no new security issues introduced
Document resolution in scan results

Validation:

# Run CodeQL scan (CI-aligned)
.github/skills/scripts/skill-runner.sh security-codeql-scan

# Expected: 0 critical/high findings for CWE-312/315/359 in crowdsec_handler.go:1378

Sprint 0 Completion Criteria:

Security functions implemented and tested
Unit tests achieve 100% coverage of security code
Manual validation confirms no key exposure
CodeQL scan shows 0 CWE-312/315/359 findings
All existing tests still pass
BLOCKER REMOVED: Ready for Supervisor Review

Sprint 1: Test Bug Fix (Day 1, 30 min)

Task 1.1: Fix E2E Test Endpoint

Assignee: Playwright_Dev Priority: P1 (Blocked by Sprint 0) Depends On: Sprint 0 must complete and pass CodeQL scan Files: tests/security/crowdsec-diagnostics.spec.ts

Steps:

Open test file (line 323)
Change /files?path=... to /file?path=...
Run test locally to verify
Commit and push

Validation:

# Rebuild E2E environment
.github/skills/scripts/skill-runner.sh docker-rebuild-e2e

# Run specific test
npx playwright test tests/security/crowdsec-diagnostics.spec.ts --project=chromium

# Expected: Test passes

Estimated Time: 30 minutes (includes validation)

Sprint 2: Import Validation (Days 2-3, 4-5 hours)

Task 2.1: Implement ConfigArchiveValidator

Assignee: TBD Priority: P1 Files: backend/internal/api/handlers/crowdsec_handler.go Estimated Time: 2 hours

Steps:

Add ConfigArchiveValidator struct (line ~2060)
Implement Validate() method
Implement detectArchiveFormat() helper
Implement listArchiveContents() helper
Write unit tests for validator

Validation:

go test ./backend/internal/api/handlers -run TestConfigArchiveValidator -v

Task 2.2: Enhance ImportConfig Handler

Assignee: TBD Priority: P1 Files: backend/internal/api/handlers/crowdsec_handler.go Estimated Time: 2 hours

Steps:

Add pre-import validation call
Implement rollback logic on errors
Add YAML syntax validation
Update error responses
Write unit tests

Validation:

go test ./backend/internal/api/handlers -run TestImportConfig -v

Task 2.3: Add E2E Tests

Assignee: TBD Priority: P1 Files: tests/security/crowdsec-import.spec.ts Estimated Time: 1 hour

Steps:

✨ NEW: Implement createTarGz() helper in tests/utils/archive-helpers.ts
Write 5 validation test cases (added zip bomb test)
Verify rollback behavior
Check error message format
Test compression ratio rejection

Validation:

npx playwright test tests/security/crowdsec-import.spec.ts --project=chromium

Testing Strategy

Unit Test Coverage Goals

Component	Target Coverage	Critical Paths
`ConfigArchiveValidator`	90%	Format detection, size check, content validation
`ImportConfig` enhanced	85%	Validation flow, rollback logic, error handling

E2E Test Scenarios

Test	Description	Expected Result
Valid archive	Import with config.yaml	200 OK, files extracted
Missing config.yaml	Import without required file	422 Unprocessable Entity
Invalid YAML	Import with syntax errors	422, backup restored
Oversized archive	Import >50MB file	413 Payload Too Large
Wrong format	Import .txt file	422 Unsupported format
Corrupted archive	Import malformed tar.gz	500, backup restored

Coverage Validation

# Backend coverage
go test ./backend/internal/api/handlers -coverprofile=coverage.out
go tool cover -func=coverage.out | grep crowdsec_handler.go

# E2E coverage
.github/skills/scripts/skill-runner.sh test-e2e-playwright-coverage

# Check Codecov patch coverage (must be 100%)
# CI workflow will enforce this

Success Criteria

Definition of Done

Sprint 0 (BLOCKER):

API key masking utility implemented
Security unit tests pass with 100% coverage
Manual validation confirms no key exposure in logs
CodeQL scan resolves CWE-312/315/359 findings
Documentation updated with security guidelines
All existing tests still pass

Sprint 1:

Issue 1 test fix deployed and passing
Issue 2 confirmed as already working
E2E test should retrieve specific config file content passes

Sprint 2:

Issue 3 validation implemented and tested
Import validation prevents malformed configs
Rollback mechanism tested and verified

Overall:

Backend coverage ≥ 85% for modified handlers
E2E coverage ≥ 85% for affected test files
All E2E tests pass on Chromium, Firefox, WebKit
No security vulnerabilities (CodeQL clean)
Pre-commit hooks pass
Code review completed

Acceptance Tests

# Test 0: Security fix validation (Sprint 0)
go test ./backend/internal/api/handlers -run TestMaskAPIKey -v
go test ./backend/internal/api/handlers -run TestLogBouncerKeyBanner -v
.github/skills/scripts/skill-runner.sh security-codeql-scan
# Expected: Tests pass, CodeQL shows 0 CWE-312/315/359 findings

# Test 1: Config file retrieval (Sprint 1)
npx playwright test tests/security/crowdsec-diagnostics.spec.ts --project=chromium
# Expected: Test passes with correct endpoint

# Test 2: Import validation (Sprint 2)
npx playwright test tests/security/crowdsec-import.spec.ts --project=chromium
# Expected: All validation tests pass

# Test 3: Backend unit tests
go test ./backend/internal/api/handlers -run TestImportConfig -v
go test ./backend/internal/api/handlers -run TestConfigArchiveValidator -v
# Expected: All tests pass

# Test 4: Coverage check
go test ./backend/internal/api/handlers -coverprofile=coverage.out
go tool cover -func=coverage.out | grep total | awk '{print $3}'
# Expected: ≥85%

# Test 5: Manual verification
curl -X POST http://localhost:8080/api/v1/admin/crowdsec/import \
  -F "file=@invalid-archive.tar.gz"
# Expected: 422 with validation error

Risks & Mitigation

Risk	Probability	Impact	Mitigation
Security fix breaks existing functionality	LOW	HIGH	Run full test suite after Sprint 0
Masked keys insufficient for debugging	MEDIUM	LOW	Document key retrieval from file
Test fix breaks other tests	LOW	MEDIUM	Run full E2E suite before merge
Import validation too strict	MEDIUM	MEDIUM	Allow optional files (acquis.yaml)
YAML parsing vulnerabilities	LOW	HIGH	Use well-tested yaml library, limit file size
Rollback failures	LOW	HIGH	Extensive testing of rollback logic

File Inventory

Files to Modify

Path	Changes	Lines Added	Impact
`backend/internal/api/handlers/crowdsec_handler.go`	Security fix + validation	+200	CRITICAL
`backend/internal/api/handlers/crowdsec_handler_test.go`	Security + validation tests	+150	CRITICAL
`tests/security/crowdsec-diagnostics.spec.ts`	Fix endpoint (line 323)	1	HIGH
`tests/security/crowdsec-import.spec.ts`	Add E2E tests	+100	MEDIUM
`README.md`	Security notes	+20	MEDIUM

Files to Create

Path	Purpose	Lines	Priority
`docs/security/api-key-handling.md`	Security guidelines	~80	CRITICAL
`docs/SECURITY_PRACTICES.md`	Best practices + compliance	~120	CRITICAL
`tests/utils/archive-helpers.ts`	Test helper functions	~50	MEDIUM

Total Effort Estimate

Phase	Hours	Confidence	Priority
Sprint 0: Security Fix	2	Very High	P0 (BLOCKER)
Sprint 1: Test Bug Fix	0.5	Very High	P1
Sprint 2: Import Validation	4-5	High	P2
Sprint 3: API Key UX	2-3	High	P2
Testing & QA	1	High	-
Code Review	0.5	High	-
Total	10-12 hours	High	-

CRITICAL PATH: Sprint 0 MUST complete before Sprint 1 can begin. Sprints 2 and 3 can run in parallel. Sprints 2 and 3 can run in parallel.

Sprint 3: Move CrowdSec API Key to Config Page (Issue 4)

Overview

Current State: CrowdSec API key is displayed on the main Security Dashboard Desired State: API key should be on the CrowdSec-specific configuration page Rationale: Better UX - security settings should be scoped to their respective feature pages

Duration: 2-3 hours Priority: P2 (UX improvement, not blocking) Complexity: MEDIUM (API endpoint changes likely) Depends On: Sprint 0 (uses masked API key)

Current Architecture (To Be Researched)

Frontend Components (Likely):

Security Dashboard: /frontend/src/pages/Security.tsx or similar
CrowdSec Config Page: /frontend/src/pages/CrowdSec.tsx or similar

Backend API Endpoints (To Be Verified):

Currently: API key retrieved via general security endpoint?
Proposed: Move to CrowdSec-specific endpoint or enhance existing endpoint

Research Required:

Identify current component displaying API key
Identify target CrowdSec config page component
Determine if API endpoint changes are needed
Check if frontend state management needs updates

Implementation Tasks

Task 3.1: Research Current Implementation (30 min)

Assignee: Frontend_Dev Priority: P2

Locate Security Dashboard component displaying API key
Locate CrowdSec configuration page component
Identify API endpoint(s) returning the API key
Check if API key is part of a larger security settings response
Verify frontend routing and navigation structure
Document current data flow

Search Patterns:

# Find components
grep -r "apiKey\|api_key\|bouncerKey" frontend/src/pages/
grep -r "CrowdSec" frontend/src/pages/
grep -r "Security" frontend/src/pages/

# Find API calls
grep -r "crowdsec.*api.*key" frontend/src/api/

Task 3.2: Update Backend API (if needed) (30 min)

Assignee: Backend_Dev Priority: P2 Files: TBD based on research

Possible Scenarios:

Scenario A: No API changes needed

API key already available via /admin/crowdsec endpoints
Frontend just needs to move the component

Scenario B: Add new endpoint

// Add to backend/internal/api/handlers/crowdsec_handler.go
func (h *CrowdsecHandler) GetAPIKey(c *gin.Context) {
    key, err := h.getBouncerAPIKeyFromEnv()
    if err != nil {
        c.JSON(http.StatusNotFound, gin.H{"error": "API key not found"})
        return
    }

    // Use masked key from Sprint 0
    maskedKey := maskAPIKey(key)

    c.JSON(http.StatusOK, gin.H{
        "apiKey": maskedKey,
        "masked": true,
        "hint": "Full key stored in keyfile",
    })
}

Scenario C: Enhance existing endpoint

Add apiKey field to existing CrowdSec status/config response

Task 3.3: Move Frontend Component (45 min)

Assignee: Frontend_Dev Priority: P2 Files: TBD based on research

Steps:

Remove API key display from Security Dashboard
Add API key display to CrowdSec Config Page
Update API calls to use correct endpoint
Add loading/error states
Add copy-to-clipboard functionality (if not present)
Add security warning about key sensitivity

Example Component (pseudocode):

// In CrowdSec Config Page
const CrowdSecAPIKeySection = () => {
  const { data, isLoading, error } = useCrowdSecAPIKey();

  return (
    <Section>
      <Heading>Bouncer API Key</Heading>
      <Alert variant="warning">
        🔐 This API key is sensitive. Never share it publicly.
      </Alert>
      {isLoading && <Spinner />}
      {error && <ErrorMessage>{error}</ErrorMessage>}
      {data && (
        <>
          <Code>{data.apiKey}</Code>
          <CopyButton value={data.apiKey} />
          {data.masked && (
            <Text muted>Full key stored in: {data.keyfile}</Text>
          )}
        </>
      )}
    </Section>
  );
};

Task 3.4: Update E2E Tests (30 min)

Assignee: Playwright_Dev Priority: P2 Files: tests/security/crowdsec-*.spec.ts

Changes:

Update test that verifies API key display location
Add navigation test (Security → CrowdSec Config)
Verify API key is NOT on Security Dashboard
Verify API key IS on CrowdSec Config Page
Test copy-to-clipboard functionality

test('CrowdSec API key displayed on config page', async ({ page }) => {
  await page.goto('/crowdsec/config');

  // Verify API key section exists
  await expect(page.getByRole('heading', { name: /bouncer api key/i })).toBeVisible();

  // Verify masked key is displayed
  const keyElement = page.getByTestId('crowdsec-api-key');
  await expect(keyElement).toBeVisible();
  const keyText = await keyElement.textContent();
  expect(keyText).toMatch(/^[a-zA-Z0-9]{4}\.\.\.{4}$/);

  // Verify security warning
  await expect(page.getByText(/never share it publicly/i)).toBeVisible();
});

test('API key NOT on security dashboard', async ({ page }) => {
  await page.goto('/security');

  // Verify API key section does NOT exist
  await expect(page.getByTestId('crowdsec-api-key')).not.toBeVisible();
});

Task 3.5: Update Documentation (15 min)

Assignee: Docs_Writer Priority: P2 Files: README.md, feature docs

Changes:

Update screenshots showing CrowdSec configuration
Update user guide referencing API key location
Add note about masked display (from Sprint 0)

Sprint 3 Acceptance Criteria

API key removed from Security Dashboard
API key displayed on CrowdSec Config Page
API key uses masked format from Sprint 0
Copy-to-clipboard functionality works
Security warning displayed
E2E tests pass (API key on correct page)
No regression: existing CrowdSec features still work
Documentation updated
Code review completed

Validation Commands:

# Test CrowdSec config page
npx playwright test tests/security/crowdsec-config.spec.ts --project=chromium

# Verify no API key on security dashboard
npx playwright test tests/security/security-dashboard.spec.ts --project=chromium

# Full regression test
npx playwright test tests/security/ --project=chromium

Sprint 3 Risk Assessment

Risk	Probability	Impact	Mitigation
API endpoint changes break existing features	LOW	MEDIUM	Full E2E test suite
Routing changes affect navigation	LOW	LOW	Test all nav paths
State management issues	MEDIUM	MEDIUM	Thorough testing of data flow
User confusion about changed location	LOW	LOW	Update docs + changelog

Future Enhancements (Out of Scope)

Real-time file watching for config changes
Diff view for config file history
Config file validation against CrowdSec schema
Bulk file operations (upload/download multiple)
WebSocket-based live config editing
Config version control integration (Git)
Import from CrowdSec Hub URLs
Export to CrowdSec console format

References

🚨 CodeQL Security Alert: CWE-312/315/359 - Cleartext Storage of Sensitive Information
Vulnerable Code: backend/internal/api/handlers/crowdsec_handler.go:1378
OWASP A02:2021: Cryptographic Failures - https://owasp.org/Top10/A02_2021-Cryptographic_Failures/
CWE-312: Cleartext Storage of Sensitive Information - https://cwe.mitre.org/data/definitions/312.html
CWE-315: Cleartext Storage of Sensitive Data in a Cookie - https://cwe.mitre.org/data/definitions/315.html
CWE-359: Exposure of Private Personal Information to an Unauthorized Actor - https://cwe.mitre.org/data/definitions/359.html
QA Report: docs/reports/qa_report.md
Current Handler: backend/internal/api/handlers/crowdsec_handler.go
Frontend API: frontend/src/api/crowdsec.ts
Failing E2E Test: tests/security/crowdsec-diagnostics.spec.ts
OWASP Path Traversal: https://owasp.org/www-community/attacks/Path_Traversal
Go filepath Security: https://pkg.go.dev/path/filepath#Clean

Plan Status: ⚠️ CONDITIONAL APPROVAL - CRITICAL BLOCKERS (Reviewed 2026-02-03) Next Steps:

IMMEDIATE (P0): Complete pre-implementation security audit (audit logs, rotate keys)
Sprint 0 (2 hours): Fix API key logging vulnerability with enhancements
Validation: CodeQL must show 0 CWE findings before Sprint 1
Sprint 1: Test bug fix (blocked by Sprint 0)
Sprint 2: Import validation with zip bomb protection (blocked by Sprint 0)

🔍 SUPERVISOR REVIEW

Reviewer: GitHub Copilot (Supervisor Mode) Review Date: 2026-02-03 Review Status: ⚠️ CONDITIONAL APPROVAL - CRITICAL BLOCKERS IDENTIFIED

Executive Summary

This plan correctly identifies a CRITICAL P0 security vulnerability (CWE-312/315/359) that must be addressed immediately. The proposed solution is sound and follows industry best practices, but several critical gaps and enhancements are required before implementation can proceed.

Overall Assessment:

✅ Security fix approach is correct and aligned with OWASP guidelines
✅ Architecture review confirms most findings are accurate (test bugs, not API bugs)
⚠️ Implementation quality needs strengthening (missing test coverage, incomplete security audit)
⚠️ Risk analysis incomplete - critical security risks not fully addressed
✅ Compliance considerations are adequate but need documentation

Recommendation: APPROVE WITH MANDATORY CHANGES - Sprint 0 must incorporate the additional requirements below before implementation.

🚨 CRITICAL GAPS IDENTIFIED

1. ❌ BLOCKER: Missing Log Output Capture in Tests

Issue: TestLogBouncerKeyBanner_NoSecretExposure() attempts to capture logs but will fail:

Uses incorrect logger API (logger.SetOutput() may not exist)
Charon uses custom logger (logger.Log() from internal/logger)
Test will fail to compile or not capture output

Required Fix: Use test logger hook, file parsing, or testing logger with buffer.

Priority: P0 - Must fix before Sprint 0 implementation

2. ❌ BLOCKER: Incomplete Security Audit

Issue: Plan only audited logBouncerKeyBanner(). Other functions handle API keys:

ensureBouncerRegistration() (line 1280-1362) - Returns apiKey
getBouncerAPIKeyFromEnv() (line 1381-1393) - Retrieves keys
saveKeyToFile() (line 1419-1432) - Writes keys to disk

Required Actions: Document audit of all API key handling functions, verify error messages don't leak keys.

Priority: P0 - Document audit results in Sprint 0

3. ⚠️ HIGH: Missing File Permission Verification

Issue: Plan mentions file permissions (0600) but doesn't verify in tests.

Required Test:

func TestSaveKeyToFile_SecurePermissions(t *testing.T) {
    // Verify file permissions are 0600 (rw-------)
    info, err := os.Stat(keyFile)
    require.Equal(t, os.FileMode(0600), info.Mode().Perm())
}

Priority: P1 - Add to Sprint 0 acceptance criteria

4. 🚨 CRITICAL RISKS NOT IN ORIGINAL PLAN

Risk 1: Production Log Exposure Risk

Probability: HIGH
Impact: CRITICAL
Mitigation Required:
1. Audit existing production logs for exposed API keys
2. Rotate any potentially compromised keys
3. Purge or redact historical logs containing keys
4. Implement log retention policy (7-30 days max)
5. Notify security team if keys found

Priority: P0 - MUST COMPLETE BEFORE SPRINT 0 IMPLEMENTATION

Risk 2: Third-Party Log Aggregation Risk

Probability: MEDIUM
Impact: CRITICAL
Mitigation Required:
1. Identify all log destinations (CloudWatch, Splunk, Datadog)
2. Check if API keys are searchable in log aggregation tools
3. Request deletion of sensitive logs from external services
4. Rotate API keys if found in external logs

Priority: P0 - MUST COMPLETE BEFORE SPRINT 0 IMPLEMENTATION

Risk 3: Zip Bomb Protection Missing

Issue: Import validation only checks compressed size
Risk: 10MB compressed → 10GB uncompressed attack possible
Required: Add compression ratio check (max 100x)
Priority: P1 - Add to Sprint 2

Risk 4: Test Helpers Missing

Issue: createTestArchive() and createTarGz() referenced but not defined
Priority: P0 - Must implement before Sprint 2

📊 Updated Risk Matrix

Risk	Probability	Impact	Mitigation	Priority	Status
Existing logs contain keys	HIGH	CRITICAL	Audit + rotate + purge	P0	BLOCKER
External log services	MEDIUM	CRITICAL	Check + delete + rotate	P0	BLOCKER
Security fix breaks tests	LOW	HIGH	Full test suite	P0	Planned ✅
Masked keys insufficient	MEDIUM	LOW	Document retrieval	P1	Planned ✅
Backups expose keys	MEDIUM	HIGH	Encrypt + access control	P1	NEW
CodeQL false negatives	LOW	HIGH	Additional scanners	P1	NEW
Zip bomb attack	MEDIUM	HIGH	Compression ratio check	P1	NEW
Test helpers missing	HIGH	MEDIUM	Implement before Sprint 2	P0	NEW

✅ CONDITIONAL APPROVAL

Status: APPROVED WITH MANDATORY CHANGES

Conditions for Implementation:

PRE-IMPLEMENTATION (IMMEDIATE - BLOCKER):
- Audit existing production logs for exposed API keys
- Check external log services (CloudWatch, Splunk, Datadog)
- Rotate any compromised keys found
- Purge sensitive historical logs
SPRINT 0 ENHANCEMENTS (P0):
- Fix test logger capture implementation
- Add file permission verification test
- Complete security audit documentation
- Create docs/SECURITY_PRACTICES.md with compliance mapping
- Run additional secret scanners (Semgrep, TruffleHog)
SPRINT 2 ENHANCEMENTS (P1):
- Add zip bomb protection (compression ratio check)
- Implement test helper functions (createTestArchive, createTarGz)
- Enhanced YAML structure validation
DOCUMENTATION (P1):
- Add compliance section to SECURITY_PRACTICES.md
- Document audit procedures and findings
- Update risk matrix with new findings

🎯 Enhanced Sprint 0 Checklist

Pre-Implementation (CRITICAL - Before ANY code changes):

Audit existing production logs for exposed API keys
Check external log aggregation services
Scan git history with TruffleHog
Rotate any compromised keys found
Purge historical logs with exposed keys
Set up log retention policy
Notify security team if keys were exposed

Implementation Phase:

Implement maskAPIKey() utility (as planned ✅)
Implement validateAPIKeyFormat() (as planned ✅)
Update logBouncerKeyBanner() (as planned ✅)
NEW: Fix test logger capture implementation
NEW: Add file permission verification test
NEW: Complete security audit documentation
NEW: Run Semgrep and TruffleHog scanners
Write unit tests (100% coverage target)

Documentation Phase:

Update README.md (as planned ✅)
Create docs/security/api-key-handling.md (as planned ✅)
NEW: Create docs/SECURITY_PRACTICES.md
NEW: Add GDPR/PCI-DSS/SOC 2 compliance documentation
Document audit results

Validation Phase:

All unit tests pass
Manual validation confirms masking
CodeQL scan shows 0 CWE-312/315/359 findings
NEW: Semgrep shows no secret exposure
NEW: TruffleHog shows no keys in git history
NEW: File permissions verified (0600)
All existing tests still pass

📋 Updated Timeline

Original Estimate: 6.5-7.5 hours (with Sprint 0) Revised Estimate: 8.5-9.5 hours (includes enhancements)

Breakdown:

Pre-Implementation Audit: +1 hour (CRITICAL)
Sprint 0 Implementation: 2 hours (as planned)
Sprint 0 Enhancements: +30 min (test fixes, additional scans)
Sprint 1: 30 min (unchanged)
Sprint 2: +30 min (zip bomb protection)

🚦 Recommended Execution Order

IMMEDIATE (TODAY): Pre-implementation security audit (1 hour)
Sprint 0 (2 hours): Security fix with enhancements
Sprint 1 (30 min): Test bug fix
Sprint 2 (5-6 hours): Import validation with zip bomb protection

Total Revised Effort: 8.5-9.5 hours

📝 Supervisor Sign-Off

Reviewed By: GitHub Copilot (Supervisor Mode) Approval Status: ⚠️ CONDITIONAL APPROVAL Blockers: 5 critical issues identified Next Step: Address pre-implementation security audit before Sprint 0

Supervisor Recommendation:

This plan demonstrates strong understanding of the security vulnerability and proposes a sound technical solution. However, the immediate risk of existing exposed keys in production logs must be addressed before implementing the fix.

The proposed maskAPIKey() implementation is secure and follows industry best practices. The additional requirements identified in this review will strengthen the implementation and ensure comprehensive security coverage.

APPROVE for implementation once pre-implementation security audit is complete and Sprint 0 blockers are addressed.

Review Complete: 2026-02-03 Next Review: After Sprint 0 completion (CodeQL re-scan results)

60 KiB Raw Blame History

CrowdSec API Enhancement Plan

🚨 CRITICAL SECURITY ALERT 🚨

Executive Summary

Key Findings

Revised Implementation Scope

Impact Assessment

Research Findings

Current Implementation Analysis

Backend Files

API Endpoint Architecture (Already Correct)

Handler Implementation (Already Correct)

Frontend Integration (Already Correct)

Test Failure Root Cause

Proposed Solution

Sprint 0: Critical Security Fix (P0 - BLOCK ALL OTHER WORK)

Security Vulnerability Details

Secure Fix Implementation

Unit Tests

Security Validation

Acceptance Criteria

Documentation Updates

Key Storage

Compliance

Phase 1: Test Bug Fix (Issue 1 & 2)

E2E Test Fix

Sprint 2: Import Validation Enhancement (Issue 3)

Enhanced Validation Architecture

Validation Strategy

Implementation Details

Unit Tests

E2E Tests

Implementation Plan

Sprint 0: Critical Security Fix (P0 - Day 0, 2 hours)

Task 0.1: Implement Secure API Key Masking

Task 0.2: Add Unit Tests for Security Fix

Task 0.3: Manual Security Validation

Task 0.4: CodeQL Security Re-scan

Sprint 1: Test Bug Fix (Day 1, 30 min)

Task 1.1: Fix E2E Test Endpoint

Sprint 2: Import Validation (Days 2-3, 4-5 hours)

Task 2.1: Implement ConfigArchiveValidator

Task 2.2: Enhance ImportConfig Handler

Task 2.3: Add E2E Tests

Testing Strategy

Unit Test Coverage Goals

E2E Test Scenarios

Coverage Validation

Success Criteria

Definition of Done

Acceptance Tests

Risks & Mitigation

File Inventory

Files to Modify

Files to Create

Total Effort Estimate

Sprint 3: Move CrowdSec API Key to Config Page (Issue 4)

Overview

Current Architecture (To Be Researched)

Implementation Tasks

Task 3.1: Research Current Implementation (30 min)

Task 3.2: Update Backend API (if needed) (30 min)

Task 3.3: Move Frontend Component (45 min)

Task 3.4: Update E2E Tests (30 min)

Task 3.5: Update Documentation (15 min)

Sprint 3 Acceptance Criteria

Sprint 3 Risk Assessment

Future Enhancements (Out of Scope)

References

🔍 SUPERVISOR REVIEW

Executive Summary

🚨 CRITICAL GAPS IDENTIFIED

1. ❌ BLOCKER: Missing Log Output Capture in Tests

2. ❌ BLOCKER: Incomplete Security Audit

3. ⚠️ HIGH: Missing File Permission Verification

4. 🚨 CRITICAL RISKS NOT IN ORIGINAL PLAN

📊 Updated Risk Matrix

✅ CONDITIONAL APPROVAL

🎯 Enhanced Sprint 0 Checklist

📋 Updated Timeline

60 KiB

Raw Blame History