Files

GitHub Actions 2a1652d0b1 feat: add IP whitelist management details to architecture documentation

2026-04-16 13:34:36 +00:00

35 KiB

Raw Blame History

CrowdSec IP Whitelist Management — Implementation Plan

Issue: #939 — CrowdSec IP Whitelist Management Date: 2026-05-20 Status: Draft — Awaiting Approval Priority: High Archived Previous Plan: Coverage Improvement Plan (patch coverage ≥ 90%) → docs/plans/archive/patch-coverage-improvement-plan-2026-05-02.md

1. Introduction

1.1 Overview

CrowdSec enforces IP ban decisions by default. Operators need a way to permanently exempt known-good IPs (uptime monitors, internal subnets, VPN exits, partners) from ever being banned. CrowdSec handles this through its whitelists parser, which intercepts alert evaluation and suppresses bans for matching IPs/CIDRs before decisions are even written.

This feature gives Charon operators a first-class UI for managing those whitelist entries: add an IP or CIDR, give it a reason, and have Charon persist it in the database, render the required YAML parser file into the CrowdSec config tree, and signal CrowdSec to reload—all without manual file editing.

1.2 Objectives

Allow operators to add, view, and remove CrowdSec whitelist entries (IPs and CIDRs) through the Charon management UI.
Persist entries in SQLite so they survive container restarts.
Generate a crowdsecurity/whitelists-compatible YAML parser file on every mutating operation and on startup.
Automatically install the crowdsecurity/whitelists hub parser so CrowdSec can process the file.
Show the Whitelist tab only when CrowdSec is in local mode, consistent with other CrowdSec-only tabs.

2. Research Findings

2.1 Existing CrowdSec Architecture

Component	Location	Notes
Hub parser installer	`configs/crowdsec/install_hub_items.sh`	Run at container start; uses `cscli parsers install --force`
CrowdSec handler	`backend/internal/api/handlers/crowdsec_handler.go`	~2750 LOC; `RegisterRoutes` at L2704
Route registration	`backend/internal/api/routes/routes.go`	`crowdsecHandler.RegisterRoutes(management)` at ~L620
CrowdSec startup	`backend/internal/services/crowdsec_startup.go`	`ReconcileCrowdSecOnStartup()` runs before process start
Security config	`backend/internal/models/security_config.go`	`CrowdSecMode`, `CrowdSecConfigDir` (via `cfg.Security.CrowdSecConfigDir`)
IP/CIDR helper	`backend/internal/security/whitelist.go`	`IsIPInCIDRList()` using `net.ParseIP` / `net.ParseCIDR`
AutoMigrate	`routes.go` ~L95–125	`&models.ManualChallenge{}` is currently the last entry

2.2 Gap Analysis

crowdsecurity/whitelists hub parser is not installed by install_hub_items.sh — the YAML file would be ignored by CrowdSec without it.
No CrowdSecWhitelist model exists in backend/internal/models/.
No whitelist service, handler methods, or API routes exist.
No frontend tab, API client functions, or TanStack Query hooks exist.
No E2E test spec covers whitelist management.

2.3 Relevant Patterns

Model pattern (from access_list.go + security_config.go):

type Model struct {
    ID        uint      `json:"-" gorm:"primaryKey"`
    UUID      string    `json:"uuid" gorm:"uniqueIndex;not null"`
    // domain fields
    CreatedAt time.Time `json:"created_at"`
    UpdatedAt time.Time `json:"updated_at"`
}

Service pattern (from access_list_service.go):

var ErrXxxNotFound = errors.New("xxx not found")

type XxxService struct { db *gorm.DB }

func NewXxxService(db *gorm.DB) *XxxService { return &XxxService{db: db} }

Handler error response pattern (from crowdsec_handler.go):

c.JSON(http.StatusBadRequest, gin.H{"error": "..."})
c.JSON(http.StatusNotFound, gin.H{"error": "..."})
c.JSON(http.StatusInternalServerError, gin.H{"error": "..."})

Frontend API client pattern (from frontend/src/api/crowdsec.ts):

export const listXxx = async (): Promise<XxxEntry[]> => {
  const resp = await client.get<XxxEntry[]>('/admin/crowdsec/xxx')
  return resp.data
}

Frontend mutation pattern (from CrowdSecConfig.tsx):

const mutation = useMutation({
  mutationFn: (data) => apiCall(data),
  onSuccess: () => {
    toast.success('...')
    queryClient.invalidateQueries({ queryKey: ['crowdsec-whitelist'] })
  },
  onError: (err) => toast.error(err instanceof Error ? err.message : '...'),
})

2.4 CrowdSec Whitelist YAML Format

CrowdSec's crowdsecurity/whitelists parser expects the following YAML structure at a path under the parsers/s02-enrich/ directory:

name: charon-whitelist
description: "Charon-managed IP/CIDR whitelist"
filter: "evt.Meta.service == 'http'"
whitelist:
  reason: "Charon managed whitelist"
  ip:
    - "1.2.3.4"
  cidr:
    - "10.0.0.0/8"
    - "192.168.0.0/16"

For an empty whitelist, both ip and cidr must be present as empty lists (not omitted) to produce valid YAML that CrowdSec can parse without error.

3. Technical Specifications

3.1 Database Schema

New model: backend/internal/models/crowdsec_whitelist.go

package models

import "time"

// CrowdSecWhitelist represents a single IP or CIDR exempted from CrowdSec banning.
type CrowdSecWhitelist struct {
    ID        uint      `json:"-"          gorm:"primaryKey"`
    UUID      string    `json:"uuid"       gorm:"uniqueIndex;not null"`
    IPOrCIDR  string    `json:"ip_or_cidr" gorm:"not null;uniqueIndex"`
    Reason    string    `json:"reason"     gorm:"not null;default:''"`
    CreatedAt time.Time `json:"created_at"`
    UpdatedAt time.Time `json:"updated_at"`
}

AutoMigrate registration (backend/internal/api/routes/routes.go, append after &models.ManualChallenge{}):

&models.CrowdSecWhitelist{},

3.2 API Design

All new endpoints live under the existing /api/v1 prefix and are registered inside CrowdsecHandler.RegisterRoutes(rg *gin.RouterGroup), following the same rg.METHOD("/admin/crowdsec/...") naming pattern as every other CrowdSec endpoint.

Endpoint Table

Method	Path	Auth	Description
`GET`	`/api/v1/admin/crowdsec/whitelist`	Management	List all whitelist entries
`POST`	`/api/v1/admin/crowdsec/whitelist`	Management	Add a new entry
`DELETE`	`/api/v1/admin/crowdsec/whitelist/:uuid`	Management	Remove an entry by UUID

`GET /admin/crowdsec/whitelist`

Response 200:

{
  "whitelist": [
    {
      "uuid": "a1b2c3d4-...",
      "ip_or_cidr": "10.0.0.0/8",
      "reason": "Internal subnet",
      "created_at": "2026-05-20T12:00:00Z",
      "updated_at": "2026-05-20T12:00:00Z"
    }
  ]
}

`POST /admin/crowdsec/whitelist`

Request body:

{ "ip_or_cidr": "10.0.0.0/8", "reason": "Internal subnet" }

Response 201:

{
  "uuid": "a1b2c3d4-...",
  "ip_or_cidr": "10.0.0.0/8",
  "reason": "Internal subnet",
  "created_at": "...",
  "updated_at": "..."
}

Error responses:

400 — missing/invalid ip_or_cidr field, unparseable IP/CIDR
409 — duplicate entry (same ip_or_cidr already exists)
500 — database or YAML write failure

`DELETE /admin/crowdsec/whitelist/:uuid`

Response 204 — no body

Error responses:

404 — entry not found
500 — database or YAML write failure

3.3 Service Design

New file: backend/internal/services/crowdsec_whitelist_service.go

package services

import (
    "context"
    "errors"
    "net"
    "os"
    "path/filepath"
    "text/template"

    "github.com/google/uuid"
    "gorm.io/gorm"

    "github.com/yourusername/charon/backend/internal/models"
    "github.com/yourusername/charon/backend/internal/logger"
)

var (
    ErrWhitelistNotFound = errors.New("whitelist entry not found")
    ErrInvalidIPOrCIDR   = errors.New("invalid IP address or CIDR notation")
    ErrDuplicateEntry    = errors.New("whitelist entry already exists")
)

type CrowdSecWhitelistService struct {
    db      *gorm.DB
    dataDir string
}

func NewCrowdSecWhitelistService(db *gorm.DB, dataDir string) *CrowdSecWhitelistService {
    return &CrowdSecWhitelistService{db: db, dataDir: dataDir}
}

// List returns all whitelist entries ordered by creation time.
func (s *CrowdSecWhitelistService) List(ctx context.Context) ([]models.CrowdSecWhitelist, error) { ... }

// Add validates, persists, and regenerates the YAML file.
func (s *CrowdSecWhitelistService) Add(ctx context.Context, ipOrCIDR, reason string) (*models.CrowdSecWhitelist, error) { ... }

// Delete removes an entry by UUID and regenerates the YAML file.
func (s *CrowdSecWhitelistService) Delete(ctx context.Context, uuid string) error { ... }

// WriteYAML renders all current entries to <dataDir>/parsers/s02-enrich/charon-whitelist.yaml
func (s *CrowdSecWhitelistService) WriteYAML(ctx context.Context) error { ... }

Validation logic in Add():

Trim whitespace from ipOrCIDR.
Attempt net.ParseIP(ipOrCIDR) — if non-nil, it's a bare IP ✓
Attempt net.ParseCIDR(ipOrCIDR) — if err == nil, it's a valid CIDR ✓; normalize host bits immediately: ipOrCIDR = network.String() (e.g., "10.0.0.1/8" → "10.0.0.0/8").
If both fail → return ErrInvalidIPOrCIDR
Attempt DB insert; if GORM unique constraint error → return ErrDuplicateEntry
On success → call WriteYAML(ctx) (non-fatal on YAML error: log + return original entry)

Note

: Add() and Delete() do not call cscli hub reload. Reload is the caller's responsibility (handled in CrowdsecHandler.AddWhitelist and DeleteWhitelist via h.CmdExec).

CIDR normalization snippet (step 3):

if ip, network, err := net.ParseCIDR(ipOrCIDR); err == nil {
    _ = ip
    ipOrCIDR = network.String() // normalizes "10.0.0.1/8" → "10.0.0.0/8"
}

YAML generation in WriteYAML():

Guard: if s.dataDir == "", return nil immediately (no-op — used in unit tests that don't need file I/O).

const whitelistTmpl = `name: charon-whitelist
description: "Charon-managed IP/CIDR whitelist"
filter: "evt.Meta.service == 'http'"
whitelist:
  reason: "Charon managed whitelist"
  ip:
{{- range .IPs}}
    - "{{.}}"
{{- end}}
{{- if not .IPs}}
    []
{{- end}}
  cidr:
{{- range .CIDRs}}
    - "{{.}}"
{{- end}}
{{- if not .CIDRs}}
    []
{{- end}}
`

Target file path: <dataDir>/config/parsers/s02-enrich/charon-whitelist.yaml

Directory created with os.MkdirAll(..., 0o750) if absent.

File written atomically: render to <path>.tmp → os.Rename(tmp, path).

3.4 Handler Design

Additions to CrowdsecHandler struct:

type CrowdsecHandler struct {
    // ... existing fields ...
    WhitelistSvc *services.CrowdSecWhitelistService  // NEW
}

NewCrowdsecHandler constructor — initialize WhitelistSvc:

h := &CrowdsecHandler{
    // ... existing assignments ...
}
if db != nil {
    h.WhitelistSvc = services.NewCrowdSecWhitelistService(db, dataDir)
}
return h

Three new methods on CrowdsecHandler:

// ListWhitelists handles GET /admin/crowdsec/whitelist
func (h *CrowdsecHandler) ListWhitelists(c *gin.Context) {
    entries, err := h.WhitelistSvc.List(c.Request.Context())
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to list whitelist entries"})
        return
    }
    c.JSON(http.StatusOK, gin.H{"whitelist": entries})
}

// AddWhitelist handles POST /admin/crowdsec/whitelist
func (h *CrowdsecHandler) AddWhitelist(c *gin.Context) {
    var req struct {
        IPOrCIDR string `json:"ip_or_cidr" binding:"required"`
        Reason   string `json:"reason"`
    }
    if err := c.ShouldBindJSON(&req); err != nil {
        c.JSON(http.StatusBadRequest, gin.H{"error": "ip_or_cidr is required"})
        return
    }
    entry, err := h.WhitelistSvc.Add(c.Request.Context(), req.IPOrCIDR, req.Reason)
    if errors.Is(err, services.ErrInvalidIPOrCIDR) {
        c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
        return
    }
    if errors.Is(err, services.ErrDuplicateEntry) {
        c.JSON(http.StatusConflict, gin.H{"error": err.Error()})
        return
    }
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to add whitelist entry"})
        return
    }
    // Reload CrowdSec so the new entry takes effect immediately (non-fatal).
    if reloadErr := h.CmdExec.Execute("cscli", "hub", "reload"); reloadErr != nil {
        logger.Log().WithError(reloadErr).Warn("failed to reload CrowdSec after whitelist add (non-fatal)")
    }
    c.JSON(http.StatusCreated, entry)
}

// DeleteWhitelist handles DELETE /admin/crowdsec/whitelist/:uuid
func (h *CrowdsecHandler) DeleteWhitelist(c *gin.Context) {
    id := strings.TrimSpace(c.Param("uuid"))
    if id == "" {
        c.JSON(http.StatusBadRequest, gin.H{"error": "uuid required"})
        return
    }
    err := h.WhitelistSvc.Delete(c.Request.Context(), id)
    if errors.Is(err, services.ErrWhitelistNotFound) {
        c.JSON(http.StatusNotFound, gin.H{"error": "whitelist entry not found"})
        return
    }
    if err != nil {
        c.JSON(http.StatusInternalServerError, gin.H{"error": "failed to delete whitelist entry"})
        return
    }
    // Reload CrowdSec so the removed entry is no longer exempt (non-fatal).
    if reloadErr := h.CmdExec.Execute("cscli", "hub", "reload"); reloadErr != nil {
        logger.Log().WithError(reloadErr).Warn("failed to reload CrowdSec after whitelist delete (non-fatal)")
    }
    c.Status(http.StatusNoContent)
}

Route registration (append inside RegisterRoutes, after existing decision/bouncer routes):

// Whitelist management
rg.GET("/admin/crowdsec/whitelist", h.ListWhitelists)
rg.POST("/admin/crowdsec/whitelist", h.AddWhitelist)
rg.DELETE("/admin/crowdsec/whitelist/:uuid", h.DeleteWhitelist)

3.5 Startup Integration

File: backend/internal/services/crowdsec_startup.go

In ReconcileCrowdSecOnStartup(), before the CrowdSec process is started:

// Regenerate whitelist YAML to ensure it reflects the current DB state.
whitelistSvc := NewCrowdSecWhitelistService(db, dataDir)
if err := whitelistSvc.WriteYAML(ctx); err != nil {
    logger.Log().WithError(err).Warn("failed to write CrowdSec whitelist YAML on startup (non-fatal)")
}

This is non-fatal: if the DB has no entries, WriteYAML still writes an empty whitelist file, which is valid.

3.6 Hub Parser Installation

File: configs/crowdsec/install_hub_items.sh

Add after the existing cscli parsers install lines:

cscli parsers install crowdsecurity/whitelists --force || echo "⚠️ Failed to install crowdsecurity/whitelists"

3.7 Frontend Design

API Client (`frontend/src/api/crowdsec.ts`)

Append the following types and functions:

export interface CrowdSecWhitelistEntry {
  uuid: string
  ip_or_cidr: string
  reason: string
  created_at: string
  updated_at: string
}

export interface AddWhitelistPayload {
  ip_or_cidr: string
  reason: string
}

export const listWhitelists = async (): Promise<CrowdSecWhitelistEntry[]> => {
  const resp = await client.get<{ whitelist: CrowdSecWhitelistEntry[] }>('/admin/crowdsec/whitelist')
  return resp.data.whitelist
}

export const addWhitelist = async (data: AddWhitelistPayload): Promise<CrowdSecWhitelistEntry> => {
  const resp = await client.post<CrowdSecWhitelistEntry>('/admin/crowdsec/whitelist', data)
  return resp.data
}

export const deleteWhitelist = async (uuid: string): Promise<void> => {
  await client.delete(`/admin/crowdsec/whitelist/${uuid}`)
}

TanStack Query Hooks (`frontend/src/hooks/useCrowdSecWhitelist.ts`)

import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query'
import { listWhitelists, addWhitelist, deleteWhitelist, AddWhitelistPayload } from '../api/crowdsec'
import { toast } from 'sonner'

export const useWhitelistEntries = () =>
  useQuery({
    queryKey: ['crowdsec-whitelist'],
    queryFn: listWhitelists,
  })

export const useAddWhitelist = () => {
  const queryClient = useQueryClient()
  return useMutation({
    mutationFn: (data: AddWhitelistPayload) => addWhitelist(data),
    onSuccess: () => {
      toast.success('Whitelist entry added')
      queryClient.invalidateQueries({ queryKey: ['crowdsec-whitelist'] })
    },
    onError: (err: unknown) => {
      toast.error(err instanceof Error ? err.message : 'Failed to add whitelist entry')
    },
  })
}

export const useDeleteWhitelist = () => {
  const queryClient = useQueryClient()
  return useMutation({
    mutationFn: (uuid: string) => deleteWhitelist(uuid),
    onSuccess: () => {
      toast.success('Whitelist entry removed')
      queryClient.invalidateQueries({ queryKey: ['crowdsec-whitelist'] })
    },
    onError: (err: unknown) => {
      toast.error(err instanceof Error ? err.message : 'Failed to remove whitelist entry')
    },
  })
}

CrowdSecConfig.tsx Changes

The CrowdSecConfig.tsx page uses a tab navigation pattern. The new "Whitelist" tab:

Visibility: Only render the tab when isLocalMode === true (same guard as Decisions tab).
Tab value: "whitelist" — append to the existing tab list.
Tab panel content (isolated component or inline JSX):
- Add entry form: ip_or_cidr text input + reason text input + "Add" button (disabled while addMutation.isPending). Validation error shown inline when backend returns 400/409.
- Quick-add current IP: A secondary "Add My IP" button that calls GET /api/v1/system/my-ip (existing endpoint) and pre-fills the ip_or_cidr field with the returned IP.
- Entries table: Columns — IP/CIDR, Reason, Added, Actions. Each row has a delete button with a confirmation dialog (matching the ban/unban modal pattern used for Decisions).
- Empty state: "No whitelist entries" message when the list is empty.
- Loading state: Skeleton rows while useWhitelistEntries is fetching.

Imports added to CrowdSecConfig.tsx:

import { useWhitelistEntries, useAddWhitelist, useDeleteWhitelist } from '../hooks/useCrowdSecWhitelist'

3.8 Data Flow Diagram

Operator adds IP in UI
        │
        ▼
POST /api/v1/admin/crowdsec/whitelist
        │
        ▼
CrowdsecHandler.AddWhitelist()
        │
        ▼
CrowdSecWhitelistService.Add()
   ├── Validate IP/CIDR (net.ParseIP / net.ParseCIDR)
   ├── Normalize CIDR host bits (network.String())
   ├── Insert into SQLite (models.CrowdSecWhitelist)
   └── WriteYAML() → <dataDir>/config/parsers/s02-enrich/charon-whitelist.yaml
        │
        ▼
h.CmdExec.Execute("cscli", "hub", "reload") [non-fatal on error]
        │
        ▼
Return 201 to frontend
        │
        ▼
invalidateQueries(['crowdsec-whitelist'])
        │
        ▼
Table re-fetches and shows new entry

Container restart
        │
        ▼
ReconcileCrowdSecOnStartup()
        │
        ▼
CrowdSecWhitelistService.WriteYAML()
   └── Reads all DB entries → renders YAML
        │
        ▼
CrowdSec process starts
        │
        ▼
CrowdSec loads parsers/s02-enrich/charon-whitelist.yaml
   └── crowdsecurity/whitelists parser activates
        │
        ▼
IPs/CIDRs in file are exempt from all ban decisions

3.9 Error Handling Matrix

Scenario	Service Error	HTTP Status	Frontend Behavior
Blank `ip_or_cidr`	—	400	Inline validation (required field)
Malformed IP/CIDR	`ErrInvalidIPOrCIDR`	400	Toast: "Invalid IP address or CIDR notation"
Duplicate entry	`ErrDuplicateEntry`	409	Toast: "This IP/CIDR is already whitelisted"
DB unavailable	generic error	500	Toast: "Failed to add whitelist entry"
UUID not found on DELETE	`ErrWhitelistNotFound`	404	Toast: "Whitelist entry not found"
YAML write failure	logged, non-fatal	201 (Add still succeeds)	No user-facing error; log warning
CrowdSec reload failure	logged, non-fatal	201/204 (operation still succeeds)	No user-facing error; log warning

3.10 Security Considerations

Input validation: All ip_or_cidr values are validated server-side with net.ParseIP / net.ParseCIDR before persisting. Arbitrary strings are rejected.
Path traversal: WriteYAML constructs the output path via filepath.Join(s.dataDir, "config", "parsers", "s02-enrich", "charon-whitelist.yaml"). dataDir is set at startup—not user-supplied at request time.
Privilege: All three endpoints require management-level access (same as all other CrowdSec endpoints).
YAML injection: Values are rendered through Go's text/template with explicit quoting of each entry; no raw string concatenation.
Log safety: IPs are logged using the same structured field pattern used in existing CrowdSec handler methods (e.g., logger.Log().WithField("ip", entry.IPOrCIDR).Info(...)).

4. Implementation Plan

Phase 1 — Hub Parser Installation (Groundwork)

Files Changed:

configs/crowdsec/install_hub_items.sh

Task 1.1: Add cscli parsers install crowdsecurity/whitelists --force after the last parser install line (currently crowdsecurity/syslog-logs).

Acceptance: File change is syntactically valid bash; shellcheck passes.

Phase 2 — Database Model

Files Changed:

backend/internal/models/crowdsec_whitelist.go (new file)
backend/internal/api/routes/routes.go (append to AutoMigrate call)

Task 2.1: Create crowdsec_whitelist.go with the CrowdSecWhitelist struct per §3.1.

Task 2.2: Append &models.CrowdSecWhitelist{} to the db.AutoMigrate(...) call in routes.go.

Validation Gate: go build ./backend/... passes; GORM generates crowdsec_whitelists table on next startup.

Phase 3 — Whitelist Service

Files Changed:

backend/internal/services/crowdsec_whitelist_service.go (new file)

Task 3.1: Implement CrowdSecWhitelistService with List, Add, Delete, WriteYAML per §3.3.

Task 3.2: Implement IP/CIDR validation in Add():

net.ParseIP(ipOrCIDR) != nil → valid bare IP
net.ParseCIDR(ipOrCIDR) returns no error → valid CIDR
Both fail → ErrInvalidIPOrCIDR

Task 3.3: Implement WriteYAML():

Query all entries from DB.
Partition into ips (bare IPs) and cidrs (CIDR notation) slices.
Render template per §2.4.
Atomic write: temp file → os.Rename.
Create directory (os.MkdirAll) if not present.

Validation Gate: go test ./backend/internal/services/... -run TestCrowdSecWhitelist passes.

Phase 4 — API Endpoints

Files Changed:

backend/internal/api/handlers/crowdsec_handler.go

Task 4.1: Add WhitelistSvc *services.CrowdSecWhitelistService field to CrowdsecHandler struct.

Task 4.2: Initialize WhitelistSvc in NewCrowdsecHandler() when db != nil.

Task 4.3: Implement ListWhitelists, AddWhitelist, DeleteWhitelist methods per §3.4.

Task 4.4: Register three routes in RegisterRoutes() per §3.4.

Task 4.5: In AddWhitelist and DeleteWhitelist, after the service call returns without error, call h.CmdExec.Execute("cscli", "hub", "reload"). Log a warning on failure; do not change the HTTP response status (reload failure is non-fatal).

Validation Gate: go test ./backend/internal/api/handlers/... -run TestWhitelist passes; make lint-fast clean.

Phase 5 — Startup Integration

Files Changed:

backend/internal/services/crowdsec_startup.go

Task 5.1: In ReconcileCrowdSecOnStartup(), after the DB and config are loaded but before calling h.Executor.Start(), instantiate CrowdSecWhitelistService and call WriteYAML(ctx). Log warning on error; do not abort startup.

Validation Gate: go test ./backend/internal/services/... -run TestReconcile passes; existing reconcile tests still pass.

Phase 6 — Frontend API + Hooks

Files Changed:

frontend/src/api/crowdsec.ts
frontend/src/hooks/useCrowdSecWhitelist.ts (new file)

Task 6.1: Add CrowdSecWhitelistEntry, AddWhitelistPayload types and listWhitelists, addWhitelist, deleteWhitelist functions to crowdsec.ts per §3.7.

Task 6.2: Create useCrowdSecWhitelist.ts with useWhitelistEntries, useAddWhitelist, useDeleteWhitelist hooks per §3.7.

Validation Gate: pnpm test (Vitest) passes; TypeScript compilation clean.

Phase 7 — Frontend UI

Files Changed:

frontend/src/pages/CrowdSecConfig.tsx

Task 7.1: Import the three hooks from useCrowdSecWhitelist.ts.

Task 7.2: Add "whitelist" to the tab list (visible only when isLocalMode === true).

Task 7.3: Implement the Whitelist tab panel:

Add-entry form with IP/CIDR + Reason inputs.
"Add My IP" button: GET /api/v1/system/my-ip → pre-fill ip_or_cidr.
Entries table with UUID key, IP/CIDR, Reason, created date, delete button.
Delete confirmation dialog (reuse existing modal pattern).

Task 7.4: Wire mutation errors to inline form validation messages (400/409 responses).

Validation Gate: pnpm test passes; TypeScript clean; make lint-fast clean.

Phase 8 — Tests

Files Changed:

backend/internal/services/crowdsec_whitelist_service_test.go (new file)
backend/internal/api/handlers/crowdsec_whitelist_handler_test.go (new file)
tests/crowdsec-whitelist.spec.ts (new file)

Task 8.1 — Service unit tests:

Test	Scenario
`TestAdd_ValidIP_Success`	Bare IPv4 inserted; YAML file created
`TestAdd_ValidIPv6_Success`	Bare IPv6 inserted
`TestAdd_ValidCIDR_Success`	CIDR range inserted
`TestAdd_CIDRNormalization`	`"10.0.0.1/8"` stored as `"10.0.0.0/8"`
`TestAdd_InvalidIPOrCIDR_Error`	Returns `ErrInvalidIPOrCIDR`
`TestAdd_DuplicateEntry_Error`	Second identical insert returns `ErrDuplicateEntry`
`TestDelete_Success`	Entry removed; YAML regenerated
`TestDelete_NotFound_Error`	Returns `ErrWhitelistNotFound`
`TestList_Empty`	Returns empty slice
`TestList_Populated`	Returns all entries ordered by `created_at`
`TestWriteYAML_EmptyList`	Writes valid YAML with empty `ip: []` and `cidr: []`
`TestWriteYAML_MixedEntries`	IPs in `ip:` block; CIDRs in `cidr:` block
`TestWriteYAML_EmptyDataDir_NoOp`	`dataDir == ""` → returns `nil`, no file written

Task 8.2 — Handler unit tests (using in-memory SQLite + mockAuthMiddleware):

Test	Scenario
`TestListWhitelists_200`	Returns 200 with entries array
`TestAddWhitelist_201`	Valid payload → 201
`TestAddWhitelist_400_MissingField`	Empty body → 400
`TestAddWhitelist_400_InvalidIP`	Malformed IP → 400
`TestAddWhitelist_409_Duplicate`	Duplicate → 409
`TestDeleteWhitelist_204`	Valid UUID → 204
`TestDeleteWhitelist_404`	Unknown UUID → 404

Task 8.3 — E2E Playwright tests (tests/crowdsec-whitelist.spec.ts):

import { test, expect } from '@playwright/test'

test.describe('CrowdSec Whitelist Management', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('http://localhost:8080')
    await page.getByRole('link', { name: 'Security' }).click()
    await page.getByRole('tab', { name: 'CrowdSec' }).click()
    await page.getByRole('tab', { name: 'Whitelist' }).click()
  })

  test('Whitelist tab only visible in local mode', async ({ page }) => {
    await page.goto('http://localhost:8080')
    await page.getByRole('link', { name: 'Security' }).click()
    await page.getByRole('tab', { name: 'CrowdSec' }).click()
    // When CrowdSec is not in local mode, the Whitelist tab must not exist
    await expect(page.getByRole('tab', { name: 'Whitelist' })).toBeHidden()
  })

  test('displays empty state when no entries exist', async ({ page }) => {
    await expect(page.getByText('No whitelist entries')).toBeVisible()
  })

  test('adds a valid IP address', async ({ page }) => {
    await page.getByRole('textbox', { name: 'IP or CIDR' }).fill('203.0.113.5')
    await page.getByRole('textbox', { name: 'Reason' }).fill('Uptime monitor')
    await page.getByRole('button', { name: 'Add' }).click()
    await expect(page.getByText('Whitelist entry added')).toBeVisible()
    await expect(page.getByRole('cell', { name: '203.0.113.5' })).toBeVisible()
  })

  test('adds a valid CIDR range', async ({ page }) => {
    await page.getByRole('textbox', { name: 'IP or CIDR' }).fill('10.0.0.0/8')
    await page.getByRole('textbox', { name: 'Reason' }).fill('Internal subnet')
    await page.getByRole('button', { name: 'Add' }).click()
    await expect(page.getByText('Whitelist entry added')).toBeVisible()
    await expect(page.getByRole('cell', { name: '10.0.0.0/8' })).toBeVisible()
  })

  test('"Add My IP" button pre-fills the detected client IP', async ({ page }) => {
    await page.getByRole('button', { name: 'Add My IP' }).click()
    const ipField = page.getByRole('textbox', { name: 'IP or CIDR' })
    const value = await ipField.inputValue()
    // Value must be a non-empty valid IP
    expect(value).toMatch(/^[\d.]+$|^[0-9a-fA-F:]+$/)
  })

  test('shows validation error for invalid input', async ({ page }) => {
    await page.getByRole('textbox', { name: 'IP or CIDR' }).fill('not-an-ip')
    await page.getByRole('button', { name: 'Add' }).click()
    await expect(page.getByText('Invalid IP address or CIDR notation')).toBeVisible()
  })

  test('removes an entry via delete confirmation', async ({ page }) => {
    // Seed an entry first
    await page.getByRole('textbox', { name: 'IP or CIDR' }).fill('198.51.100.1')
    await page.getByRole('button', { name: 'Add' }).click()
    await expect(page.getByRole('cell', { name: '198.51.100.1' })).toBeVisible()

    // Delete it
    await page.getByRole('row', { name: /198\.51\.100\.1/ }).getByRole('button', { name: 'Delete' }).click()
    await page.getByRole('button', { name: 'Confirm' }).click()
    await expect(page.getByText('Whitelist entry removed')).toBeVisible()
    await expect(page.getByRole('cell', { name: '198.51.100.1' })).toBeHidden()
  })
})

Phase 9 — Documentation

Files Changed:

ARCHITECTURE.md
docs/features/crowdsec-whitelist.md (new file, optional for this PR)

Task 9.1: Update the CrowdSec row in the Cerberus security components table in ARCHITECTURE.md to mention whitelist management.

5. Acceptance Criteria

Functional

Operator can add a bare IPv4 address (e.g., 203.0.113.5) to the whitelist.
Operator can add a bare IPv6 address (e.g., 2001:db8::1) to the whitelist.
Operator can add a CIDR range (e.g., 10.0.0.0/8) to the whitelist.
Adding an invalid IP/CIDR (e.g., not-an-ip) returns a 400 error with a clear message.
Adding a duplicate entry returns a 409 conflict error.
Operator can delete an entry; it disappears from the list.
The Whitelist tab is only visible when CrowdSec is in local mode.
After adding or deleting an entry, the whitelist YAML file is regenerated in <dataDir>/config/parsers/s02-enrich/charon-whitelist.yaml.
Adding or removing a whitelist entry triggers cscli hub reload via h.CmdExec so changes take effect immediately without a container restart.
On container restart, the YAML file is regenerated from DB entries before CrowdSec starts.
Admin IP protection: The "Add My IP" button pre-fills the operator's current IP in the ip_or_cidr field; a Playwright E2E test verifies the button correctly pre-fills the detected client IP.

Technical

go test ./backend/... passes — no regressions.
pnpm test (Vitest) passes.
make lint-fast clean — no new lint findings.
GORM Security Scanner returns zero CRITICAL/HIGH findings.
Playwright E2E suite passes (Firefox, --project=firefox).
crowdsecurity/whitelists parser is installed by install_hub_items.sh.

6. Commit Slicing Strategy

Decision: Single PR with ordered logical commits. No scope overlap between commits; each commit leaves the codebase in a compilable state.

Trigger reasons: Cross-domain change (infra script + model + service + handler + startup + frontend) benefits from ordered commits for surgical rollback and focused review.

#	Type	Commit Message	Files	Depends On	Validation Gate
1	`chore`	`install crowdsecurity/whitelists parser by default`	`configs/crowdsec/install_hub_items.sh`	—	`shellcheck`
2	`feat`	`add CrowdSecWhitelist model and automigrate registration`	`backend/internal/models/crowdsec_whitelist.go`, `backend/internal/api/routes/routes.go`	#1	`go build ./backend/...`
3	`feat`	`add CrowdSecWhitelistService with YAML generation`	`backend/internal/services/crowdsec_whitelist_service.go`	#2	`go test ./backend/internal/services/...`
4	`feat`	`add whitelist API endpoints to CrowdsecHandler`	`backend/internal/api/handlers/crowdsec_handler.go`	#3	`go test ./backend/...` + `make lint-fast`
5	`feat`	`regenerate whitelist YAML on CrowdSec startup reconcile`	`backend/internal/services/crowdsec_startup.go`	#3	`go test ./backend/internal/services/...`
6	`feat`	`add whitelist API client functions and TanStack hooks`	`frontend/src/api/crowdsec.ts`, `frontend/src/hooks/useCrowdSecWhitelist.ts`	#4	`pnpm test`
7	`feat`	`add Whitelist tab to CrowdSecConfig UI`	`frontend/src/pages/CrowdSecConfig.tsx`	#6	`pnpm test` + `make lint-fast`
8	`test`	`add whitelist service and handler unit tests`	`*_test.go` files	#4	`go test ./backend/...`
9	`test`	`add E2E tests for CrowdSec whitelist management`	`tests/crowdsec-whitelist.spec.ts`	#7	Playwright Firefox
10	`docs`	`update architecture docs for CrowdSec whitelist feature`	`ARCHITECTURE.md`	#7	`make lint-fast`

Rollback notes:

Commits 1–3 are pure additions (no existing code modified except the AutoMigrate list append in commit 2 and install_hub_items.sh in commit 1). Reverting them is safe.
Commit 4 modifies crowdsec_handler.go by adding fields and methods without altering existing ones; reverting is mechanical.
Commit 5 modifies crowdsec_startup.go — the added block is isolated in a clearly marked section; revert is a 5-line removal.
Commits 6–7 are frontend-only; reverting has no backend impact.

7. Open Questions / Risks

Risk	Likelihood	Mitigation
CrowdSec does not hot-reload parser files — requires `cscli reload` or process restart	Resolved	`cscli hub reload` is called via `h.CmdExec.Execute(...)` in `AddWhitelist` and `DeleteWhitelist` after each successful `WriteYAML()`. Failure is non-fatal; logged as a warning.
`crowdsecurity/whitelists` parser path may differ across CrowdSec versions	Low	Use `<dataDir>/config/parsers/s02-enrich/` which is the canonical path; add a note to verify on version upgrades
Large whitelist files could cause CrowdSec performance issues	Very Low	Reasonable for typical use; document a soft limit recommendation (< 500 entries) in the UI
`dataDir` empty string in tests	Resolved	Guard added to `WriteYAML`: `if s.dataDir == "" { return nil }` — no-op when `dataDir` is unset
`CROWDSEC_TRUSTED_IPS` env var seeding	—	Follow-up / future enhancement (not in scope for this PR): if `CROWDSEC_TRUSTED_IPS` is set at runtime, parse comma-separated IPs and include them as read-only seed entries in the generated YAML (separate from DB-managed entries). Document in a follow-up issue.

35 KiB Raw Blame History Unescape Escape

CrowdSec IP Whitelist Management — Implementation Plan

1. Introduction

1.1 Overview

1.2 Objectives

2. Research Findings

2.1 Existing CrowdSec Architecture

2.2 Gap Analysis

2.3 Relevant Patterns

2.4 CrowdSec Whitelist YAML Format

3. Technical Specifications

3.1 Database Schema

3.2 API Design

Endpoint Table

GET /admin/crowdsec/whitelist

POST /admin/crowdsec/whitelist

DELETE /admin/crowdsec/whitelist/:uuid

3.3 Service Design

3.4 Handler Design

3.5 Startup Integration

3.6 Hub Parser Installation

3.7 Frontend Design

API Client (frontend/src/api/crowdsec.ts)

TanStack Query Hooks (frontend/src/hooks/useCrowdSecWhitelist.ts)

CrowdSecConfig.tsx Changes

3.8 Data Flow Diagram

3.9 Error Handling Matrix

3.10 Security Considerations

4. Implementation Plan

Phase 1 — Hub Parser Installation (Groundwork)

Phase 2 — Database Model

Phase 3 — Whitelist Service

Phase 4 — API Endpoints

Phase 5 — Startup Integration

Phase 6 — Frontend API + Hooks

Phase 7 — Frontend UI

Phase 8 — Tests

Phase 9 — Documentation

5. Acceptance Criteria

Functional

Technical

6. Commit Slicing Strategy

7. Open Questions / Risks

35 KiB

Raw Blame History

`GET /admin/crowdsec/whitelist`

`POST /admin/crowdsec/whitelist`

`DELETE /admin/crowdsec/whitelist/:uuid`

API Client (`frontend/src/api/crowdsec.ts`)

TanStack Query Hooks (`frontend/src/hooks/useCrowdSecWhitelist.ts`)