Charon/docs/plans/crowdsec_source_build.md

CrowdSec Source Build Implementation Plan - CVE-2025-68156 Remediation

Date: January 11, 2026 Priority: CRITICAL Estimated Time: 3-4 hours Target Completion: Within 48 hours

---

Executive Summary

This plan details the implementation to build CrowdSec from source in the Dockerfile to remediate CVE-2025-68156 affecting expr-lang/expr v1.17.2. Currently, the Dockerfile downloads pre-compiled CrowdSec binaries which contain the vulnerable dependency. We will implement a multi-stage build following the existing Caddy pattern to compile CrowdSec with patched expr-lang/expr v1.17.7.

Key Insight: The current Dockerfile already has a crowdsec-builder stage (lines 199-244) that builds CrowdSec from source using Go 1.25.5+. However, it does not patch the expr-lang/expr dependency. This plan adds the missing patch step.

---

Current State Analysis

Existing Dockerfile Structure

CrowdSec Builder Stage (Lines 199-244):

# ---- CrowdSec Builder ----
# Build CrowdSec from source to ensure we use Go 1.25.5+ and avoid stdlib vulnerabilities
FROM --platform=$BUILDPLATFORM golang:1.25.5-alpine AS crowdsec-builder
COPY --from=xx / /

WORKDIR /tmp/crowdsec

ARG TARGETPLATFORM
ARG TARGETOS
ARG TARGETARCH
# CrowdSec version - Renovate can update this
# renovate: datasource=github-releases depName=crowdsecurity/crowdsec
ARG CROWDSEC_VERSION=1.7.5

RUN apk add --no-cache git clang lld
RUN xx-apk add --no-cache gcc musl-dev

# Clone CrowdSec source
RUN git clone --depth 1 --branch "v${CROWDSEC_VERSION}" https://github.com/crowdsecurity/crowdsec.git .

# Build CrowdSec binaries for target architecture
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=1 xx-go build -o /crowdsec-out/crowdsec \
        -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v${CROWDSEC_VERSION}" \
        ./cmd/crowdsec && \
    xx-verify /crowdsec-out/crowdsec

RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=1 xx-go build -o /crowdsec-out/cscli \
        -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v${CROWDSEC_VERSION}" \
        ./cmd/crowdsec-cli && \
    xx-verify /crowdsec-out/cscli

# Copy config files
RUN mkdir -p /crowdsec-out/config && \
    cp -r config/* /crowdsec-out/config/ || true

Critical Findings:

1. ✅ Already builds from source with Go 1.25.5+
2. ❌ Missing expr-lang/expr dependency patch (similar to Caddy at line 181)
3. ✅ Uses multi-stage build with xx-go for cross-compilation
4. ✅ Has proper cache mounts for Go builds
5. ✅ Verifies binaries with xx-verify

Caddy Build Pattern (Reference Model)

Lines 150-195 show the pattern we need to replicate:

# Build Caddy for the target architecture with security plugins.
# Two-stage approach: xcaddy generates go.mod, we patch it, then build from scratch.
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    sh -c 'set -e; \
        export XCADDY_SKIP_CLEANUP=1; \
        echo "Stage 1: Generate go.mod with xcaddy..."; \
        # Run xcaddy to generate the build directory and go.mod
        GOOS=$TARGETOS GOARCH=$TARGETARCH xcaddy build v${CADDY_VERSION} \
            --with github.com/greenpau/caddy-security \
            --with github.com/corazawaf/coraza-caddy/v2 \
            --with github.com/hslatman/caddy-crowdsec-bouncer \
            --with github.com/zhangjiayin/caddy-geoip2 \
            --with github.com/mholt/caddy-ratelimit \
            --output /tmp/caddy-initial || true; \
        # Find the build directory created by xcaddy
        BUILDDIR=$(ls -td /tmp/buildenv_* 2>/dev/null | head -1); \
        if [ ! -d "$BUILDDIR" ] || [ ! -f "$BUILDDIR/go.mod" ]; then \
            echo "ERROR: Build directory not found or go.mod missing"; \
            exit 1; \
        fi; \
        echo "Found build directory: $BUILDDIR"; \
        cd "$BUILDDIR"; \
        echo "Stage 2: Apply security patches to go.mod..."; \
        # Patch ALL dependencies BEFORE building the final binary
        # renovate: datasource=go depName=github.com/expr-lang/expr
        go get github.com/expr-lang/expr@v1.17.7; \
        # Clean up go.mod and ensure all dependencies are resolved
        go mod tidy; \
        echo "Dependencies patched successfully"; \
        # ... build final binary with patched dependencies

Key Pattern Elements:

1. Work in cloned source directory
2. Patch go.mod with go get for vulnerable dependencies
3. Run go mod tidy to resolve dependencies
4. Build final binaries with patched dependencies
5. Add Renovate tracking comments for dependency versions

---

CrowdSec Dependency Analysis

Research Findings

From CrowdSec GitHub (crowdsecurity/crowdsec v1.7.5):

• Language: Go 81.3%
• License: MIT
• Build System: Makefile with Go build commands
• Dependencies: Managed via go.mod and go.sum
• Key Commands: ./cmd/crowdsec (daemon), ./cmd/crowdsec-cli (cscli tool)

expr-lang/expr Usage in CrowdSec:

CrowdSec uses expr-lang/expr extensively for:

• Scenario evaluation (attack pattern matching)
• Parser filters (log parsing conditional logic)
• Whitelist expressions (decision exceptions)
• Conditional rule execution

Vulnerability Impact:

CVE-2025-68156 (GHSA-cfpf-hrx2-8rv6) affects expression evaluation, potentially allowing:

• Arbitrary code execution via crafted expressions
• Denial of service through malicious scenarios
• Security bypass in rule evaluation

Required Patch: Upgrade from expr-lang/expr v1.17.2 → v1.17.7

Verification Strategy

Check Current CrowdSec Dependency:

# Extract cscli binary from built image
docker create --name crowdsec-temp charon:local
docker cp crowdsec-temp:/usr/local/bin/cscli ./cscli_binary
docker rm crowdsec-temp

# Inspect dependencies (requires Go toolchain)
go version -m ./cscli_binary | grep expr-lang
# Expected BEFORE patch: github.com/expr-lang/expr v1.17.2
# Expected AFTER patch:  github.com/expr-lang/expr v1.17.7

---

Implementation Plan

Phase 1: Add expr-lang Patch to CrowdSec Builder

Objective: Modify the crowdsec-builder stage to patch expr-lang/expr before compiling binaries.

1.1 Dockerfile Modifications

File: Dockerfile

Location: Lines 199-244 (crowdsec-builder stage)

Change Strategy: Insert dependency patching step after git clone and before go build commands.

New Implementation:

# ---- CrowdSec Builder ----
# Build CrowdSec from source to ensure we use Go 1.25.5+ and avoid stdlib vulnerabilities
# (CVE-2025-58183, CVE-2025-58186, CVE-2025-58187, CVE-2025-61729)
# Additionally, patch expr-lang/expr to v1.17.7 to fix CVE-2025-68156
# renovate: datasource=docker depName=golang versioning=docker
FROM --platform=$BUILDPLATFORM golang:1.25.5-alpine AS crowdsec-builder
COPY --from=xx / /

WORKDIR /tmp/crowdsec

ARG TARGETPLATFORM
ARG TARGETOS
ARG TARGETARCH
# CrowdSec version - Renovate can update this
# renovate: datasource=github-releases depName=crowdsecurity/crowdsec
ARG CROWDSEC_VERSION=1.7.5

# hadolint ignore=DL3018
RUN apk add --no-cache git clang lld
# hadolint ignore=DL3018,DL3059
RUN xx-apk add --no-cache gcc musl-dev

# Clone CrowdSec source
RUN git clone --depth 1 --branch "v${CROWDSEC_VERSION}" https://github.com/crowdsecurity/crowdsec.git .

# Patch expr-lang/expr dependency to fix CVE-2025-68156
# This follows the same pattern as Caddy's expr-lang patch (Dockerfile line 181)
# renovate: datasource=go depName=github.com/expr-lang/expr
RUN go get github.com/expr-lang/expr@v1.17.7 && \
    go mod tidy

# Build CrowdSec binaries for target architecture with patched dependencies
# hadolint ignore=DL3059
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=1 xx-go build -o /crowdsec-out/crowdsec \
        -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v${CROWDSEC_VERSION}" \
        ./cmd/crowdsec && \
    xx-verify /crowdsec-out/crowdsec

# hadolint ignore=DL3059
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    CGO_ENABLED=1 xx-go build -o /crowdsec-out/cscli \
        -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v${CROWDSEC_VERSION}" \
        ./cmd/crowdsec-cli && \
    xx-verify /crowdsec-out/cscli

# Copy config files
RUN mkdir -p /crowdsec-out/config && \
    cp -r config/* /crowdsec-out/config/ || true

Key Changes:

1. Line 201: Updated comment to mention CVE-2025-68156 remediation
2. Lines 220-223: NEW - Added expr-lang/expr patch step with Renovate tracking
3. Line 225: Updated comment to clarify "patched dependencies"

Rationale:

• Mirrors Caddy's proven patching approach (line 181)
• Maintains multi-stage build architecture
• Preserves cross-compilation with xx-go
• Adds Renovate tracking for automated updates
• Minimal changes (4 lines added, 2 modified)

1.2 Dockerfile Comment Updates

Add Security Note to Top-Level Comments:

Location: Near line 7-17 (version tracking section)

Addition:

## Security Patches:
## - Caddy: expr-lang/expr@v1.17.7 (CVE-2025-68156)
## - CrowdSec: expr-lang/expr@v1.17.7 (CVE-2025-68156)

This documents that both Caddy and CrowdSec have the same critical patch applied.

---

Phase 2: CI/CD Verification Integration

Objective: Add automated verification to GitHub Actions workflow to ensure CrowdSec binaries contain patched expr-lang.

2.1 GitHub Actions Workflow Enhancement

File: .github/workflows/docker-build.yml

Location: After Caddy verification (lines 157-217), add parallel CrowdSec verification

New Step:

      - name: Verify CrowdSec Security Patches (CVE-2025-68156)
        if: success()
        run: |
          echo "🔍 Verifying CrowdSec binaries contain patched expr-lang/expr@v1.17.7..."
          echo ""

          # Determine the image reference based on event type
          if [ "${{ github.event_name }}" = "pull_request" ]; then
            IMAGE_REF="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:pr-${{ github.event.pull_request.number }}"
            echo "Using PR image: $IMAGE_REF"
          else
            IMAGE_REF="${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build-and-push.outputs.digest }}"
            echo "Using digest: $IMAGE_REF"
          fi

          echo ""
          echo "==> CrowdSec cscli version:"
          timeout 30s docker run --rm $IMAGE_REF cscli version || echo "⚠️ CrowdSec version check timed out or failed (may not be installed for this architecture)"

          echo ""
          echo "==> Extracting cscli binary for inspection..."
          CONTAINER_ID=$(docker create $IMAGE_REF)
          docker cp ${CONTAINER_ID}:/usr/local/bin/cscli ./cscli_binary 2>/dev/null || {
            echo "⚠️ cscli binary not found - CrowdSec may not be available for this architecture"
            docker rm ${CONTAINER_ID}
            exit 0
          }
          docker rm ${CONTAINER_ID}

          echo ""
          echo "==> Checking if Go toolchain is available locally..."
          if command -v go >/dev/null 2>&1; then
            echo "✅ Go found locally, inspecting binary dependencies..."
            go version -m ./cscli_binary > cscli_deps.txt

            echo ""
            echo "==> Searching for expr-lang/expr dependency:"
            if grep -i "expr-lang/expr" cscli_deps.txt; then
              EXPR_VERSION=$(grep "expr-lang/expr" cscli_deps.txt | awk '{print $3}')
              echo ""
              echo "✅ Found expr-lang/expr: $EXPR_VERSION"

              # Check if version is v1.17.7 or higher (vulnerable version is v1.17.2)
              if echo "$EXPR_VERSION" | grep -E "^v1\.(1[7-9]|[2-9][0-9])\.[7-9][0-9]*$|^v1\.17\.([7-9]|[1-9][0-9]+)$" >/dev/null; then
                echo "✅ PASS: expr-lang version $EXPR_VERSION is patched (>= v1.17.7)"
              else
                echo "❌ FAIL: expr-lang version $EXPR_VERSION is vulnerable (< v1.17.7)"
                echo "⚠️ WARNING: expr-lang version $EXPR_VERSION may be vulnerable (< v1.17.7)"
                echo "Expected: v1.17.7 or higher to mitigate CVE-2025-68156"
                exit 1
              fi
            else
              echo "⚠️ expr-lang/expr not found in binary dependencies"
              echo "This could mean:"
              echo "  1. The dependency was stripped/optimized out"
              echo "  2. CrowdSec was built without the expression evaluator"
              echo "  3. Binary inspection failed"
              echo ""
              echo "Displaying all dependencies for review:"
              cat cscli_deps.txt
            fi
          else
            echo "⚠️ Go toolchain not available in CI environment"
            echo "Cannot inspect binary modules - skipping dependency verification"
            echo "Note: Runtime image does not require Go as CrowdSec is a standalone binary"
          fi

          # Cleanup
          rm -f ./cscli_binary cscli_deps.txt

          echo ""
          echo "==> CrowdSec verification complete"

Key Features:

1. Parallels Caddy verification logic exactly
2. Extracts cscli binary (more reliable than crowdsec daemon)
3. Uses go version -m to inspect embedded dependencies
4. Validates expr-lang version >= v1.17.7
5. Gracefully handles architectures where CrowdSec isn't built
6. Fails CI if vulnerable version detected

2.2 Workflow Success Criteria

Expected CI Output (Successful Patch):

✅ Go found locally, inspecting binary dependencies...

==> Searching for expr-lang/expr dependency:
github.com/expr-lang/expr v1.17.7

✅ Found expr-lang/expr: v1.17.7
✅ PASS: expr-lang version v1.17.7 is patched (>= v1.17.7)

==> CrowdSec verification complete

Expected CI Output (Failure - Pre-Patch):

❌ FAIL: expr-lang version v1.17.2 is vulnerable (< v1.17.7)
⚠️ WARNING: expr-lang version v1.17.2 may be vulnerable (< v1.17.7)
Expected: v1.17.7 or higher to mitigate CVE-2025-68156

---

Phase 3: Testing & Validation

3.1 Local Build Verification

Step 1: Clean Build

# Force rebuild without cache to ensure fresh dependencies
docker build --no-cache --progress=plain -t charon:crowdsec-patch .

Expected Build Time: 15-20 minutes (no cache)

Step 2: Extract and Verify Binary

# Extract cscli binary
docker create --name crowdsec-verify charon:crowdsec-patch
docker cp crowdsec-verify:/usr/local/bin/cscli ./cscli_binary
docker rm crowdsec-verify

# Verify expr-lang version (requires Go toolchain)
go version -m ./cscli_binary | grep expr-lang
# Expected output: github.com/expr-lang/expr v1.17.7

# Cleanup
rm ./cscli_binary

Step 3: Functional Test

# Start container
docker run -d --name crowdsec-test -p 8080:8080 -p 80:80 charon:crowdsec-patch

# Wait for CrowdSec to start
sleep 30

# Verify CrowdSec functionality
docker exec crowdsec-test cscli version
docker exec crowdsec-test cscli hub list
docker exec crowdsec-test cscli metrics

# Check logs for errors
docker logs crowdsec-test 2>&1 | grep -i "crowdsec" | tail -20

# Cleanup
docker stop crowdsec-test
docker rm crowdsec-test

Expected Results:

• ✅ cscli version shows CrowdSec v1.7.5
• ✅ cscli hub list displays installed scenarios/parsers
• ✅ cscli metrics shows metrics (or "No data" if no logs processed yet)
• ✅ No critical errors in logs

3.2 Integration Test Execution

Use Existing Test Suite:

# Run CrowdSec-specific integration tests
.vscode/tasks.json -> "Integration: CrowdSec"
# Or:
.github/skills/scripts/skill-runner.sh integration-test-crowdsec

# Run CrowdSec startup test
.vscode/tasks.json -> "Integration: CrowdSec Startup"
# Or:
.github/skills/scripts/skill-runner.sh integration-test-crowdsec-startup

# Run CrowdSec decisions test
.vscode/tasks.json -> "Integration: CrowdSec Decisions"
# Or:
.github/skills/scripts/skill-runner.sh integration-test-crowdsec-decisions

# Run all integration tests
.vscode/tasks.json -> "Integration: Run All"
# Or:
.github/skills/scripts/skill-runner.sh integration-test-all

Success Criteria:

• All CrowdSec integration tests pass
• CrowdSec starts without errors
• Hub items install correctly
• Bouncers can register
• Decisions are created and enforced
• No expr-lang evaluation errors in logs

3.3 Security Scan Verification

Trivy Scan (Post-Patch):

# Scan for CVE-2025-68156 specifically
trivy image --severity HIGH,CRITICAL charon:crowdsec-patch | grep -i "68156"
# Expected: No matches (CVE remediated)

# Full scan
trivy image --severity HIGH,CRITICAL charon:crowdsec-patch
# Expected: Zero HIGH/CRITICAL vulnerabilities

govulncheck (Go Module Scan):

# Scan built binaries for Go vulnerabilities
cd /tmp
docker create --name vuln-check charon:crowdsec-patch
docker cp vuln-check:/usr/local/bin/cscli ./cscli_test
docker cp vuln-check:/usr/local/bin/crowdsec ./crowdsec_test
docker rm vuln-check

# Check vulnerabilities (requires Go toolchain)
go run golang.org/x/vuln/cmd/govulncheck@latest -mode=binary ./cscli_test
go run golang.org/x/vuln/cmd/govulncheck@latest -mode=binary ./crowdsec_test

# Expected: No vulnerabilities found
# Cleanup
rm ./cscli_test ./crowdsec_test

---

Phase 4: Documentation Updates

4.1 Update Security Remediation Plan

File: docs/plans/security_vulnerability_remediation.md

Changes:

1. Add new section: "Phase 1.3: CrowdSec expr-lang Patch"
2. Update vulnerability inventory with CrowdSec-specific findings
3. Add task breakdown for CrowdSec patching (separate from Caddy)
4. Update timeline estimates
5. Document verification procedures

Location: After "Phase 1.2: expr-lang/expr Upgrade" (line ~120)

4.2 Update Dockerfile Comments

File: Dockerfile

Add/Update:

1. Top-level security patch documentation (line ~7-17)
2. CrowdSec builder stage comments (line ~199-202)
3. Final stage comment documenting patched binaries (line ~278-284)

4.3 Update CI Workflow Comments

File: .github/workflows/docker-build.yml

Add:

1. Comment header for CrowdSec verification step
2. Reference to CVE-2025-68156
3. Link to remediation plan document

---

Architecture Considerations

Multi-Architecture Builds

Current Support:

• ✅ amd64 (x86_64): Full CrowdSec build from source
• ✅ arm64 (aarch64): Full CrowdSec build from source
• ⚠️ Other architectures: Fallback to pre-built binaries (lines 246-274)

Impact of Patch:

• amd64/arm64: Will receive patched expr-lang binaries
• Other archs: Still vulnerable (uses pre-built binaries from fallback stage)

Recommendation: Document limitation and consider dropping support for other architectures if security is critical.

Build Performance

Expected Impact:

Stage	Before Patch	After Patch	Delta
CrowdSec Clone	10s	10s	0s
Dependency Download	30s	35s	+5s (go get)
Compilation	60s	60s	0s
Total CrowdSec Build	100s	105s	+5s

Total Dockerfile Build: ~15 minutes (no significant impact)

Cache Optimization:

• Go module cache: target=/go/pkg/mod (already present)
• Build cache: target=/root/.cache/go-build (already present)
• Subsequent builds: ~5-7 minutes (with cache)

Compatibility Considerations

CrowdSec Version Pinning:

• Current: v1.7.5 (January 2026 release)
• expr-lang in v1.7.5: Uses patched v1.17.7
• Post-patch: v1.17.7 (forced upgrade via go get)

Potential Issues:

1. Breaking API Changes: expr-lang v1.17.2 → v1.17.7 may have breaking changes in expression evaluation
2. Scenario Compatibility: Hub scenarios may rely on specific expr-lang behavior
3. Parser Compatibility: Custom parsers may break with newer expr-lang

Mitigation:

• Test comprehensive scenario evaluation (Phase 3.2)
• Monitor CrowdSec logs for expr-lang errors
• Document rollback procedure if incompatibilities found

---

Rollback Procedures

Scenario 1: expr-lang Patch Breaks CrowdSec

Symptoms:

• CrowdSec fails to start
• Scenario evaluation errors: expr: unknown function or expr: syntax error
• Hub items fail to install
• Parsers crash on log processing

Rollback Steps:

# Revert Dockerfile changes
git diff HEAD Dockerfile > /tmp/crowdsec_patch.diff
git checkout Dockerfile

# Verify original Dockerfile restored
git diff Dockerfile
# Should show no changes

# Rebuild with original (vulnerable) binaries
docker build --no-cache -t charon:rollback .

# Test rollback
docker run -d --name charon-rollback charon:rollback
docker exec charon-rollback cscli version
docker logs charon-rollback

# If successful, commit rollback
git add Dockerfile
git commit -m "revert: rollback CrowdSec expr-lang patch due to incompatibility"

Post-Rollback Actions:

1. Document specific expr-lang incompatibility
2. Check CrowdSec v1.7.5+ for native expr-lang v1.17.7 support
3. Consider waiting for upstream CrowdSec release with patched dependency
4. Re-evaluate CVE severity vs. functionality trade-off

Scenario 2: CI Verification Fails

Symptoms:

• GitHub Actions fails on "Verify CrowdSec Security Patches" step
• CI shows ❌ FAIL: expr-lang version v1.17.2 is vulnerable
• Build succeeds but verification fails

Debugging Steps:

# Check if patch was applied during build
docker build --no-cache --progress=plain -t charon:debug . 2>&1 | grep -A5 "go get.*expr-lang"

# Expected output:
# renovate: datasource=go depName=github.com/expr-lang/expr
# go get github.com/expr-lang/expr@v1.17.7
# go mod tidy

# If patch step is missing, Dockerfile wasn't updated correctly

Resolution:

1. Verify Dockerfile changes were committed
2. Check git diff against this plan
3. Ensure Renovate comment is present (for tracking)
4. Re-run build with --no-cache

Scenario 3: Integration Tests Fail

Symptoms:

• CrowdSec starts but doesn't process logs
• Scenarios don't trigger
• Decisions aren't created
• expr-lang errors in logs: failed to compile expression

Debugging:

# Check CrowdSec logs for expr errors
docker exec <container-id> cat /var/log/crowdsec/crowdsec.log | grep -i expr

# Test scenario evaluation manually
docker exec <container-id> cscli scenarios inspect crowdsecurity/http-probing

# Check parser compilation
docker exec <container-id> cscli parsers list

Resolution:

1. Identify specific scenario/parser with expr-lang issue
2. Check expr-lang v1.17.7 changelog for breaking changes
3. Update scenario expression syntax if needed
4. Report issue to CrowdSec community if upstream bug

---

Success Criteria

Critical Success Metrics

1. CVE Remediation:

   • ✅ Trivy scan shows zero instances of CVE-2025-68156
   • ✅ go version -m cscli shows expr-lang/expr v1.17.7
   • ✅ CI verification passes on all builds

2. Functional Verification:

   • ✅ CrowdSec starts without errors
   • ✅ Hub items install successfully
   • ✅ Scenarios evaluate correctly (no expr-lang errors)
   • ✅ Decisions are created and enforced
   • ✅ Bouncers function correctly

3. Integration Tests:

   • ✅ All CrowdSec integration tests pass
   • ✅ CrowdSec Startup test passes
   • ✅ CrowdSec Decisions test passes
   • ✅ Coraza WAF integration unaffected

Secondary Success Metrics

1. Build Performance:

   • ✅ Build time increase < 10 seconds
   • ✅ Image size increase < 5MB
   • ✅ Cache efficiency maintained

2. Documentation:

   • ✅ Dockerfile comments updated
   • ✅ CI workflow documented
   • ✅ Security remediation plan updated
   • ✅ Rollback procedures documented

3. CI/CD:

   • ✅ GitHub Actions includes CrowdSec verification
   • ✅ Renovate tracks expr-lang version
   • ✅ PR builds trigger verification
   • ✅ Main branch builds pass all checks

---

Timeline & Resource Allocation

Estimated Timeline (Total: 3-4 hours)

Task	Duration	Dependencies	Critical Path
Phase 1: Dockerfile Modifications	30 mins	None	Yes
Phase 2: CI/CD Integration	45 mins	Phase 1	Yes
Phase 3: Testing & Validation	90 mins	Phase 2	Yes
Phase 4: Documentation	30 mins	Phase 3	Yes

Critical Path: Phase 1 → Phase 2 → Phase 3 → Phase 4 = 3.25 hours

Contingency Buffer: +30-45 mins for troubleshooting = Total: 4 hours

Resource Requirements

Personnel:

• 1x DevOps Engineer (Dockerfile modifications, CI/CD integration)
• 1x QA Engineer (Integration testing)
• 1x Security Specialist (Verification, documentation)

Infrastructure:

• Docker build environment (20GB disk space)
• Go 1.25+ toolchain (for local verification)
• GitHub Actions runners (for CI validation)

Tools:

• Docker Desktop / Docker CLI
• Go toolchain (optional, for local binary inspection)
• trivy (security scanner)
• govulncheck (Go vulnerability scanner)

---

Post-Implementation Actions

Immediate (Within 24 hours)

1. Monitor CI/CD Pipelines:

   • Verify all builds pass CrowdSec verification
   • Check for false positives/negatives
   • Adjust regex patterns if needed

2. Update Documentation:

   • Link this plan to security_vulnerability_remediation.md
   • Update main README.md if security posture mentioned
   • Add to CHANGELOG.md

3. Notify Stakeholders:

   • Security team: CVE remediated
   • Development team: New CI check added
   • Users: Security update in next release

Short-term (Within 1 week)

1. Monitor CrowdSec Functionality:

   • Review CrowdSec logs for expr-lang errors
   • Check scenario execution metrics
   • Validate decision creation rates

2. Renovate Configuration:

   • Verify Renovate detects expr-lang tracking comment
   • Test automated PR creation for expr-lang updates
   • Document Renovate configuration for future maintainers

3. Performance Baseline:

   • Measure build time with/without cache
   • Document image size changes
   • Optimize if performance degradation observed

Long-term (Within 1 month)

1. Upstream Monitoring:

   • Watch for CrowdSec v1.7.5+ release with native expr-lang v1.17.7
   • Consider removing manual patch if upstream includes fix
   • Track expr-lang security advisories

2. Architecture Review:

   • Evaluate multi-arch support (drop unsupported architectures?)
   • Consider distroless base images for security
   • Review CrowdSec fallback stage necessity

3. Security Posture Audit:

   • Schedule quarterly Trivy scans
   • Enable Dependabot for Go modules
   • Implement automated CVE monitoring

---

Appendix A: CrowdSec Build Commands Reference

Manual CrowdSec Build (Outside Docker)

# Clone CrowdSec
git clone --depth 1 --branch v1.7.5 https://github.com/crowdsecurity/crowdsec.git
cd crowdsec

# Patch expr-lang
go get github.com/expr-lang/expr@v1.17.7
go mod tidy

# Build binaries
CGO_ENABLED=1 go build -o crowdsec \
  -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v1.7.5" \
  ./cmd/crowdsec

CGO_ENABLED=1 go build -o cscli \
  -ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v1.7.5" \
  ./cmd/crowdsec-cli

# Verify expr-lang version
go version -m ./cscli | grep expr-lang
# Expected: github.com/expr-lang/expr v1.17.7

Verify Patched Binaries

# Check embedded dependencies
go version -m /usr/local/bin/cscli | grep expr-lang

# Alternative: Use strings (less reliable)
strings /usr/local/bin/cscli | grep -i "expr-lang"

# Check version
cscli version
# Output:
# version: v1.7.5
# ...

---

Appendix B: Dockerfile Diff Preview

Expected git diff after implementation:

diff --git a/Dockerfile b/Dockerfile
index abc1234..def5678 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -5,6 +5,9 @@
 ARG VERSION=dev
 ARG BUILD_DATE
 ARG VCS_REF
+## Security Patches:
+## - Caddy: expr-lang/expr@v1.17.7 (CVE-2025-68156)
+## - CrowdSec: expr-lang/expr@v1.17.7 (CVE-2025-68156)

 # Allow pinning Caddy version - Renovate will update this
 # Build the most recent Caddy 2.x release (keeps major pinned under v3).
@@ -197,7 +200,8 @@ RUN --mount=type=cache,target=/root/.cache/go-build \

 # ---- CrowdSec Builder ----
 # Build CrowdSec from source to ensure we use Go 1.25.5+ and avoid stdlib vulnerabilities
-# (CVE-2025-58183, CVE-2025-58186, CVE-2025-58187, CVE-2025-61729)
+# (CVE-2025-58183, CVE-2025-58186, CVE-2025-58187, CVE-2025-61729)
+# Additionally, patch expr-lang/expr to v1.17.7 to fix CVE-2025-68156
 # renovate: datasource=docker depName=golang versioning=docker
 FROM --platform=$BUILDPLATFORM golang:1.25.5-alpine AS crowdsec-builder
 COPY --from=xx / /
@@ -218,7 +222,12 @@ RUN xx-apk add --no-cache gcc musl-dev
 # Clone CrowdSec source
 RUN git clone --depth 1 --branch "v${CROWDSEC_VERSION}" https://github.com/crowdsecurity/crowdsec.git .

-# Build CrowdSec binaries for target architecture
+# Patch expr-lang/expr dependency to fix CVE-2025-68156
+# This follows the same pattern as Caddy's expr-lang patch (Dockerfile line 181)
+# renovate: datasource=go depName=github.com/expr-lang/expr
+RUN go get github.com/expr-lang/expr@v1.17.7 && \
+    go mod tidy
+
+# Build CrowdSec binaries for target architecture with patched dependencies
 # hadolint ignore=DL3059
 RUN --mount=type=cache,target=/root/.cache/go-build \
     --mount=type=cache,target=/go/pkg/mod \

Total Changes:

• Added: 10 lines
• Modified: 3 lines
• Deleted: 0 lines

---

Appendix C: Troubleshooting Guide

Issue: go get fails with "no required module provides package"

Error:

go: github.com/expr-lang/expr@v1.17.7: reading github.com/expr-lang/expr/go.mod at revision v1.17.7: unknown revision v1.17.7

Cause: expr-lang/expr v1.17.7 doesn't exist or version tag is incorrect

Solution:

# Check available versions
go list -m -versions github.com/expr-lang/expr

# Use latest available version >= v1.17.7
go get github.com/expr-lang/expr@latest

Issue: go mod tidy fails after patch

Error:

go: inconsistent vendoring in /tmp/crowdsec:
 github.com/expr-lang/expr@v1.17.7: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt

Cause: Vendored dependencies out of sync

Solution:

# Remove vendor directory before patching
RUN go get github.com/expr-lang/expr@v1.17.7 && \
    rm -rf vendor && \
    go mod tidy && \
    go mod vendor

Issue: CrowdSec binary size increases significantly

Symptoms:

• cscli grows from 50MB to 80MB
• crowdsec grows from 60MB to 100MB
• Image size increases by 50MB+

Cause: Debug symbols not stripped, or expr-lang includes extra dependencies

Solution:

# Ensure -s -w flags are present in ldflags
-ldflags "-s -w -X github.com/crowdsecurity/crowdsec/pkg/cwversion.Version=v${CROWDSEC_VERSION}"

# -s: Strip symbol table
# -w: Strip DWARF debugging info

Issue: xx-verify fails after patching

Error:

ERROR: /crowdsec-out/cscli: ELF 64-bit LSB executable, x86-64, ... (NEEDED): wrong architecture

Cause: Binary built for wrong architecture (BUILDPLATFORM instead of TARGETPLATFORM)

Solution:

# Verify xx-go is being used (not regular go)
RUN ... \
    CGO_ENABLED=1 xx-go build -o /crowdsec-out/cscli \
    ...

# NOT:
RUN ... \
    CGO_ENABLED=1 go build -o /crowdsec-out/cscli \
    ...

---

References

1. CVE-2025-68156: GitHub Security Advisory GHSA-cfpf-hrx2-8rv6

   • https://github.com/advisories/GHSA-cfpf-hrx2-8rv6

2. expr-lang/expr Repository:

   • https://github.com/expr-lang/expr

3. CrowdSec GitHub Repository:

   • https://github.com/crowdsecurity/crowdsec

4. CrowdSec Build Documentation:

   • https://doc.crowdsec.net/docs/next/contributing/build_crowdsec

5. Dockerfile Best Practices:

   • https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

6. Go Module Documentation:

   • https://go.dev/ref/mod

7. Renovate Documentation:

   • https://docs.renovatebot.com/

---

Document Version: 1.0 Last Updated: January 11, 2026 Status: DRAFT - Awaiting Supervisor Review Next Review: After implementation completion

---

END OF DOCUMENT