Charon/docs/plans/current_spec.md

CI Docker Build Failure - Root Cause Analysis and Remediation Plan

Version: 1.0 Date: 2025-12-14 Status: 🔴 CRITICAL - Docker builds failing in CI

---

Executive Summary

The CI Docker build is failing during the xcaddy build process. The root cause is a Go version mismatch introduced by a recent commit that downgraded Go from 1.25.x to 1.23.x based on the incorrect assumption that Go 1.25.5 doesn't exist.

Key Finding

Go 1.25.5 IS a valid, released version (as of December 2025). The commit 481208c ("fix: correct Go version to 1.23 in Dockerfile (1.25.5 does not exist)") incorrectly downgraded Go and broke the build.

---

Root Cause Analysis

1. Version Compatibility Matrix (Current State)

Component	Version Required	Version in Dockerfile	Status
Go (for Caddy build)	1.25+	1.23 ❌	INCOMPATIBLE
Go (for backend build)	1.23+	1.23 ✅	Compatible
Caddy	2.10.2	2.10.2 ✅	Correct
xcaddy	0.4.5	latest ✅	Correct

2. The Problem

Caddy 2.10.2's go.mod declares:

go 1.25

When xcaddy tries to build Caddy 2.10.2 with Go 1.23, it fails because:

• Go's toolchain directive enforcement (Go 1.21+) prevents building modules that require a newer Go version
• The error manifests during the xcaddy build step in the Dockerfile

3. Error Location

File: Dockerfile Stage: caddy-builder (lines 101-145) Root Cause Lines:

• Line 51: FROM --platform=$BUILDPLATFORM golang:1.23-alpine AS backend-builder
• Line 101: FROM --platform=$BUILDPLATFORM golang:1.23-alpine AS caddy-builder

4. Evidence from go.mod Files

Caddy 2.10.2 (github.com/caddyserver/caddy/v2):

go 1.25

xcaddy 0.4.5 (github.com/caddyserver/xcaddy):

go 1.21
toolchain go1.23.0

Backend (/projects/Charon/backend/go.mod):

go 1.23

Workspace (/projects/Charon/go.work):

go 1.23

5. Plugin Compatibility

Plugin	Go Version Required	Caddy Version Tested
caddy-security	1.24	v2.9.1
coraza-caddy/v2	1.23	v2.9.1
caddy-crowdsec-bouncer	1.23	v2.9.1
caddy-geoip2	varies	-
caddy-ratelimit	varies	-

Note: Plugin compatibility with Caddy 2.10.2 requires Go 1.25 since Caddy itself requires it.

---

Remediation Plan

Option A: Upgrade Go to 1.25 (RECOMMENDED)

Rationale: Go 1.25.5 exists and is stable. Upgrading aligns with Caddy 2.10.2 requirements.

File Changes Required

1. Dockerfile (lines 51, 101)

Current (BROKEN):

FROM --platform=$BUILDPLATFORM golang:1.23-alpine AS backend-builder
...
FROM --platform=$BUILDPLATFORM golang:1.23-alpine AS caddy-builder

Fix:

FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS backend-builder
...
FROM --platform=$BUILDPLATFORM golang:1.25-alpine AS caddy-builder

2. backend/go.mod (line 3)

Current:

go 1.23

Fix:

go 1.25

3. go.work (line 1)

Current:

go 1.23

Fix:

go 1.25

---

Option B: Downgrade Caddy to 2.9.x (NOT RECOMMENDED)

Rationale: Would require pinning to an older Caddy version that still supports Go 1.23.

Downsides:

• Miss security fixes in Caddy 2.10.x
• Need to update CADDY_VERSION ARG
• Still need to verify plugin compatibility

File Changes:

ARG CADDY_VERSION=2.9.1  # Downgrade from 2.10.2

Not recommended because it's a regression and delays inevitable Go upgrade.

---

Recommended Implementation: Option A

Step-by-Step Remediation

Step 1: Update Dockerfile

File: Dockerfile

Line	Current	New
51	golang:1.23-alpine	golang:1.25-alpine
101	golang:1.23-alpine	golang:1.25-alpine

Step 2: Update go.mod

File: backend/go.mod

Line	Current	New
3	go 1.23	go 1.25

Then run:

cd backend && go mod tidy

Step 3: Update go.work

File: go.work

Line	Current	New
1	go 1.23	go 1.25

Step 4: Verify Local Build

# Build Docker image locally
docker build -t charon:test .

# Run the test suite
cd backend && go test ./...
cd frontend && npm run test

Step 5: Validate CI Workflows

The following workflows use Go and will automatically use the container's Go version:

• docker-build.yml - Uses Dockerfile Go version
• docker-publish.yml - Uses Dockerfile Go version
• quality-checks.yml - May need go-version update

Check if quality-checks.yml specifies Go version explicitly and update if needed.

---

Version Compatibility Matrix (After Fix)

Component	Version	Source
Go	1.25	Dockerfile, go.mod, go.work
Caddy	2.10.2	Dockerfile ARG
xcaddy	latest (0.4.5+)	go install
Node.js	24.12.0	Dockerfile
Alpine	3.23	Dockerfile

Plugin Versions (auto-resolved by xcaddy)

Plugin	Current Version	Notes
caddy-security	1.1.31	Works with Caddy 2.x
coraza-caddy/v2	2.1.0	Works with Caddy 2.x
caddy-crowdsec-bouncer	main	Works with Caddy 2.x
caddy-geoip2	main	Works with Caddy 2.x
caddy-ratelimit	main	Works with Caddy 2.x

---

Potential Side Effects

1. Backend Code Compatibility

Go 1.25 is backwards compatible with Go 1.23 code. The backend should compile without issues.

Risk: Low Mitigation: Run go build ./... and go test ./... after update.

2. CI/CD Pipeline

Some workflows may cache Go 1.23 artifacts. Force cache invalidation if builds fail after fix.

Risk: Low Mitigation: Clear GitHub Actions cache if needed.

3. Local Development

Developers using Go 1.23 locally will need to upgrade to Go 1.25.

Risk: Medium Mitigation: Document required Go version in README.md.

---

Testing Checklist

Before merging the fix:

• Local Docker build succeeds: docker build -t charon:test .
• Backend compiles: cd backend && go build ./...
• Backend tests pass: cd backend && go test ./...
• Frontend builds: cd frontend && npm run build
• Frontend tests pass: cd frontend && npm run test
• Pre-commit passes: pre-commit run --all-files
• Container starts: docker run --rm charon:test /app/charon --version
• Caddy works: docker run --rm charon:test caddy version

---

Commit Message

fix: upgrade Go to 1.25 for Caddy 2.10.2 compatibility

Caddy 2.10.2 requires Go 1.25 (declared in its go.mod). The previous
commit incorrectly downgraded to Go 1.23 based on the false assumption
that Go 1.25.5 doesn't exist.

This fix:
- Updates Dockerfile Go images from 1.23-alpine to 1.25-alpine
- Updates backend/go.mod to go 1.25
- Updates go.work to go 1.25

Fixes CI Docker build failures in xcaddy stage.

---

Files to Modify (Summary)

File	Line(s)	Change
Dockerfile	51	golang:1.23-alpine → golang:1.25-alpine
Dockerfile	101	golang:1.23-alpine → golang:1.25-alpine
backend/go.mod	3	go 1.23 → go 1.25
go.work	1	go 1.23 → go 1.25

---

Related Issues

• Previous (incorrect) fix commit: 481208c "fix: correct Go version to 1.23 in Dockerfile (1.25.5 does not exist)"
• Previous commit: 65443a1 "fix: correct Go version to 1.23 (1.25.5 does not exist)"

Both commits should be effectively reverted by this fix.

---

Appendix: Go Version Verification

As of December 14, 2025, Go 1.25.5 is available:

{
  "version": "go1.25.5",
  "stable": true,
  "files": [
    {"filename": "go1.25.5.linux-amd64.tar.gz", "...": "..."},
    {"filename": "go1.25.5.linux-arm64.tar.gz", "...": "..."},
    {"filename": "go1.25.5.darwin-amd64.tar.gz", "...": "..."}
  ]
}

Source: https://go.dev/dl/?mode=json

---

Next Steps

1. Implement the file changes listed above
2. Run local validation tests
3. Push fix with conventional commit message
4. Monitor CI pipeline for successful build
5. Update any documentation that references Go version requirements