akanealw/Charon
1
0
Fork 0
Code Issues Pull Requests Actions 16 Packages Projects Releases Wiki Activity

chore: clean .gitignore cache

Browse Source
This commit is contained in:
GitHub Actions
2026-01-26 19:21:33 +00:00
parent 1b1b3a70b1
commit e5f0fec5db
1483 changed files with 0 additions and 472793 deletions
Download Patch File Download Diff File Expand all files Collapse all files

709
docs/troubleshooting/crowdsec.md
View File

@@ -1,709 +0,0 @@
---
title: CrowdSec Troubleshooting
description: Troubleshooting guide for CrowdSec integration issues in Charon. LAPI initialization, console enrollment, and common problems.
---
## CrowdSec Troubleshooting
Keep Cerberus terminology and the Configuration Packages flow in mind while debugging Hub presets.
### Quick checks
- Cerberus is enabled and you are signed in with admin scope.
- `cscli` is available (preferred path); HTTPS CrowdSec Hub endpoints only.
- Docker images (v1.7.4+): cscli is pre-installed.
- Bare-metal deployments: install cscli for Hub preset sync or use HTTP fallback with HUB_BASE_URL.
- HUB_BASE_URL points to a JSON hub endpoint (default: <https://hub-data.crowdsec.net/api/index.json>). Redirects to HTML will be rejected.
- Proxy env is set when required: HTTP(S)_PROXY and NO_PROXY are respected by the hub client.
- For slow or proxied networks, increase HUB_PULL_TIMEOUT_SECONDS (default 25) and HUB_APPLY_TIMEOUT_SECONDS (default 45) to avoid premature timeouts.
- Preset workflow: pull from Hub using cache keys/ETags → preview changes → apply with automatic backup and reload flag.
- Preset pull/apply requires either cscli or cached presets.
- Offline/curated presets remain available at all times.
## LAPI Initialization and Timing
### Understanding LAPI Startup
When you enable CrowdSec via the GUI toggle, the Local API (LAPI) needs time to initialize before it's ready to accept requests. This is normal behavior.
**Typical startup times:**
- **Initial start:** 5-10 seconds
- **First start after container restart:** 10-15 seconds
- **Maximum wait:** 30 seconds (with automatic retries)
**What happens during startup:**
1. CrowdSec process starts
2. Configuration is loaded
3. Database connections are established
4. Parsers and scenarios are loaded
5. LAPI becomes available on port 8085
6. Status changes from "Starting" to "Active"
### Expected User Experience
When you toggle CrowdSec ON in the Security dashboard:
1. **Loading overlay appears** — "Starting CrowdSec... This may take up to 30 seconds"
2. **Backend polls LAPI** — Checks every 500ms for up to 30 seconds
3. **Success toast displays** — One of two messages:
- ✅ "CrowdSec started and LAPI is ready" — You can immediately enroll in Console
- ⚠️ "CrowdSec started but LAPI is still initializing" — Wait before enrolling
### Verifying LAPI Status
**Check if LAPI is running:**
```bash
docker exec charon cscli lapi status
```
**Expected output when ready:**
```
✓ You can successfully interact with Local API (LAPI)
```
**If LAPI is not ready yet:**
```
ERROR: connection refused
```
**Check LAPI health endpoint directly:**
```bash
docker exec charon curl -s http://localhost:8085/health
```
**Expected response when healthy:**
```json
{"status":"up"}
```
### Troubleshooting LAPI Initialization
#### Problem: LAPI takes longer than 30 seconds
**Symptoms:**
- Warning message: "LAPI is still initializing"
- Console enrollment fails with "LAPI not available"
**Solution 1 - Wait and retry:**
```bash
# Wait 15 seconds, then check again
sleep 15
docker exec charon cscli lapi status
```
**Solution 2 - Check CrowdSec logs:**
```bash
docker logs charon | grep -i crowdsec | tail -20
```
Look for:
- ✅ "CrowdSec Local API listening" — LAPI started successfully
- ✅ "parsers loaded" — Configuration loaded
- ❌ "error" or "fatal" — Initialization problem
**Solution 3 - Restart CrowdSec:**
1. Go to Security dashboard
2. Toggle CrowdSec **OFF**
3. Wait 5 seconds
4. Toggle CrowdSec **ON**
5. Wait 15 seconds
6. Verify status shows "Active"
#### Problem: LAPI never becomes available
**Check if CrowdSec process is running:**
```bash
docker exec charon ps aux | grep crowdsec
```
**Expected output:**
```
crowdsec 203 0.5 2.3 /usr/local/bin/crowdsec -c /app/data/crowdsec/config/config.yaml
```
**If no process is running:**
1. Check config directory exists:
```bash
docker exec charon ls -la /app/data/crowdsec/config
```
2. If directory is missing:
```bash
docker compose restart
```
3. Check for port conflicts:
```bash
docker exec charon netstat -tulpn | grep 8085
```
4. Remove deprecated environment variables from docker-compose.yml (see migration section below)
#### Problem: LAPI responds but enrollment fails
**Check LAPI can process requests:**
```bash
docker exec charon cscli machines list
```
**Expected output:**
```
Name IP Address Auth Type Version
charon-local-machine 127.0.0.1 password v1.x.x
```
**If command fails:**
- LAPI is running but database is not ready
- Wait 10 more seconds and retry
- Check logs for database errors
**If enrollment still fails:**
- Enrollment has automatic retry (3 attempts, 2 seconds apart)
- If all retries fail, toggle CrowdSec OFF/ON and try again
- See Console Enrollment section below for token troubleshooting
## Common issues
- Hub unreachable (503): retry once, then Charon falls back to cached Hub data if available; otherwise stay on curated/offline presets until connectivity returns.
- Hub returns HTML/redirect: set HUB_BASE_URL to the JSON endpoint above or install cscli so the index is fetched locally.
- Bad preset slug (400): the slug must match Hub naming; correct the slug before retrying.
- Apply failed: review the apply response and restore from the backup that was taken automatically, then retry after fixing the underlying issue.
- Apply not supported (501): use curated/offline presets; Hub apply will be re-enabled when supported in your environment.
- **Security Engine Offline**: If your dashboard says "Offline", it means CrowdSec LAPI is not running.
- **Fix**: Ensure CrowdSec is **enabled via GUI toggle** in the Security dashboard. Do NOT use environment variables.
- **Action**: Go to Security dashboard, toggle CrowdSec ON, wait 15 seconds, verify status shows "Active".
## CrowdSec Not Starting After Container Restart
### Problem: Toggle shows ON but CrowdSec is not running
**Symptoms:**
- Container restarted (reboot, Docker restart, etc.)
- Security dashboard toggle shows "ON"
- Status badge shows "Not Running" or "Offline"
- Manually toggling OFF then ON fixes it
**Root Cause:**
The reconciliation function couldn't determine if CrowdSec should auto-start. This happens when:
1. **SecurityConfig table is missing/corrupted** (database issue)
2. **Settings table and SecurityConfig are out of sync** (partial update)
3. **Reconciliation logs show silent exit** (no "starting based on" message)
### Diagnosis: Check Reconciliation Logs
**View container startup logs:**
```bash
docker logs charon | grep -i "crowdsec reconciliation"
```
**Expected output when working correctly:**
```json
{"level":"info","msg":"CrowdSec reconciliation: starting startup check","time":"..."}
{"level":"info","msg":"CrowdSec reconciliation: starting based on SecurityConfig mode='local'","time":"..."}
{"level":"info","msg":"CrowdSec Local API listening on 127.0.0.1:8085","time":"..."}
```
**Problematic output (silent exit - BUG):**
```json
{"level":"info","msg":"CrowdSec reconciliation: starting startup check","time":"..."}
[NO FURTHER LOGS - Function exited without starting CrowdSec]
```
This indicates reconciliation found conflicting state between Settings and SecurityConfig tables.
### Solution 1: Verify Database State
**Check Settings table:**
```bash
docker exec charon sqlite3 /app/data/charon.db \
"SELECT key, value FROM settings WHERE key = 'security.crowdsec.enabled';"
```
**Expected output:**
```
security.crowdsec.enabled|true
```
**Check SecurityConfig table:**
```bash
docker exec charon sqlite3 /app/data/charon.db \
"SELECT uuid, crowdsec_mode, enabled FROM security_configs WHERE uuid = 'default';"
```
**Expected output:**
```
default|local|1
```
**Mismatch scenarios:**
| Settings | SecurityConfig | Behavior | Fix Needed |
|----------|----------------|----------|------------|
| `true` | `local` | ✅ Auto-starts | None |
| `true` | `disabled` | ❌ Does NOT start | Run Solution 2 |
| `true` | (missing) | ⚠️ Should auto-create | Run Solution 3 |
| `false` | `local` | ⚠️ Conflicting state | Run Solution 2 |
| `false` | `disabled` | ✅ Correctly skipped | None (expected) |
### Solution 2: Manually Sync SecurityConfig to Settings
**If you want CrowdSec enabled (Settings = true, SecurityConfig = disabled):**
```bash
docker exec charon sqlite3 /app/data/charon.db \
"UPDATE security_configs SET crowdsec_mode = 'local', enabled = 1 WHERE uuid = 'default';"
docker restart charon
```
**If you want CrowdSec disabled (Settings = false, SecurityConfig = local):**
```bash
docker exec charon sqlite3 /app/data/charon.db \
"UPDATE security_configs SET crowdsec_mode = 'disabled', enabled = 0 WHERE uuid = 'default';"
# Also update Settings for consistency
docker exec charon sqlite3 /app/data/charon.db \
"UPDATE settings SET value = 'false' WHERE key = 'security.crowdsec.enabled';"
docker restart charon
```
### Solution 3: Force Recreation of SecurityConfig
**If SecurityConfig table is missing (record not found):**
```bash
# Delete SecurityConfig (if partial record exists)
docker exec charon sqlite3 /app/data/charon.db \
"DELETE FROM security_configs WHERE uuid = 'default';"
# Restart container - reconciliation will auto-create matching Settings state
docker restart charon
# Wait 15 seconds for startup
sleep 15
# Verify CrowdSec started
docker exec charon cscli lapi status
```
**Expected behavior:**
- Reconciliation detects missing SecurityConfig
- Checks Settings table for user preference
- Creates SecurityConfig with matching state
- Starts CrowdSec if Settings = true
**Check logs to confirm:**
```bash
docker logs charon | grep "default SecurityConfig created"
```
Expected:
```json
{"level":"info","msg":"CrowdSec reconciliation: default SecurityConfig created from Settings preference","crowdsec_mode":"local","enabled":true,"source":"settings_table"}
```
### Solution 4: Use GUI Toggle (Safest)
**The GUI toggle synchronizes both tables atomically:**
1. Go to **Security** dashboard
2. Toggle CrowdSec **OFF** (if it shows ON)
3. Wait 5 seconds
4. Toggle CrowdSec **ON**
5. Wait 15 seconds for LAPI to initialize
6. Verify status shows "Active"
**Why this works:**
- Toggle updates Settings table
- Toggle updates SecurityConfig table
- Start handler ensures both tables match
- Future restarts use reconciliation correctly
### Solution 5: Manual Reset (Nuclear Option)
**If all else fails, reset both tables:**
```bash
# Stop CrowdSec if running
docker exec charon pkill crowdsec || true
# Reset both tables
docker exec charon sqlite3 /app/data/charon.db <<EOF
UPDATE settings SET value = 'false' WHERE key = 'security.crowdsec.enabled';
DELETE FROM security_configs WHERE uuid = 'default';
EOF
# Restart container
docker restart charon
# Re-enable via GUI
# Go to Security dashboard and toggle CrowdSec ON
```
### Prevention: Verify After Manual Database Changes
**If you manually edit the database:**
```bash
# Always verify both tables match
docker exec charon sqlite3 /app/data/charon.db <<EOF
SELECT 'Settings:' as table_name, value as state
FROM settings WHERE key = 'security.crowdsec.enabled'
UNION ALL
SELECT 'SecurityConfig:', crowdsec_mode
FROM security_configs WHERE uuid = 'default';
EOF
```
**Expected output (both enabled):**
```
Settings:|true
SecurityConfig:|local
```
**Expected output (both disabled):**
```
Settings:|false
SecurityConfig:|disabled
```
### When to Contact Support
If after following all solutions:
- ❌ Reconciliation logs still show silent exit
- ❌ Both tables show correct state but CrowdSec doesn't start
- ❌ Manual `cscli lapi status` fails even after toggle
**Gather diagnostic info:**
```bash
# Collect logs
docker logs charon > charon-logs.txt 2>&1
# Collect database state
docker exec charon sqlite3 /app/data/charon.db ".dump security_configs" > db-state.sql
docker exec charon sqlite3 /app/data/charon.db ".dump settings" >> db-state.sql
# Collect process state
docker exec charon ps aux > process-state.txt
```
**Report issue:** <https://github.com/Wikid82/charon/issues>
Include:
- Output of all diagnostic commands above
- Steps you tried from this guide
- Container restart logs showing reconciliation behavior
## Tips
- Keep the CrowdSec Hub reachable over HTTPS; HTTP is blocked.
- If you switch to offline mode, clear pending Hub pulls before retrying so cache keys/ETags refresh cleanly.
- After restoring from a backup, re-run preview before applying again to verify changes.
- **Always use the GUI toggle** for enabling/disabling CrowdSec—it ensures Settings and SecurityConfig stay synchronized.
- **Check reconciliation logs** after container restart to verify auto-start behavior.
## Database Migrations After Upgrade
### Problem: CrowdSec not starting after upgrading Charon
**Symptoms:**
- CrowdSec toggle appears enabled but status shows "Not Running"
- CrowdSec console shows "Starting..." indefinitely
- Container logs show: `WARN CrowdSec reconciliation: security tables missing`
- Console enrollment fails immediately
**Root Cause:**
Upgrading from an older version with a **persistent database** may be missing the new security tables introduced in version 2.0. The database schema needs to be migrated.
**Solution: Run Database Migration**
1. **Execute the migration command:**
```bash
docker exec charon /app/charon migrate
```
**Expected output:**
```json
{"level":"info","msg":"Running database migrations for security tables...","time":"..."}
{"level":"info","msg":"Migration completed successfully","time":"..."}
```
2. **Verify tables were created:**
```bash
docker exec charon sqlite3 /app/data/charon.db ".tables"
```
**Expected tables include:**
- `security_configs`
- `security_decisions`
- `security_audits`
- `security_rule_sets`
- `crowdsec_preset_events`
- `crowdsec_console_enrollments`
3. **Restart container to apply changes:**
```bash
docker restart charon
```
4. **Verify CrowdSec starts automatically:**
If you had CrowdSec enabled before the upgrade:
```bash
# Wait 15 seconds after restart, then check
docker exec charon cscli lapi status
```
**Expected output:**
```
✓ You can successfully interact with Local API (LAPI)
```
5. **If CrowdSec doesn't auto-start:**
Enable it manually via the GUI:
- Go to **Security** dashboard
- Toggle CrowdSec **ON**
- Wait 15 seconds
- Verify status shows "Active"
**Why This Happens:**
Charon version 2.0 moved CrowdSec configuration from environment variables to the database (see [Migration Guide](../migration-guide.md)). Persistent databases from older versions need the new security tables added via migration.
**Prevention:**
Future upgrades will run migrations automatically on startup. For now, manual migration is required for existing installations.
**Related Documentation:**
- [Getting Started - Database Migrations](../getting-started.md#step-15-database-migrations-if-upgrading)
- [Migration Guide - CrowdSec Control](../migration-guide.md)
---
## Console Enrollment
### Prerequisites
Before attempting Console enrollment, ensure:
✅ **CrowdSec is enabled** — Toggle must be ON in Security dashboard
✅ **LAPI is running** — Check with: `docker exec charon cscli lapi status`
✅ **Feature flag enabled** — `feature.crowdsec.console_enrollment` must be ON
✅ **Valid token** — Obtain from crowdsec.net
### "missing login field" or CAPI errors
Charon automatically attempts to register your instance with CrowdSec's Central API (CAPI) before enrolling. Ensure your server has internet access to `api.crowdsec.net`.
### Enrollment shows "enrolled" but not on crowdsec.net
**Root cause:** LAPI was not running when enrollment was attempted.
Charon now checks LAPI availability before enrollment and retries automatically (3 attempts with 2-second delays), but in rare cases enrollment may still fail if LAPI is initializing.
**Solution:**
1. Verify LAPI status:
```bash
docker exec charon cscli lapi status
```
**Expected output when ready:**
```
✓ You can successfully interact with Local API (LAPI)
```
**If LAPI is not running:**
```
ERROR: cannot contact local API
```
2. If LAPI is not running:
- Go to Security dashboard
- Toggle CrowdSec **OFF**
- Wait 5 seconds
- Toggle CrowdSec **ON**
- **Wait 15 seconds** (important: LAPI needs time to initialize)
- Re-check LAPI status
3. Verify LAPI health endpoint:
```bash
docker exec charon curl -s http://localhost:8085/health
```
**Expected response:**
```json
{"status":"up"}
```
4. Re-submit enrollment token:
- Go to **Cerberus → CrowdSec**
- Click **Enroll with CrowdSec Console**
- Paste the same enrollment token (tokens are reusable)
- Click **Submit**
- Wait 30-60 seconds for confirmation
5. Verify enrollment on crowdsec.net:
- Log in to your CrowdSec Console account
- Navigate to **Instances**
- Your Charon instance should appear in the list
**Understanding the automatic retry:**
Charon automatically retries LAPI checks during enrollment:
- **Attempt 1:** Immediate check
- **Attempt 2:** After 2 seconds (if LAPI not ready)
- **Attempt 3:** After 4 seconds (if still not ready)
- **Total:** 3 attempts over 6 seconds
This handles most cases where LAPI is still initializing. If all 3 attempts fail, follow the solution above.
### CrowdSec won't start via GUI toggle
**Solution:**
1. Check container logs:
```bash
docker logs charon | grep -i crowdsec
```
Look for:
- ✅ "Starting CrowdSec Local API"
- ✅ "CrowdSec Local API listening on 127.0.0.1:8085"
- ❌ "failed to start" or "error loading config"
2. Verify config directory:
```bash
docker exec charon ls -la /app/data/crowdsec/config
```
Expected files:
- `config.yaml` — Main configuration
- `local_api_credentials.yaml` — LAPI authentication
- `acquis.yaml` — Log sources
3. Check for common startup errors:
**Error: "config.yaml not found"**
```bash
# Restart container to regenerate config
docker compose restart
```
**Error: "port 8085 already in use"**
```bash
# Check for conflicting services
docker exec charon netstat -tulpn | grep 8085
# Stop conflicting service or change CrowdSec LAPI port
```
**Error: "permission denied"**
```bash
# Fix ownership (run on host)
sudo chown -R 1000:1000 ./data/crowdsec
docker compose restart
```
4. Remove any deprecated environment variables from docker-compose.yml:
```yaml
# REMOVE THESE:
- CHARON_SECURITY_CROWDSEC_MODE=local
- CERBERUS_SECURITY_CROWDSEC_MODE=local
- CPM_SECURITY_CROWDSEC_MODE=local
```
5. Restart and try GUI toggle again:
```bash
docker compose restart
# Wait 30 seconds for container to fully start
# Then toggle CrowdSec ON in GUI
```
6. Verify CrowdSec is running:
```bash
# Check process
docker exec charon ps aux | grep crowdsec
# Check LAPI health
docker exec charon cscli lapi status
# Check LAPI endpoint
docker exec charon curl -s http://localhost:8085/health
```
### Environment Variable Migration
🚨 **DEPRECATED:** The `CHARON_SECURITY_CROWDSEC_MODE` environment variable is no longer used.
If you have this in your docker-compose.yml, remove it and use the GUI toggle instead. See [Migration Guide](../migration-guide.md) for step-by-step instructions.
### Configuration File
Charon uses the configuration located in `data/crowdsec/config.yaml`. Ensure this file exists and is readable if you are manually modifying it.

479
docs/troubleshooting/dns-challenges.md
View File

@@ -1,479 +0,0 @@
# DNS Challenge Troubleshooting
This guide covers common issues with DNS-01 ACME challenges and how to resolve them.
## Table of Contents
- [Connection Test Failures](#connection-test-failures)
- [Certificate Issuance Failures](#certificate-issuance-failures)
- [DNS Propagation Issues](#dns-propagation-issues)
- [Provider-Specific Errors](#provider-specific-errors)
- [Network and Firewall Issues](#network-and-firewall-issues)
- [Credential Problems](#credential-problems)
- [Debugging Tips](#debugging-tips)
## Connection Test Failures
### Invalid Credentials
**Symptoms:**
- "Invalid API token" or "Unauthorized" error
- Connection test fails immediately
**Solutions:**
1. Verify credentials were copied correctly (no extra spaces/newlines)
2. Check token/key hasn't expired
3. Ensure token has required permissions:
- Cloudflare: Zone → DNS → Edit
- AWS: `route53:ChangeResourceRecordSets`
- DigitalOcean: Write scope
4. Regenerate credentials if necessary
5. Update configuration in Charon with new credentials
### DNS Provider Unreachable
**Symptoms:**
- "Connection timeout" or "Network error"
- Test hangs for 30+ seconds
**Solutions:**
1. Check internet connectivity from Charon server
2. Verify firewall allows outbound HTTPS (port 443)
3. Test DNS resolution:
```bash
# Test DNS provider API endpoint resolution
nslookup api.cloudflare.com
curl -I https://api.cloudflare.com
```
4. Check provider status page for service outages
5. Verify proxy settings if using HTTP proxy
### Zone/Domain Not Found
**Symptoms:**
- "Hosted zone not found"
- "Domain not configured"
**Solutions:**
1. Verify domain is added to DNS provider account
2. Ensure domain status is Active (not Pending)
3. Check nameservers are correctly configured:
```bash
dig NS example.com +short
```
4. Wait 24-48 hours if nameservers were recently changed
5. Verify API token is scoped to include the domain (if applicable)
## Certificate Issuance Failures
### DNS Propagation Timeout
**Symptoms:**
- Certificate issuance fails after 2-5 minutes
- Error: "DNS propagation timeout" or "TXT record not found"
**Solutions:**
1. **Increase propagation timeout:**
- Navigate to DNS Provider settings
- Increase Propagation Timeout to 180-300 seconds
- Save and retry certificate issuance
2. **Verify DNS propagation:**
```bash
# Check if TXT record was created
dig _acme-challenge.example.com TXT +short
# Check from multiple DNS servers
dig _acme-challenge.example.com TXT @8.8.8.8 +short
dig _acme-challenge.example.com TXT @1.1.1.1 +short
```
3. **Check DNS provider configuration:**
- Ensure domain's nameservers point to your DNS provider
- Verify no conflicting DNS records exist
- Check DNSSEC is properly configured (if enabled)
4. **Provider-specific adjustments:**
- **Cloudflare:** Usually fast (60s), check Cloudflare status
- **Route 53:** Often slow (120-180s), increase timeout
- **DigitalOcean:** Moderate (90s), verify nameservers
### ACME Server Errors
**Symptoms:**
- "Too many requests" or "Rate limit exceeded"
- "Invalid response from ACME server"
**Solutions:**
1. **Let's Encrypt rate limits:**
- 50 certificates per domain per week
- 5 failed validation attempts per hour
- Wait before retrying if limit hit
- Use staging environment for testing:
```bash
# In Caddy config (for testing only)
acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
```
2. **ACME challenge failures:**
- Review Charon logs for specific ACME error codes
- Verify TXT record was created correctly
- Ensure DNS provider has write permissions
- Test with a different DNS provider (if available)
3. **Boulder (Let's Encrypt) validation errors:**
- Error indicates which authoritative DNS server was queried
- Verify all nameservers return the TXT record
- Check for split-horizon DNS issues
### Wildcard Domain Issues
**Symptoms:**
- Wildcard certificate issuance fails
- Error: "DNS challenge required for wildcard domains"
**Solutions:**
1. Verify DNS provider is configured in Charon
2. Select DNS provider when creating proxy host
3. Ensure wildcard syntax is correct: `*.example.com`
4. Confirm DNS provider has permissions for the root domain
5. Test with non-wildcard domain first (e.g., `test.example.com`)
## DNS Propagation Issues
### Slow Global Propagation
**Symptoms:**
- Certificate issuance succeeds locally but fails remotely
- Inconsistent results from different DNS resolvers
**Diagnostic Commands:**
```bash
# Check propagation from multiple locations
dig _acme-challenge.example.com TXT @8.8.8.8
dig _acme-challenge.example.com TXT @1.1.1.1
dig _acme-challenge.example.com TXT @208.67.222.222
# Check TTL of existing records
dig example.com +noall +answer | grep -i ttl
```
**Solutions:**
1. Increase Propagation Timeout to 300-600 seconds
2. Lower TTL on existing DNS records (set 1 hour before changes)
3. Wait for previous high-TTL records to expire
4. Use DNS provider with faster global propagation (e.g., Cloudflare)
### Cached DNS Records
**Symptoms:**
- Old TXT records still visible after deletion
- Certificate renewal fails with "Incorrect TXT record"
**Solutions:**
1. Wait for TTL expiry (default: 300-3600 seconds)
2. Flush local DNS cache:
```bash
# Linux
sudo systemd-resolve --flush-caches
# macOS
sudo dscacheutil -flushcache
```
3. Test with authoritative nameservers directly:
```bash
dig _acme-challenge.example.com TXT @ns1.your-provider.com
```
## Provider-Specific Errors
### Cloudflare
**Error:** `Cloudflare API error 6003: Invalid request headers`
- **Cause:** Malformed API token
- **Solution:** Regenerate token, ensure no invisible characters
**Error:** `Cloudflare API error 10000: Authentication error`
- **Cause:** Token revoked or expired
- **Solution:** Create new token with correct permissions
**Error:** `Zone is not active`
- **Cause:** Nameservers not updated
- **Solution:** Update domain nameservers, wait for activation
### AWS Route 53
**Error:** `AccessDenied: User is not authorized`
- **Cause:** IAM permissions insufficient
- **Solution:** Attach IAM policy with `route53:ChangeResourceRecordSets`
**Error:** `InvalidChangeBatch: RRSet with duplicate name`
- **Cause:** Conflicting TXT record already exists
- **Solution:** Remove manual `_acme-challenge` TXT records
**Error:** `Throttling: Rate exceeded`
- **Cause:** Too many API requests
- **Solution:** Increase polling interval to 15-20 seconds
### DigitalOcean
**Error:** `The resource you requested could not be found`
- **Cause:** Domain not in DigitalOcean DNS
- **Solution:** Add domain to Networking → Domains
**Error:** `Unable to authenticate you`
- **Cause:** Token has Read scope instead of Write
- **Solution:** Regenerate token with Write scope
## Network and Firewall Issues
### Outbound HTTPS Blocked
**Symptoms:**
- Connection tests timeout
- "Network unreachable" errors
**Diagnostic Commands:**
```bash
# Test connectivity to DNS provider API
curl -v https://api.cloudflare.com/client/v4/user
curl -v https://api.digitalocean.com/v2/account
# Check if firewall is blocking
sudo iptables -L OUTPUT -v -n | grep -i drop
```
**Solutions:**
1. Allow outbound HTTPS (port 443) in firewall
2. Whitelist DNS provider API endpoints
3. Configure HTTP proxy if required:
```bash
export HTTP_PROXY=http://proxy.example.com:8080
export HTTPS_PROXY=http://proxy.example.com:8080
```
### DNS Resolution Failures
**Symptoms:**
- Cannot resolve DNS provider API domains
- Error: "No such host"
**Diagnostic Commands:**
```bash
# Test DNS resolution
nslookup api.cloudflare.com
dig api.cloudflare.com
# Check /etc/resolv.conf
cat /etc/resolv.conf
```
**Solutions:**
1. Verify DNS server is configured correctly
2. Test with public DNS (8.8.8.8, 1.1.1.1)
3. Check network interface configuration
4. Restart networking service
## Credential Problems
### Encryption Key Issues
**Symptoms:**
- "Encryption key not configured"
- "Failed to decrypt credentials"
**Solutions:**
1. **Set encryption key:**
```bash
# Generate new key
openssl rand -base64 32
# Set environment variable
export CHARON_ENCRYPTION_KEY="your-base64-key-here"
```
2. **Verify key in environment:**
```bash
echo $CHARON_ENCRYPTION_KEY
# Should show 44-character base64 string
```
3. **Docker/Docker Compose:**
```yaml
# docker-compose.yml
services:
charon:
environment:
- CHARON_ENCRYPTION_KEY=${CHARON_ENCRYPTION_KEY}
```
4. **Restart Charon after setting key**
### Credentials Lost After Restart
**Symptoms:**
- DNS provider shows "Unconfigured" status after restart
- Connection test fails with "Invalid credentials"
**Cause:** Encryption key changed or missing
**Solutions:**
1. Ensure `CHARON_ENCRYPTION_KEY` is persistent (not temporary)
2. Add to systemd service file, docker-compose, or .env file
3. Never change encryption key (all credentials will be unrecoverable)
4. If key is lost, reconfigure all DNS providers
## Debugging Tips
### Enable Debug Logging
```bash
# Set log level in Charon configuration
export CHARON_LOG_LEVEL=debug
# Restart Charon
```
### Review Charon Logs
```bash
# Docker
docker logs charon -f --tail 100
# Systemd
journalctl -u charon -f -n 100
# Look for lines containing:
# - "DNS provider"
# - "ACME challenge"
# - "Certificate issuance"
```
### Test DNS Provider Manually
Use Caddy directly to test DNS provider:
```bash
# Create test Caddyfile
cat > Caddyfile << 'EOF'
test.example.com {
tls {
dns cloudflare {env.CLOUDFLARE_API_TOKEN}
}
respond "Test successful"
}
EOF
# Run Caddy with test config
CLOUDFLARE_API_TOKEN=your-token caddy run --config Caddyfile
```
### Check ACME Challenge TXT Record
Monitor DNS changes during certificate issuance:
```bash
# Watch for TXT record creation
watch -n 5 'dig _acme-challenge.example.com TXT +short'
# Check authoritative nameservers
dig _acme-challenge.example.com TXT @$(dig NS example.com +short | head -1)
```
### Common Log Messages
**Success:**
```
[INFO] DNS provider test successful
[INFO] ACME challenge completed
[INFO] Certificate issued successfully
```
**Errors:**
```
[ERROR] Failed to create TXT record: <reason>
[ERROR] DNS propagation timeout after 120 seconds
[ERROR] ACME validation failed: <acme-error>
```
## Getting Help
If you're still experiencing issues:
1. **Review Documentation:**
- [DNS Providers Overview](../guides/dns-providers.md)
- Provider-specific setup guides
- [Security best practices](../security/best-practices.md)
2. **Gather Information:**
- Charon version and log excerpt
- DNS provider type
- Error message (exact text)
- Network environment (Docker, VPS, etc.)
3. **Check Known Issues:**
- [GitHub Issues](https://github.com/Wikid82/Charon/issues)
- Release notes and changelogs
4. **Contact Support:**
- Open a GitHub issue with debug logs
- Join community Discord/forum
- Include relevant diagnostic output (sanitize credentials)
## Related Documentation
- [DNS Providers Guide](../guides/dns-providers.md)
- [Cloudflare Setup](../guides/dns-providers/cloudflare.md)
- [AWS Route 53 Setup](../guides/dns-providers/route53.md)
- [DigitalOcean Setup](../guides/dns-providers/digitalocean.md)
- [Certificate Management](../guides/certificates.md)

17
docs/troubleshooting/go-gopls.md
View File

@@ -1,17 +0,0 @@
---
title: Troubleshooting gopls / VS Code Go Errors
description: Resolve gopls and VS Code Go extension errors in the Charon repository. Log collection and common fixes.
---
## Troubleshooting gopls / VS Code Go Errors in Charon
This page documents how to triage and collect logs for persistent Go errors shown by gopls or VS Code in the Charon repository.
### Steps
1. Open the Charon workspace in VS Code (project root).
2. Accept the workspace settings prompt to apply .vscode/settings.json.
3. Run the workspace task: Go: Build Backend (or run ./scripts/check_go_build.sh).
4. If errors persist, run ./scripts/gopls_collect.sh and attach the output directory to an issue.
When filing upstream issues, include gopls.log, go-version.txt, go-env.txt, and a short reproduction.

476
docs/troubleshooting/proxy-headers.md
View File

@@ -1,476 +0,0 @@
---
title: Troubleshooting Standard Proxy Headers
description: Resolve issues with Charon's X-Real-IP, X-Forwarded-Proto, and other standard proxy headers.
---
## Troubleshooting Standard Proxy Headers
This guide helps resolve issues with Charon's standard proxy headers feature.
---
## Understanding Standard Proxy Headers
When enabled, Charon adds these headers to requests sent to your backend:
- **X-Real-IP**: The actual client IP address (not Charon's)
- **X-Forwarded-Proto**: Original protocol (http or https)
- **X-Forwarded-Host**: Original hostname from the client
- **X-Forwarded-Port**: Original port number
- **X-Forwarded-For**: Chain of proxy IPs (managed by Caddy)
---
## Problem: Backend Still Sees Charon's IP
### Symptoms
Your application logs show Charon's internal IP (e.g., `172.17.0.1`) instead of the real client IP.
### Solutions
**1. Verify the feature is enabled**
- Go to **Proxy Hosts** → Edit the affected host
- Scroll to **Standard Proxy Headers** section
- Ensure the checkbox is **checked**
- Click **Save**
**2. Configure backend to trust proxy headers**
Your backend application must be configured to read these headers. Here's how:
**Express.js/Node.js:**
```javascript
// Enable trust proxy
app.set('trust proxy', true);
// Now req.ip will use X-Real-IP or X-Forwarded-For
app.get('/', (req, res) => {
console.log('Client IP:', req.ip);
});
```
**Django:**
```python
# settings.py
USE_X_FORWARDED_HOST = True
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# For real IP detection, install django-ipware:
# pip install django-ipware
from ipware import get_client_ip
client_ip, is_routable = get_client_ip(request)
```
**Flask:**
```python
from werkzeug.middleware.proxy_fix import ProxyFix
# Apply proxy fix middleware
app.wsgi_app = ProxyFix(
app.wsgi_app,
x_for=1, # Trust 1 proxy for X-Forwarded-For
x_proto=1, # Trust X-Forwarded-Proto
x_host=1, # Trust X-Forwarded-Host
x_port=1 # Trust X-Forwarded-Port
)
# Now request.remote_addr will show the real client IP
@app.route('/')
def index():
client_ip = request.remote_addr
return f'Your IP: {client_ip}'
```
**Go (net/http):**
```go
func handler(w http.ResponseWriter, r *http.Request) {
// Read X-Real-IP header
clientIP := r.Header.Get("X-Real-IP")
if clientIP == "" {
// Fallback to X-Forwarded-For
clientIP = strings.Split(r.Header.Get("X-Forwarded-For"), ",")[0]
}
if clientIP == "" {
// Last resort: connection IP
clientIP = r.RemoteAddr
}
log.Printf("Client IP: %s", strings.TrimSpace(clientIP))
}
```
**NGINX (as backend):**
```nginx
# In your server block
real_ip_header X-Real-IP;
set_real_ip_from 172.16.0.0/12; # Charon's Docker network
real_ip_recursive on;
```
**Apache (as backend):**
```apache
# Enable mod_remoteip
<IfModule mod_remoteip.c>
RemoteIPHeader X-Real-IP
RemoteIPInternalProxy 172.16.0.0/12
</IfModule>
```
**3. Verify headers are present**
Use `curl` to check if headers are actually being sent:
```bash
# Replace with your backend's URL
curl -H "Host: yourdomain.com" http://your-backend:8080 -v 2>&1 | grep -i "x-"
```
Look for:
```
> X-Real-IP: 203.0.113.42
> X-Forwarded-Proto: https
> X-Forwarded-Host: yourdomain.com
> X-Forwarded-Port: 443
```
---
## Problem: HTTPS Redirect Loop
### Symptoms
Visiting `https://yourdomain.com` redirects endlessly, browser shows "too many redirects" error.
### Cause
Your backend application is checking the connection protocol instead of the `X-Forwarded-Proto` header.
### Solutions
**Update your redirect logic:**
**Express.js:**
```javascript
// BAD: Checks the direct connection (always http from Charon)
if (req.protocol !== 'https') {
return res.redirect('https://' + req.get('host') + req.originalUrl);
}
// GOOD: Checks X-Forwarded-Proto header
if (req.get('x-forwarded-proto') !== 'https') {
return res.redirect('https://' + req.get('host') + req.originalUrl);
}
// BEST: Use trust proxy (it checks X-Forwarded-Proto automatically)
app.set('trust proxy', true);
if (req.protocol !== 'https') {
return res.redirect('https://' + req.get('host') + req.originalUrl);
}
```
**Django:**
```python
# settings.py
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
# Now request.is_secure() will check X-Forwarded-Proto
if not request.is_secure():
return redirect('https://' + request.get_host() + request.get_full_path())
```
**Laravel:**
```php
// app/Http/Middleware/TrustProxies.php
protected $proxies = '*'; // Trust all proxies (or specify Charon's IP)
protected $headers = Request::HEADER_X_FORWARDED_ALL;
```
---
## Problem: Application Breaks After Enabling Headers
### Symptoms
- 500 Internal Server Error
- Application behaves unexpectedly
- Features stop working
### Possible Causes
1. **Strict header validation**: Your app rejects unexpected headers
2. **Conflicting logic**: App has custom IP detection that conflicts with proxy headers
3. **Security middleware**: App blocks requests with proxy headers (anti-spoofing)
### Solutions
**1. Check application logs**
Look for errors mentioning:
- X-Real-IP
- X-Forwarded-*
- Proxy headers
- IP validation
**2. Temporarily disable the feature**
- Edit the proxy host
- Uncheck **"Enable Standard Proxy Headers"**
- Save and test if the app works again
**3. Configure security middleware**
Some security frameworks block proxy headers by default:
**Helmet.js (Express):**
```javascript
// Allow proxy headers
app.use(helmet({
frameguard: false // If you need to allow iframes
}));
app.set('trust proxy', true);
```
**Django Security Middleware:**
```python
# settings.py
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
USE_X_FORWARDED_HOST = True
```
**4. Whitelist Charon's IP**
If your app has IP filtering:
```javascript
// Express example
const trustedProxies = ['172.17.0.1', '172.18.0.1'];
app.set('trust proxy', trustedProxies);
```
---
## Problem: Wrong IP in Rate Limiting
### Symptoms
All users share the same rate limit, or all requests appear to come from one IP.
### Cause
Rate limiting middleware is checking the connection IP instead of proxy headers.
### Solutions
**Express-rate-limit:**
```javascript
import rateLimit from 'express-rate-limit';
// Trust proxy first
app.set('trust proxy', true);
// Then apply rate limit
const limiter = rateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // 100 requests per window
standardHeaders: true,
legacyHeaders: false,
// Rate limit will use req.ip (which uses X-Real-IP when trust proxy is set)
});
app.use(limiter);
```
**Custom middleware:**
```javascript
function getRealIP(req) {
return req.headers['x-real-ip'] ||
req.headers['x-forwarded-for']?.split(',')[0] ||
req.ip;
}
```
---
## Problem: GeoIP Location is Wrong
### Symptoms
Users from the US are detected as being from your server's location.
### Cause
GeoIP lookup is using Charon's IP instead of the client IP.
### Solutions
**Ensure proxy headers are enabled** in Charon, then:
**MaxMind GeoIP2 (Node.js):**
```javascript
import maxmind from 'maxmind';
const lookup = await maxmind.open('/path/to/GeoLite2-City.mmdb');
function getLocation(req) {
const clientIP = req.headers['x-real-ip'] || req.ip;
return lookup.get(clientIP);
}
```
**Python geoip2:**
```python
import geoip2.database
reader = geoip2.database.Reader('/path/to/GeoLite2-City.mmdb')
def get_location(request):
client_ip = request.META.get('HTTP_X_REAL_IP') or request.META.get('REMOTE_ADDR')
return reader.city(client_ip)
```
---
## Problem: Logs Show Multiple IPs in X-Forwarded-For
### Symptoms
X-Forwarded-For shows: `203.0.113.42, 172.17.0.1`
### Explanation
This is **correct behavior**. X-Forwarded-For is a comma-separated list:
1. First IP = Real client (`203.0.113.42`)
2. Second IP = Charon's IP (`172.17.0.1`)
### Get the Real Client IP
**Always read the FIRST IP** in X-Forwarded-For:
```javascript
const forwardedFor = req.headers['x-forwarded-for'];
const clientIP = forwardedFor ? forwardedFor.split(',')[0].trim() : req.ip;
```
**Or use X-Real-IP** (simpler):
```javascript
const clientIP = req.headers['x-real-ip'] || req.ip;
```
---
## Testing Headers Locally
**1. Check if headers reach your backend:**
Add temporary logging:
```javascript
// Express
app.use((req, res, next) => {
console.log('Headers:', req.headers);
next();
});
```
```python
# Django middleware
def log_headers(get_response):
def middleware(request):
print('Headers:', request.META)
return get_response(request)
return middleware
```
**2. Simulate client request with curl:**
```bash
# Test from outside Charon
curl -H "Host: yourdomain.com" https://yourdomain.com/test
# Check your backend logs for:
# X-Real-IP: <your actual IP>
# X-Forwarded-Proto: https
# X-Forwarded-Host: yourdomain.com
```
---
## Security Considerations
### IP Spoofing Prevention
Charon configures Caddy with `trusted_proxies` to prevent clients from spoofing headers.
**What this means:**
- Clients CANNOT inject fake X-Real-IP headers
- Caddy overwrites any client-provided proxy headers
- Only Charon's headers are trusted
**Backend security:**
Your backend should still:
1. Only trust proxy headers from Charon's IP
2. Validate IP addresses before using them for access control
3. Use a proper IP parsing library (not regex)
### Example: Trust Proxy Configuration
```javascript
// Express: Only trust specific IPs
app.set('trust proxy', ['127.0.0.1', '172.17.0.0/16']);
// Nginx: Specify allowed proxy IPs
real_ip_header X-Real-IP;
set_real_ip_from 172.17.0.0/16; # Docker network
set_real_ip_from 10.0.0.0/8; # Internal network
```
---
## When to Contact Support
If you've tried the above solutions and:
- ✅ Standard headers are enabled in Charon
- ✅ Backend is configured to trust proxies
- ✅ Headers are visible in logs but still not working
- ✅ No redirect loops or errors
**[Open an issue](https://github.com/Wikid82/charon/issues)** with:
1. Backend framework and version (e.g., Express 4.18.2)
2. Charon version (from Dashboard)
3. Proxy host configuration (screenshot or JSON)
4. Sample backend logs showing the headers
5. Expected vs actual behavior
---
## Additional Resources
- [Features Guide: Standard Proxy Headers](../features.md#-standard-proxy-headers)
- [Getting Started: Adding Your First Website](../getting-started.md#step-2-add-your-first-website)
- [API Documentation: Proxy Hosts](../api.md#proxy-hosts)
- [RFC 7239: Forwarded HTTP Extension](https://tools.ietf.org/html/rfc7239)

100
docs/troubleshooting/react-production-errors.md
View File

@@ -1,100 +0,0 @@
# Troubleshooting: React Production Build Errors
## "Cannot set properties of undefined" Error
If you encounter this error when running Charon in production (typically appearing as "Cannot set properties of undefined (setting 'root')" in the browser console), this is usually caused by stale browser cache or outdated Docker images.
### Quick Fixes
#### 1. Hard Refresh Browser
Clear the browser cache and force a full reload:
- **Chrome/Edge:** `Ctrl+Shift+R` (Windows/Linux) or `Cmd+Shift+R` (Mac)
- **Firefox:** `Ctrl+F5` (Windows/Linux) or `Cmd+Shift+R` (Mac)
- **Safari:** `Cmd+Option+R` (Mac)
#### 2. Clear Browser Cache
Open Browser DevTools and clear all site data:
1. Open DevTools (F12 or right-click → Inspect)
2. Navigate to **Application** tab (Chrome/Edge) or **Storage** tab (Firefox)
3. Click **Clear Site Data** or **Clear All**
4. Reload the page
#### 3. Rebuild Docker Image
If the error persists after clearing browser cache, your Docker image may be outdated:
```bash
# Stop and remove the current container
docker compose -f .docker/compose/docker-compose.yml down
# Rebuild with no cache
docker compose -f .docker/compose/docker-compose.yml up -d --build --no-cache
```
#### 4. Verify Docker Image Tag
Check that you're running the latest version:
```bash
docker images charon/app --format "{{.Tag}}" | head -1
```
Compare with the latest release at: <https://github.com/Wikid82/charon/releases>
### Root Cause
This error typically occurs when:
1. **Browser cached old JavaScript files** that are incompatible with the new HTML template
2. **Docker image wasn't rebuilt** after updating dependencies
3. **CDN or proxy cache** is serving stale assets
React 19.2.3 and lucide-react@0.562.0 are fully compatible and tested. The issue is almost always environment-related, not a code bug.
### Still Having Issues?
If the error persists after trying all fixes above, please report the issue with:
- **Browser console screenshot** (DevTools → Console tab, screenshot the full error)
- **Browser name and version** (e.g., Chrome 120.0.6099.109)
- **Docker image tag** (from `docker images` command)
- **Any browser extensions enabled** (especially ad blockers or privacy tools)
- **Steps to reproduce** (what page you visited, what you clicked)
Open an issue at: <https://github.com/Wikid82/charon/issues>
### Prevention
To avoid this issue in the future:
1. **Always rebuild Docker images** when upgrading Charon:
```bash
docker compose pull
docker compose up -d --build
```
2. **Clear browser cache** after major updates
3. **Use versioned Docker tags** instead of `:latest` to avoid unexpected updates:
```yaml
image: ghcr.io/wikid82/charon:v0.4.0
```
### Technical Background
React's production build uses optimized bundle splitting and code minification. When the browser caches old JavaScript chunks but receives a new HTML template, the chunks may reference objects that don't exist in the new bundle structure, causing "Cannot set properties of undefined" errors.
**Verified Compatible Versions:**
- React: 19.2.3
- React DOM: 19.2.3
- lucide-react: 0.562.0
- Vite: 7.3.0
All 1403 unit tests pass, and production builds succeed without errors. See [diagnostic report](../implementation/react-19-lucide-error-DIAGNOSTIC-REPORT.md) for full test results.

385
docs/troubleshooting/websocket.md
View File

@@ -1,385 +0,0 @@
---
title: Troubleshooting WebSocket Issues
description: Resolve WebSocket connection problems in Charon. Proxy configuration, timeouts, and connection stability.
---
## Troubleshooting WebSocket Issues
WebSocket connections are used in Charon for real-time features like live log streaming. If you're experiencing issues with WebSocket connections (e.g., logs not updating in real-time), this guide will help you diagnose and resolve the problem.
### Quick Diagnostics
### Check WebSocket Connection Status
1. Go to **System Settings** in the Charon UI
2. Scroll to the **WebSocket Connections** card
3. Check if there are active connections displayed
The WebSocket status card shows:
- Total number of active WebSocket connections
- Breakdown by type (General Logs vs Security Logs)
- Oldest connection age
- Detailed connection info (when expanded)
### Browser Console Check
Open your browser's Developer Tools (F12) and check the Console tab for:
- WebSocket connection errors
- Connection refused messages
- Authentication failures
- CORS errors
## Common Issues and Solutions
### 1. Proxy/Load Balancer Configuration
**Symptom:** WebSocket connections fail to establish or disconnect immediately.
**Cause:** If running Charon behind a reverse proxy (Nginx, Apache, HAProxy, or load balancer), the proxy might be terminating WebSocket connections or not forwarding the upgrade request properly.
**Solution:**
#### Nginx Configuration
```nginx
location /api/v1/logs/live {
proxy_pass http://charon:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Increase timeouts for long-lived connections
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
location /api/v1/cerberus/logs/ws {
proxy_pass http://charon:8080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Increase timeouts for long-lived connections
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
```
Key requirements:
- `proxy_http_version 1.1` — Required for WebSocket support
- `Upgrade` and `Connection` headers — Required for WebSocket upgrade
- Long `proxy_read_timeout` — Prevents connection from timing out
#### Apache Configuration
```apache
<VirtualHost *:443>
ServerName charon.example.com
# Enable WebSocket proxy
ProxyRequests Off
ProxyPreserveHost On
# WebSocket endpoints
ProxyPass /api/v1/logs/live ws://localhost:8080/api/v1/logs/live retry=0 timeout=3600
ProxyPassReverse /api/v1/logs/live ws://localhost:8080/api/v1/logs/live
ProxyPass /api/v1/cerberus/logs/ws ws://localhost:8080/api/v1/cerberus/logs/ws retry=0 timeout=3600
ProxyPassReverse /api/v1/cerberus/logs/ws ws://localhost:8080/api/v1/cerberus/logs/ws
# Regular HTTP endpoints
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
</VirtualHost>
```
Required modules:
```bash
a2enmod proxy proxy_http proxy_wstunnel
```
### 2. Network Timeouts
**Symptom:** WebSocket connections work initially but disconnect after some idle time.
**Cause:** Intermediate network infrastructure (firewalls, load balancers, NAT devices) may have idle timeout settings shorter than the WebSocket keepalive interval.
**Solution:**
Charon sends WebSocket ping frames every 30 seconds to keep connections alive. If you're still experiencing timeouts:
1. **Check proxy timeout settings** (see above)
2. **Check firewall idle timeout:**
```bash
# Linux iptables
iptables -L -v -n | grep ESTABLISHED
# If timeout is too short, increase it:
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
echo 3600 > /proc/sys/net/netfilter/nf_conntrack_tcp_timeout_established
```
3. **Check load balancer settings:**
- AWS ALB/ELB: Set idle timeout to 3600 seconds
- GCP Load Balancer: Set timeout to 1 hour
- Azure Load Balancer: Set idle timeout to maximum
### 3. HTTPS Certificate Errors (Docker)
**Symptom:** WebSocket connections fail with TLS/certificate errors, especially in Docker environments.
**Cause:** Missing CA certificates in the Docker container, or self-signed certificates not trusted by the browser.
**Solution:**
#### Install CA Certificates (Docker)
Add to your Dockerfile:
```dockerfile
RUN apt-get update && apt-get install -y ca-certificates && update-ca-certificates
```
Or for existing containers:
```bash
docker exec -it charon apt-get update && apt-get install -y ca-certificates
```
#### For Self-Signed Certificates (Development Only)
**Warning:** This compromises security. Only use in development environments.
Set environment variable:
```bash
docker run -e FF_IGNORE_CERT_ERRORS=1 charon:latest
```
Or in docker-compose.yml:
```yaml
services:
charon:
environment:
- FF_IGNORE_CERT_ERRORS=1
```
#### Better Solution: Use Valid Certificates
1. Use Let's Encrypt (free, automated)
2. Use a trusted CA certificate
3. Import your self-signed cert into the browser's trust store
### 4. Firewall Settings
**Symptom:** WebSocket connections fail or time out.
**Cause:** Firewall blocking WebSocket traffic on ports 80/443.
**Solution:**
#### Linux (iptables)
Allow WebSocket traffic:
```bash
# Allow HTTP/HTTPS
iptables -A INPUT -p tcp --dport 80 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -j ACCEPT
# Allow established connections (for WebSocket)
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Save rules
iptables-save > /etc/iptables/rules.v4
```
#### Docker
Ensure ports are exposed:
```yaml
services:
charon:
ports:
- "8080:8080"
- "443:443"
```
#### Cloud Providers
- **AWS:** Add inbound rules to Security Group for ports 80/443
- **GCP:** Add firewall rules for ports 80/443
- **Azure:** Add Network Security Group rules for ports 80/443
### 5. Connection Stability / Packet Loss
**Symptom:** Frequent WebSocket disconnections and reconnections.
**Cause:** Unstable network with packet loss prevents WebSocket connections from staying open.
**Solution:**
#### Check Network Stability
```bash
# Ping test
ping -c 100 charon.example.com
# Check packet loss (should be < 1%)
mtr charon.example.com
```
#### Enable Connection Retry (Client-Side)
The Charon frontend automatically handles reconnection for security logs but not general logs. If you need more robust reconnection:
1. Monitor the WebSocket status in System Settings
2. Refresh the page if connections are frequently dropping
3. Consider using a more stable network connection
4. Check if VPN or proxy is causing issues
### 6. Browser Compatibility
**Symptom:** WebSocket connections don't work in certain browsers.
**Cause:** Very old browsers don't support WebSocket protocol.
**Supported Browsers:**
- Chrome 16+ ✅
- Firefox 11+ ✅
- Safari 7+ ✅
- Edge (all versions) ✅
- IE 10+ ⚠️ (deprecated, use Edge)
**Solution:** Update to a modern browser.
### 7. CORS Issues
**Symptom:** Browser console shows CORS errors with WebSocket connections.
**Cause:** Cross-origin WebSocket connection blocked by browser security policy.
**Solution:**
WebSocket connections should be same-origin (from the same domain as the Charon UI). If you're accessing Charon from a different domain:
1. **Preferred:** Access Charon UI from the same domain
2. **Alternative:** Configure CORS in Charon (if supported)
3. **Development Only:** Use browser extension to disable CORS (NOT for production)
### 8. Authentication Issues
**Symptom:** WebSocket connection fails with 401 Unauthorized.
**Cause:** Authentication token not being sent with WebSocket connection.
**Solution:**
Charon WebSocket endpoints support three authentication methods:
1. **HttpOnly Cookie** (automatic) — Used by default when accessing UI from browser
2. **Query Parameter** — `?token=<your-token>`
3. **Authorization Header** — Not supported for browser WebSocket connections
If you're accessing WebSocket from a script or tool:
```javascript
const ws = new WebSocket('wss://charon.example.com/api/v1/logs/live?token=YOUR_TOKEN');
```
## Monitoring WebSocket Connections
### Using the System Settings UI
1. Navigate to **System Settings** in Charon
2. View the **WebSocket Connections** card
3. Expand details to see:
- Connection ID
- Connection type (General/Security)
- Remote address
- Active filters
- Connection duration
### Using the API
Check WebSocket statistics programmatically:
```bash
# Get connection statistics
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://charon.example.com/api/v1/websocket/stats
# Get detailed connection list
curl -H "Authorization: Bearer YOUR_TOKEN" \
https://charon.example.com/api/v1/websocket/connections
```
Response example:
```json
{
"total_active": 2,
"logs_connections": 1,
"cerberus_connections": 1,
"oldest_connection": "2024-01-15T10:30:00Z",
"last_updated": "2024-01-15T11:00:00Z"
}
```
### Using Browser DevTools
1. Open DevTools (F12)
2. Go to **Network** tab
3. Filter by **WS** (WebSocket)
4. Look for connections to:
- `/api/v1/logs/live`
- `/api/v1/cerberus/logs/ws`
Check:
- Status should be `101 Switching Protocols`
- Messages tab shows incoming log entries
- No errors in Frames tab
## Still Having Issues?
If none of the above solutions work:
1. **Check Charon logs:**
```bash
docker logs charon | grep -i websocket
```
2. **Enable debug logging** (if available)
3. **Report an issue on GitHub:**
- [Charon Issues](https://github.com/Wikid82/charon/issues)
- Include:
- Charon version
- Browser and version
- Proxy/load balancer configuration
- Error messages from browser console
- Charon server logs
## See Also
- [Live Logs Guide](../live-logs-guide.md)
- [Security Documentation](../security.md)
- [API Documentation](../api.md)