Migrate analytics from SQLite to ClickHouse
SQLite was too slow for analytical aggregations on traffic_events and waf_events (millions of rows, GROUP BY, COUNT DISTINCT). ClickHouse is a columnar OLAP database purpose-built for this workload. - Add ClickHouse container to Docker Compose with health check - Create src/lib/clickhouse/client.ts with singleton client, table DDL, insert helpers, and all analytics query functions - Update log-parser.ts and waf-log-parser.ts to write to ClickHouse - Remove purgeOldEntries — ClickHouse TTL handles 90-day retention - Rewrite analytics-db.ts and waf-events.ts to query ClickHouse - Remove trafficEvents/wafEvents from SQLite schema, add migration - CLICKHOUSE_PASSWORD is required (no hardcoded default) - Update .env.example, README, and test infrastructure API response shapes are unchanged — no frontend modifications needed. Parse state (file offsets) remains in SQLite. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Generated
+19
@@ -8,6 +8,7 @@
|
||||
"name": "caddy-proxy-manager",
|
||||
"version": "1.0.0",
|
||||
"dependencies": {
|
||||
"@clickhouse/client": "^1.18.2",
|
||||
"@radix-ui/react-accordion": "^1.2.12",
|
||||
"@radix-ui/react-avatar": "^1.1.11",
|
||||
"@radix-ui/react-checkbox": "^1.3.3",
|
||||
@@ -579,6 +580,24 @@
|
||||
"node": ">=6.9.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@clickhouse/client": {
|
||||
"version": "1.18.2",
|
||||
"resolved": "https://registry.npmjs.org/@clickhouse/client/-/client-1.18.2.tgz",
|
||||
"integrity": "sha512-fuquQswRSHWM6D079ZeuGqkMOsqtcUPL06UdTnowmoeeYjVrqisfVmvnw8pc3OeKS4kVb91oygb/MfLDiMs0TQ==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@clickhouse/client-common": "1.18.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=16"
|
||||
}
|
||||
},
|
||||
"node_modules/@clickhouse/client-common": {
|
||||
"version": "1.18.2",
|
||||
"resolved": "https://registry.npmjs.org/@clickhouse/client-common/-/client-common-1.18.2.tgz",
|
||||
"integrity": "sha512-J0SG6q9V31ydxonglpj9xhNRsUxCsF71iEZ784yldqMYwsHixj/9xHFDgBDX3DuMiDx/kPDfXnf+pimp08wIBA==",
|
||||
"license": "Apache-2.0"
|
||||
},
|
||||
"node_modules/@date-fns/tz": {
|
||||
"version": "1.4.1",
|
||||
"resolved": "https://registry.npmjs.org/@date-fns/tz/-/tz-1.4.1.tgz",
|
||||
|
||||
Reference in New Issue
Block a user