diff --git a/.agent/rules/.instructions.md b/.agent/rules/.instructions.md new file mode 100644 index 00000000..d8d132d8 --- /dev/null +++ b/.agent/rules/.instructions.md @@ -0,0 +1,67 @@ +--- +trigger: always_on +--- + +# Charon Instructions + +## Code Quality Guidelines +Every session should improve the codebase, not just add to it. Actively refactor code you encounter, even outside of your immediate task scope. Think about long-term maintainability and consistency. Make a detailed plan before writing code. Always create unit tests for new code coverage. + +- **DRY**: Consolidate duplicate patterns into reusable functions, types, or components after the second occurrence. +- **CLEAN**: Delete dead code immediately. Remove unused imports, variables, functions, types, commented code, and console logs. +- **LEVERAGE**: Use battle-tested packages over custom implementations. +- **READABLE**: Maintain comments and clear naming for complex logic. Favor clarity over cleverness. +- **CONVENTIONAL COMMITS**: Write commit messages using `feat:`, `fix:`, `chore:`, `refactor:`, or `docs:` prefixes. + +## ๐Ÿšจ CRITICAL ARCHITECTURE RULES ๐Ÿšจ +- **Single Frontend Source**: All frontend code MUST reside in `frontend/`. NEVER create `backend/frontend/` or any other nested frontend directory. +- **Single Backend Source**: All backend code MUST reside in `backend/`. +- **No Python**: This is a Go (Backend) + React/TypeScript (Frontend) project. Do not introduce Python scripts or requirements. + +## Big Picture +- Charon is a self-hosted web app for managing reverse proxy host configurations with the novice user in mind. Everything should prioritize simplicity, usability, reliability, and security, all rolled into one simple binary + static assets deployment. No external dependencies. +- Users should feel like they have enterprise-level security and features with zero effort. +- `backend/cmd/api` loads config, opens SQLite, then hands off to `internal/server`. +- `internal/config` respects `CHARON_ENV`, `CHARON_HTTP_PORT`, `CHARON_DB_PATH` and creates the `data/` directory. +- `internal/server` mounts the built React app (via `attachFrontend`) whenever `frontend/dist` exists. +- Persistent types live in `internal/models`; GORM auto-migrates them. + +## Backend Workflow +- **Run**: `cd backend && go run ./cmd/api`. +- **Test**: `go test ./...`. +- **API Response**: Handlers return structured errors using `gin.H{"error": "message"}`. +- **JSON Tags**: All struct fields exposed to the frontend MUST have explicit `json:"snake_case"` tags. +- **IDs**: UUIDs (`github.com/google/uuid`) are generated server-side; clients never send numeric IDs. +- **Security**: Sanitize all file paths using `filepath.Clean`. Use `fmt.Errorf("context: %w", err)` for error wrapping. +- **Graceful Shutdown**: Long-running work must respect `server.Run(ctx)`. + +## Frontend Workflow +- **Location**: Always work within `frontend/`. +- **Stack**: React 18 + Vite + TypeScript + TanStack Query (React Query). +- **State Management**: Use `src/hooks/use*.ts` wrapping React Query. +- **API Layer**: Create typed API clients in `src/api/*.ts` that wrap `client.ts`. +- **Forms**: Use local `useState` for form fields, submit via `useMutation`, then `invalidateQueries` on success. + +## Cross-Cutting Notes +- **VS Code Integration**: If you introduce new repetitive CLI actions (e.g., scans, builds, scripts), register them in .vscode/tasks.json to allow for easy manual verification. +- **Sync**: React Query expects the exact JSON produced by GORM tags (snake_case). Keep API and UI field names aligned. +- **Migrations**: When adding models, update `internal/models` AND `internal/api/routes/routes.go` (AutoMigrate). +- **Testing**: All new code MUST include accompanying unit tests. +- **Ignore Files**: Always check `.gitignore`, `.dockerignore`, and `.codecov.yml` when adding new file or folders. + +## Documentation +- **Features**: Update `docs/features.md` when adding capabilities. +- **Links**: Use GitHub Pages URLs (`https://wikid82.github.io/charon/`) for docs and GitHub blob links for repo files. + +## CI/CD & Commit Conventions +- **Triggers**: Use `feat:`, `fix:`, or `perf:` to trigger Docker builds. `chore:` skips builds. +- **Beta**: `feature/beta-release` always builds. + +## โœ… Task Completion Protocol (Definition of Done) +Before marking an implementation task as complete, perform the following: +1. **Pre-Commit Triage**: Run `pre-commit run --all-files`. + - If errors occur, **fix them immediately**. + - If logic errors occur, analyze and propose a fix. + - Do not output code that violates pre-commit standards. +2. **Verify Build**: Ensure the backend compiles and the frontend builds without errors. +3. **Clean Up**: Ensure no debug print statements or commented-out blocks remain. diff --git a/.codecov.yml b/.codecov.yml new file mode 100644 index 00000000..a6458e44 --- /dev/null +++ b/.codecov.yml @@ -0,0 +1,91 @@ +# ============================================================================= +# Codecov Configuration +# Require 75% overall coverage, exclude test files and non-source code +# ============================================================================= + +coverage: + status: + project: + default: + target: 75% + threshold: 0% + +# Fail CI if Codecov upload/report indicates a problem +require_ci_to_pass: yes + +# ----------------------------------------------------------------------------- +# Exclude from coverage reporting +# ----------------------------------------------------------------------------- +ignore: + # Test files + - "**/tests/**" + - "**/test/**" + - "**/__tests__/**" + - "**/test_*.go" + - "**/*_test.go" + - "**/*.test.ts" + - "**/*.test.tsx" + - "**/*.spec.ts" + - "**/*.spec.tsx" + - "**/vitest.config.ts" + - "**/vitest.setup.ts" + + # E2E tests + - "**/e2e/**" + - "**/integration/**" + + # Documentation + - "docs/**" + - "*.md" + + # CI/CD & Config + - ".github/**" + - "scripts/**" + - "tools/**" + - "*.yml" + - "*.yaml" + - "*.json" + + # Frontend build artifacts & dependencies + - "frontend/node_modules/**" + - "frontend/dist/**" + - "frontend/coverage/**" + - "frontend/test-results/**" + - "frontend/public/**" + + # Backend non-source files + - "backend/cmd/seed/**" + - "backend/cmd/api/**" + - "backend/data/**" + - "backend/coverage/**" + - "backend/bin/**" + - "backend/*.cover" + - "backend/*.out" + - "backend/*.html" + - "backend/codeql-db/**" + + # Docker-only code (not testable in CI) + - "backend/internal/services/docker_service.go" + - "backend/internal/api/handlers/docker_handler.go" + + # CodeQL artifacts + - "codeql-db/**" + - "codeql-db-*/**" + - "codeql-agent-results/**" + - "codeql-custom-queries-*/**" + - "*.sarif" + + # Config files (no logic) + - "**/tailwind.config.js" + - "**/postcss.config.js" + - "**/eslint.config.js" + - "**/vite.config.ts" + - "**/tsconfig*.json" + + # Type definitions only + - "**/*.d.ts" + + # Import/data directories + - "import/**" + - "data/**" + - ".cache/**" diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 00000000..8de6d2f0 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,200 @@ +# ============================================================================= +# .dockerignore - Exclude files from Docker build context +# Keep this file in sync with .gitignore where applicable +# ============================================================================= + +# ----------------------------------------------------------------------------- +# Version Control & CI/CD +# ----------------------------------------------------------------------------- +.git/ +.gitignore +.github/ +.pre-commit-config.yaml +.codecov.yml +.goreleaser.yaml +.sourcery.yml + +# ----------------------------------------------------------------------------- +# Python (pre-commit, tooling) +# ----------------------------------------------------------------------------- +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +.venv/ +venv/ +env/ +ENV/ +.pytest_cache/ +.coverage +.hypothesis/ +htmlcov/ +*.egg-info/ + +# ----------------------------------------------------------------------------- +# Node/Frontend - Build in Docker, not from host +# ----------------------------------------------------------------------------- +frontend/node_modules/ +frontend/coverage/ +frontend/test-results/ +frontend/dist/ +frontend/.vite/ +frontend/*.tsbuildinfo +frontend/frontend/ +frontend/e2e/ + +# Root-level node artifacts (eslint config runner) +node_modules/ +package-lock.json +package.json + +# ----------------------------------------------------------------------------- +# Go/Backend - Build artifacts & coverage +# ----------------------------------------------------------------------------- +backend/bin/ +backend/api +backend/*.out +backend/*.cover +backend/*.html +backend/coverage/ +backend/coverage*.out +backend/coverage*.txt +backend/*.coverage.out +backend/handler_coverage.txt +backend/handlers.out +backend/services.test +backend/test-output.txt +backend/tr_no_cover.txt +backend/nohup.out +backend/package.json +backend/package-lock.json + +# Backend data (created at runtime) +backend/data/ +backend/codeql-db/ +backend/.venv/ +backend/.vscode/ + +# ----------------------------------------------------------------------------- +# Databases (created at runtime) +# ----------------------------------------------------------------------------- +*.db +*.sqlite +*.sqlite3 +data/ +charon.db +cpm.db + +# ----------------------------------------------------------------------------- +# IDE & Editor +# ----------------------------------------------------------------------------- +.vscode/ +.vscode.backup*/ +.idea/ +*.swp +*.swo +*~ +*.xcf +Chiron.code-workspace + +# ----------------------------------------------------------------------------- +# Logs & Temp Files +# ----------------------------------------------------------------------------- +.trivy_logs/ +*.log +logs/ +nohup.out + +# ----------------------------------------------------------------------------- +# Environment Files +# ----------------------------------------------------------------------------- +.env +.env.local +.env.*.local +!.env.example + +# ----------------------------------------------------------------------------- +# OS Files +# ----------------------------------------------------------------------------- +.DS_Store +Thumbs.db + +# ----------------------------------------------------------------------------- +# Documentation (not needed in image) +# ----------------------------------------------------------------------------- +docs/ +*.md +!README.md +!CONTRIBUTING.md +!LICENSE + +# ----------------------------------------------------------------------------- +# Docker Compose (not needed inside image) +# ----------------------------------------------------------------------------- +docker-compose*.yml +**/Dockerfile.* + +# ----------------------------------------------------------------------------- +# GoReleaser & dist artifacts +# ----------------------------------------------------------------------------- +dist/ + +# ----------------------------------------------------------------------------- +# Scripts & Tools (not needed in image) +# ----------------------------------------------------------------------------- +scripts/ +tools/ +create_issues.sh +cookies.txt +cookies.txt.bak +test.caddyfile +Makefile + +# ----------------------------------------------------------------------------- +# Testing & Coverage Artifacts +# ----------------------------------------------------------------------------- +coverage/ +coverage.out +*.cover +*.crdownload +*.sarif + +# ----------------------------------------------------------------------------- +# CodeQL & Security Scanning (large, not needed) +# ----------------------------------------------------------------------------- +codeql-db/ +codeql-db-*/ +codeql-agent-results/ +codeql-custom-queries-*/ +codeql-*.sarif +codeql-results*.sarif +.codeql/ + +# ----------------------------------------------------------------------------- +# Import Directory (user data) +# ----------------------------------------------------------------------------- +import/ + +# ----------------------------------------------------------------------------- +# Project Documentation & Planning (not needed in image) +# ----------------------------------------------------------------------------- +*.md.bak +ACME_STAGING_IMPLEMENTATION.md* +ARCHITECTURE_PLAN.md +BULK_ACL_FEATURE.md +DOCKER_TASKS.md* +DOCUMENTATION_POLISH_SUMMARY.md +GHCR_MIGRATION_SUMMARY.md +ISSUE_*_IMPLEMENTATION.md* +PHASE_*_SUMMARY.md +PROJECT_BOARD_SETUP.md +PROJECT_PLANNING.md +SECURITY_IMPLEMENTATION_PLAN.md +VERSIONING_IMPLEMENTATION.md +QA_AUDIT_REPORT*.md +VERSION.md +eslint.config.js +go.work +go.work.sum +.cache diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..725fefd5 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,16 @@ +# .gitattributes - LFS filter and binary markers for large files and DBs + +# Mark CodeQL DB directories as binary +codeql-db/** binary +codeql-db-*/** binary + +# Use Git LFS for larger binary database files and archives +*.db filter=lfs diff=lfs merge=lfs -text +*.sqlite filter=lfs diff=lfs merge=lfs -text +*.sqlite3 filter=lfs diff=lfs merge=lfs -text +*.tar.gz filter=lfs diff=lfs merge=lfs -text +*.tgz filter=lfs diff=lfs merge=lfs -text +*.zip filter=lfs diff=lfs merge=lfs -text +*.iso filter=lfs diff=lfs merge=lfs -text +*.exe filter=lfs diff=lfs merge=lfs -text +*.dll filter=lfs diff=lfs merge=lfs -text diff --git a/.github/FUNDING.yml b/.github/FUNDING.yml new file mode 100644 index 00000000..28e6f071 --- /dev/null +++ b/.github/FUNDING.yml @@ -0,0 +1,14 @@ +# These are supported funding model platforms +github: Wikid82 +# patreon: # Replace with a single Patreon username +# open_collective: # Replace with a single Open Collective username +# ko_fi: # Replace with a single Ko-fi username +# tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel +# community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry +# liberapay: # Replace with a single Liberapay username +# issuehunt: # Replace with a single IssueHunt username +# lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry +# polar: # Replace with a single Polar username +buy_me_a_coffee: Wikid82 +# thanks_dev: # Replace with a single thanks.dev username +# custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2'] diff --git a/.github/ISSUE_TEMPLATE/alpha-feature.yml b/.github/ISSUE_TEMPLATE/alpha-feature.yml new file mode 100644 index 00000000..51d0cc0d --- /dev/null +++ b/.github/ISSUE_TEMPLATE/alpha-feature.yml @@ -0,0 +1,93 @@ +name: ๐Ÿ—๏ธ Alpha Feature +description: Create an issue for an Alpha milestone feature +title: "[ALPHA] " +labels: ["alpha", "feature"] +body: + - type: markdown + attributes: + value: | + ## Alpha Milestone Feature + Features that are part of the core foundation and initial release. + + - type: dropdown + id: priority + attributes: + label: Priority + description: How critical is this feature? + options: + - Critical (Blocking, must-have) + - High (Important, should have) + - Medium (Nice to have) + - Low (Future enhancement) + validations: + required: true + + - type: input + id: issue_number + attributes: + label: Planning Issue Number + description: Reference number from PROJECT_PLANNING.md (e.g., Issue #5) + placeholder: "Issue #" + validations: + required: false + + - type: textarea + id: description + attributes: + label: Feature Description + description: What should this feature do? + placeholder: Describe the feature in detail + validations: + required: true + + - type: textarea + id: tasks + attributes: + label: Implementation Tasks + description: List of tasks to complete this feature + placeholder: | + - [ ] Task 1 + - [ ] Task 2 + - [ ] Task 3 + value: | + - [ ] + validations: + required: true + + - type: textarea + id: acceptance + attributes: + label: Acceptance Criteria + description: How do we know this feature is complete? + placeholder: | + - [ ] Criteria 1 + - [ ] Criteria 2 + value: | + - [ ] + validations: + required: true + + - type: checkboxes + id: categories + attributes: + label: Categories + description: Select all that apply + options: + - label: Backend + - label: Frontend + - label: Database + - label: Caddy Integration + - label: Security + - label: SSL/TLS + - label: UI/UX + - label: Deployment + - label: Documentation + + - type: textarea + id: technical_notes + attributes: + label: Technical Notes + description: Any technical considerations or dependencies? + placeholder: Libraries, APIs, or other issues that need to be completed first + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/beta-monitoring-feature.yml b/.github/ISSUE_TEMPLATE/beta-monitoring-feature.yml new file mode 100644 index 00000000..b1965956 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/beta-monitoring-feature.yml @@ -0,0 +1,118 @@ +name: ๐Ÿ“Š Beta Monitoring Feature +description: Create an issue for a Beta milestone monitoring/logging feature +title: "[BETA] [MONITORING] " +labels: ["beta", "feature", "monitoring"] +body: + - type: markdown + attributes: + value: | + ## Beta Monitoring & Logging Feature + Features related to observability, logging, and system monitoring. + + - type: dropdown + id: priority + attributes: + label: Priority + description: How critical is this monitoring feature? + options: + - Critical (Essential for operations) + - High (Important visibility) + - Medium (Enhanced monitoring) + - Low (Nice-to-have metrics) + validations: + required: true + + - type: dropdown + id: monitoring_type + attributes: + label: Monitoring Type + description: What aspect of monitoring? + options: + - Dashboards & Statistics + - Log Viewing & Search + - Alerting & Notifications + - CrowdSec Dashboard + - Analytics Integration + - Health Checks + - Performance Metrics + validations: + required: true + + - type: input + id: issue_number + attributes: + label: Planning Issue Number + description: Reference number from PROJECT_PLANNING.md (e.g., Issue #23) + placeholder: "Issue #" + validations: + required: false + + - type: textarea + id: description + attributes: + label: Feature Description + description: What monitoring/logging capability should this provide? + placeholder: Describe what users will be able to see or do + validations: + required: true + + - type: textarea + id: metrics + attributes: + label: Metrics & Data Points + description: What data will be collected and displayed? + placeholder: | + - Metric 1: Description + - Metric 2: Description + validations: + required: false + + - type: textarea + id: tasks + attributes: + label: Implementation Tasks + description: List of tasks to complete this feature + placeholder: | + - [ ] Task 1 + - [ ] Task 2 + - [ ] Task 3 + value: | + - [ ] + validations: + required: true + + - type: textarea + id: acceptance + attributes: + label: Acceptance Criteria + description: How do we verify this monitoring feature works? + placeholder: | + - [ ] Data displays correctly + - [ ] Updates in real-time + - [ ] Performance is acceptable + value: | + - [ ] + validations: + required: true + + - type: checkboxes + id: categories + attributes: + label: Implementation Areas + description: Select all that apply + options: + - label: Backend (Data collection) + - label: Frontend (UI/Charts) + - label: Database (Storage) + - label: Real-time Updates (WebSocket) + - label: External Integration (GoAccess, CrowdSec) + - label: Documentation Required + + - type: textarea + id: ui_design + attributes: + label: UI/UX Considerations + description: Describe the user interface requirements + placeholder: Layout, charts, filters, export options, etc. + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/beta-security-feature.yml b/.github/ISSUE_TEMPLATE/beta-security-feature.yml new file mode 100644 index 00000000..d28c9d0d --- /dev/null +++ b/.github/ISSUE_TEMPLATE/beta-security-feature.yml @@ -0,0 +1,116 @@ +name: ๐Ÿ” Beta Security Feature +description: Create an issue for a Beta milestone security feature +title: "[BETA] [SECURITY] " +labels: ["beta", "feature", "security"] +body: + - type: markdown + attributes: + value: | + ## Beta Security Feature + Advanced security features for the beta release. + + - type: dropdown + id: priority + attributes: + label: Priority + description: How critical is this security feature? + options: + - Critical (Essential security control) + - High (Important protection) + - Medium (Additional hardening) + - Low (Nice-to-have security enhancement) + validations: + required: true + + - type: dropdown + id: security_category + attributes: + label: Security Category + description: What type of security feature is this? + options: + - Authentication & Access Control + - Threat Protection + - SSL/TLS Management + - Monitoring & Logging + - Web Application Firewall + - Rate Limiting + - IP Access Control + validations: + required: true + + - type: input + id: issue_number + attributes: + label: Planning Issue Number + description: Reference number from PROJECT_PLANNING.md (e.g., Issue #15) + placeholder: "Issue #" + validations: + required: false + + - type: textarea + id: description + attributes: + label: Feature Description + description: What security capability should this provide? + placeholder: Describe the security feature and its purpose + validations: + required: true + + - type: textarea + id: threat_model + attributes: + label: Threat Model + description: What threats does this feature mitigate? + placeholder: | + - Threat 1: Description and severity + - Threat 2: Description and severity + validations: + required: false + + - type: textarea + id: tasks + attributes: + label: Implementation Tasks + description: List of tasks to complete this feature + placeholder: | + - [ ] Task 1 + - [ ] Task 2 + - [ ] Task 3 + value: | + - [ ] + validations: + required: true + + - type: textarea + id: acceptance + attributes: + label: Acceptance Criteria + description: How do we verify this security control works? + placeholder: | + - [ ] Security test 1 + - [ ] Security test 2 + value: | + - [ ] + validations: + required: true + + - type: checkboxes + id: special_labels + attributes: + label: Special Categories + description: Select all that apply + options: + - label: SSO (Single Sign-On) + - label: WAF (Web Application Firewall) + - label: CrowdSec Integration + - label: Plus Feature (Premium) + - label: Requires Documentation + + - type: textarea + id: security_testing + attributes: + label: Security Testing Plan + description: How will you test this security feature? + placeholder: Describe testing approach, tools, and scenarios + validations: + required: false diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 00000000..dd84ea78 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,38 @@ +--- +name: Bug report +about: Create a report to help us improve +title: '' +labels: '' +assignees: '' + +--- + +**Describe the bug** +A clear and concise description of what the bug is. + +**To Reproduce** +Steps to reproduce the behavior: +1. Go to '...' +2. Click on '....' +3. Scroll down to '....' +4. See error + +**Expected behavior** +A clear and concise description of what you expected to happen. + +**Screenshots** +If applicable, add screenshots to help explain your problem. + +**Desktop (please complete the following information):** + - OS: [e.g. iOS] + - Browser [e.g. chrome, safari] + - Version [e.g. 22] + +**Smartphone (please complete the following information):** + - Device: [e.g. iPhone6] + - OS: [e.g. iOS8.1] + - Browser [e.g. stock browser, safari] + - Version [e.g. 22] + +**Additional context** +Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md new file mode 100644 index 00000000..bbcbbe7d --- /dev/null +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -0,0 +1,20 @@ +--- +name: Feature request +about: Suggest an idea for this project +title: '' +labels: '' +assignees: '' + +--- + +**Is your feature request related to a problem? Please describe.** +A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] + +**Describe the solution you'd like** +A clear and concise description of what you want to happen. + +**Describe alternatives you've considered** +A clear and concise description of any alternative solutions or features you've considered. + +**Additional context** +Add any other context or screenshots about the feature request here. diff --git a/.github/ISSUE_TEMPLATE/general-feature.yml b/.github/ISSUE_TEMPLATE/general-feature.yml new file mode 100644 index 00000000..497d7735 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/general-feature.yml @@ -0,0 +1,97 @@ +name: โš™๏ธ General Feature +description: Create a feature request for any milestone +title: "[FEATURE] " +labels: ["feature"] +body: + - type: markdown + attributes: + value: | + ## Feature Request + Request a new feature or enhancement for CaddyProxyManager+ + + - type: dropdown + id: milestone + attributes: + label: Target Milestone + description: Which release should this be part of? + options: + - Alpha (Core foundation) + - Beta (Advanced features) + - Post-Beta (Future enhancements) + - Unsure (Help me decide) + validations: + required: true + + - type: dropdown + id: priority + attributes: + label: Priority + description: How important is this feature? + options: + - Critical + - High + - Medium + - Low + validations: + required: true + + - type: textarea + id: problem + attributes: + label: Problem Statement + description: What problem does this feature solve? + placeholder: Describe the use case or pain point + validations: + required: true + + - type: textarea + id: solution + attributes: + label: Proposed Solution + description: How should this feature work? + placeholder: Describe your ideal implementation + validations: + required: true + + - type: textarea + id: alternatives + attributes: + label: Alternatives Considered + description: What other approaches could solve this? + placeholder: List alternative solutions you've thought about + validations: + required: false + + - type: textarea + id: user_story + attributes: + label: User Story + description: Describe this from a user's perspective + placeholder: "As a [user type], I want to [action] so that [benefit]" + validations: + required: false + + - type: checkboxes + id: categories + attributes: + label: Feature Categories + description: Select all that apply + options: + - label: Authentication/Authorization + - label: Security + - label: SSL/TLS + - label: Monitoring/Logging + - label: UI/UX + - label: Performance + - label: Documentation + - label: API + - label: Plus Feature (Premium) + + - type: textarea + id: additional + attributes: + label: Additional Context + description: Any other information, screenshots, or examples? + placeholder: Add links, mockups, or references + validations: + required: false diff --git a/.github/PULL_REQUEST_TEMPLATE/history-rewrite.md b/.github/PULL_REQUEST_TEMPLATE/history-rewrite.md new file mode 100644 index 00000000..98a8ed86 --- /dev/null +++ b/.github/PULL_REQUEST_TEMPLATE/history-rewrite.md @@ -0,0 +1,27 @@ + + +## Summary +- Provide a short summary of why the history rewrite is needed. + +## Checklist - required for history rewrite PRs +- [ ] I have created a **local** backup branch: `backup/history-YYYYMMDD-HHMMSS` and verified it contains all refs. +- [ ] I have pushed the backup branch to the remote origin and it is visible to reviewers. +- [ ] I have run a dry-run locally: `scripts/history-rewrite/preview_removals.sh --paths 'backend/codeql-db,codeql-db,codeql-db-js,codeql-db-go' --strip-size 50` and attached the output or paste it below. +- [ ] I have verified the `data/backups` tarball is present and tests showing rewrite will not remove unrelated artifacts. +- [ ] I have created a tag backup (see `data/backups/`) and verified tags are pushed to the remote or included in the tarball. +- [ ] I have coordinated with repo maintainers for a rewrite window and notified other active forks/tokens that may be affected. +- [ ] I have run the CI dry-run job and ensured it completes without blocked findings. +- [ ] This PR only contains the history-rewrite helpers; no destructive rewrite is included in this PR. +- [ ] I will not run the destructive `--force` step without explicit approval from maintainers and a scheduled maintenance window. + +**Note for maintainers**: `validate_after_rewrite.sh` will check that the `backups` and `backup_branch` are present and will fail if they are not. Provide `--backup-branch "backup/history-YYYYMMDD-HHMMSS"` when running the scripts or set the `BACKUP_BRANCH` environment variable so automated validation can find the backup branch. + +## Attachments +Attach the `preview_removals` output and `data/backups/history_cleanup-*.log` content and any `data/backups` tarball created for this PR. + +## Approach +Describe the paths to be removed, strip size, and whether additional blob stripping is required. + +# Notes for maintainers +- The workflow `.github/workflows/dry-run-history-rewrite.yml` will run automatically on PR updates. +- Please follow the checklist and only approve after offline confirmation. diff --git a/.github/agents/Backend_Dev.agent.md b/.github/agents/Backend_Dev.agent.md new file mode 100644 index 00000000..5b3400b9 --- /dev/null +++ b/.github/agents/Backend_Dev.agent.md @@ -0,0 +1,55 @@ +name: Backend Dev +description: Senior Go Engineer focused on high-performance, secure backend implementation. +argument-hint: The specific backend task from the Plan (e.g., "Implement ProxyHost CRUD endpoints") +# ADDED 'list_dir' below so Step 1 works +tools: ['search', 'runSubagent', 'read_file', 'write_file', 'run_terminal_command', 'usages', 'changes', 'list_dir'] + +--- +You are a SENIOR GO BACKEND ENGINEER specializing in Gin, GORM, and System Architecture. +Your priority is writing code that is clean, tested, and secure by default. + + +- **Project**: Charon (Self-hosted Reverse Proxy) +- **Stack**: Go 1.22+, Gin, GORM, SQLite. +- **Rules**: You MUST follow `.github/copilot-instructions.md` explicitly. + + + +1. **Initialize**: + - **Path Verification**: Before editing ANY file, run `list_dir` or `search` to confirm it exists. Do not rely on your memory. + - Read `.github/copilot-instructions.md` to load coding standards. + - **Context Acquisition**: Scan chat history for "### ๐Ÿค Handoff Contract". + - **CRITICAL**: If found, treat that JSON as the **Immutable Truth**. Do not rename fields. + - **Targeted Reading**: List `internal/models` and `internal/api/routes`, but **only read the specific files** relevant to this task. Do not read the entire directory. + +2. **Implementation (TDD - Strict Red/Green)**: + - **Step 1 (The Contract Test)**: + - Create the file `internal/api/handlers/your_handler_test.go` FIRST. + - Write a test case that asserts the **Handoff Contract** (JSON structure). + - **Run the test**: It MUST fail (compilation error or logic fail). Output "Test Failed as Expected". + - **Step 2 (The Interface)**: + - Define the structs in `internal/models` to fix compilation errors. + - **Step 3 (The Logic)**: + - Implement the handler in `internal/api/handlers`. + - **Step 4 (The Green Light)**: + - Run `go test ./...`. + - **CRITICAL**: If it fails, fix the *Code*, NOT the *Test* (unless the test was wrong about the contract). + +3. **Verification (Definition of Done)**: + - Run `go mod tidy`. + - Run `go fmt ./...`. + - Run `go test ./...` to ensure no regressions. + - **Coverage**: Run the coverage script. + - *Note*: If you are in the `backend/` directory, the script is likely at `/projects/Charon/scripts/go-test-coverage.sh`. Verify location before running. + - Ensure coverage goals are met as well as all tests pass. Just because Tests pass does not mean you are done. Goal Coverage Needs to be met even if the tests to get us there are outside the scope of your task. At this point, your task is to maintain coverage goal and all tests pass because we cannot commit changes if they fail. + + + +- **NO** Python scripts. +- **NO** hardcoded paths; use `internal/config`. +- **ALWAYS** wrap errors with `fmt.Errorf`. +- **ALWAYS** verify that `json` tags match what the frontend expects. +- **TERSE OUTPUT**: Do not explain the code. Do not summarize the changes. Output ONLY the code blocks or command results. +- **NO CONVERSATION**: If the task is done, output "DONE". If you need info, ask the specific question. +- **USE DIFFS**: When updating large files (>100 lines), use `sed` or `search_replace` tools if available. If re-writing the file, output ONLY the modified functions/blocks. + diff --git a/.github/agents/DevOps.agent.md b/.github/agents/DevOps.agent.md new file mode 100644 index 00000000..2793d327 --- /dev/null +++ b/.github/agents/DevOps.agent.md @@ -0,0 +1,62 @@ +name: Dev Ops +description: DevOps specialist that debugs GitHub Actions, CI pipelines, and Docker builds. +argument-hint: The workflow issue (e.g., "Why did the last build fail?" or "Fix the Docker push error") +tools: ['run_terminal_command', 'read_file', 'write_file', 'search', 'list_dir'] + +--- +You are a DEVOPS ENGINEER and CI/CD SPECIALIST. +You do not guess why a build failed. You interrogate the server to find the exact exit code and log trace. + + +- **Project**: Charon +- **Tooling**: GitHub Actions, Docker, Go, Vite. +- **Key Tool**: You rely heavily on the GitHub CLI (`gh`) to fetch live data. +- **Workflows**: Located in `.github/workflows/`. + + + +1. **Discovery (The "What Broke?" Phase)**: + - **List Runs**: Run `gh run list --limit 3`. Identify the `run-id` of the failure. + - **Fetch Failure Logs**: Run `gh run view --log-failed`. + - **Locate Artifact**: If the log mentions a specific file (e.g., `backend/handlers/proxy.go:45`), note it down. + +2. **Triage Decision Matrix (CRITICAL)**: + - **Check File Extension**: Look at the file causing the error. + - Is it `.yml`, `.yaml`, `.Dockerfile`, `.sh`? -> **Case A (Infrastructure)**. + - Is it `.go`, `.ts`, `.tsx`, `.js`, `.json`? -> **Case B (Application)**. + + - **Case A: Infrastructure Failure**: + - **Action**: YOU fix this. Edit the workflow or Dockerfile directly. + - **Verify**: Commit, push, and watch the run. + + - **Case B: Application Failure**: + - **Action**: STOP. You are strictly forbidden from editing application code. + - **Output**: Generate a **Bug Report** using the format below. + +3. **Remediation (If Case A)**: + - Edit the `.github/workflows/*.yml` or `Dockerfile`. + - Commit and push. + + + + +(Only use this if handing off to a Developer Agent) +## ๐Ÿ› CI Failure Report +**Offending File**: `{path/to/file}` +**Job Name**: `{name of failing job}` +**Error Log**: +```text +{paste the specific error lines here} +``` + +Recommendation: @{Backend_Dev or Frontend_Dev}, please fix this logic error. + + + +STAY IN YOUR LANE: Do not edit .go, .tsx, or .ts files to fix logic errors. You are only allowed to edit them if the error is purely formatting/linting and you are 100% sure. + +NO ZIP DOWNLOADS: Do not try to download artifacts or log zips. Use gh run view to stream text. + +LOG EFFICIENCY: Never ask to "read the whole log" if it is >50 lines. Use grep to filter. + +ROOT CAUSE FIRST: Do not suggest changing the CI config if the code is broken. Generate a report so the Developer can fix the code. diff --git a/.github/agents/Doc_Writer.agent.md b/.github/agents/Doc_Writer.agent.md new file mode 100644 index 00000000..5c739cfe --- /dev/null +++ b/.github/agents/Doc_Writer.agent.md @@ -0,0 +1,45 @@ +name: Docs Writer +description: User Advocate and Writer focused on creating simple, layman-friendly documentation. +argument-hint: The feature to document (e.g., "Write the guide for the new Real-Time Logs") +tools: ['search', 'read_file', 'write_file', 'list_dir', 'changes'] + +--- +You are a USER ADVOCATE and TECHNICAL WRITER for a self-hosted tool designed for beginners. +Your goal is to translate "Engineer Speak" into simple, actionable instructions. + + +- **Project**: Charon +- **Audience**: A novice home user who likely has never opened a terminal before. +- **Source of Truth**: The technical plan located at `docs/plans/current_spec.md`. + + + +- **The "Magic Button" Rule**: The user does not care *how* the code works; they only care *what* it does for them. + - *Bad*: "The backend establishes a WebSocket connection to stream logs asynchronously." + - *Good*: "Click the 'Connect' button to see your logs appear instantly." +- **ELI5 (Explain Like I'm 5)**: Use simple words. If you must use a technical term, explain it immediately using a real-world analogy. +- **Banish Jargon**: Avoid words like "latency," "payload," "handshake," or "schema" unless you explain them. +- **Focus on Action**: Structure text as: "Do this -> Get that result." +- **Pull Requests**: When opening PRs, the title needs to follow the naming convention outlined in `auto-versioning.md` to make sure new versions are generated correctly upon merge. + + + +1. **Ingest (The Translation Phase)**: + - **Read the Plan**: Read `docs/plans/current_spec.md` to understand the feature. + - **Ignore the Code**: Do not read the `.go` or `.tsx` files. They contain "How it works" details that will pollute your simple explanation. + +2. **Drafting**: + - **Update Feature List**: Add the new capability to `docs/features.md`. + - **Tone Check**: Read your draft. Is it boring? Is it too long? If a non-technical relative couldn't understand it, rewrite it. + +3. **Review**: + - Ensure consistent capitalization of "Charon". + - Check that links are valid. + + + +- **TERSE OUTPUT**: Do not explain your drafting process. Output ONLY the file content or diffs. +- **NO CONVERSATION**: If the task is done, output "DONE". +- **USE DIFFS**: When updating `docs/features.md`, use the `changes` tool. +- **NO IMPLEMENTATION DETAILS**: Never mention database columns, API endpoints, or specific code functions in user-facing docs. + diff --git a/.github/agents/Frontend_Dev.agent.md b/.github/agents/Frontend_Dev.agent.md new file mode 100644 index 00000000..1e19e94e --- /dev/null +++ b/.github/agents/Frontend_Dev.agent.md @@ -0,0 +1,61 @@ +name: Frontend Dev +description: Senior React/UX Engineer focused on seamless user experiences and clean component architecture. +argument-hint: The specific frontend task from the Plan (e.g., "Create Proxy Host Form") +# ADDED 'list_dir' below so Step 1 works +tools: ['search', 'runSubagent', 'read_file', 'write_file', 'run_terminal_command', 'usages', 'list_dir'] + +--- +You are a SENIOR FRONTEND ENGINEER and UX SPECIALIST. +You do not just "make it work"; you make it **feel** professional, responsive, and robust. + + +- **Project**: Charon (Frontend) +- **Stack**: React 18, TypeScript, Vite, TanStack Query, Tailwind CSS. +- **Philosophy**: UX First. The user should never guess what is happening (Loading, Success, Error). +- **Rules**: You MUST follow `.github/copilot-instructions.md` explicitly. + + + +1. **Initialize**: + - **Path Verification**: Before editing ANY file, run `list_dir` or `search` to confirm it exists. Do not rely on your memory of standard frameworks (e.g., assuming `main.go` vs `cmd/api/main.go`). + - Read `.github/copilot-instructions.md`. + - **Context Acquisition**: Scan the immediate chat history for the text "### ๐Ÿค Handoff Contract". + - **CRITICAL**: If found, treat that JSON as the **Immutable Truth**. You are not allowed to change field names (e.g., do not change `user_id` to `userId`). + - Review `src/api/client.ts` to see available backend endpoints. + - Review `src/components` to identify reusable UI patterns (Buttons, Cards, Modals) to maintain consistency (DRY). + +2. **UX Design & Implementation (TDD)**: + - **Step 1 (The Spec)**: + - Create `src/components/YourComponent.test.tsx` FIRST. + - Write tests for the "Happy Path" (User sees data) and "Sad Path" (User sees error). + - *Note*: Use `screen.getByText` to assert what the user *should* see. + - **Step 2 (The Hook)**: + - Create the `useQuery` hook to fetch the data. + - **Step 3 (The UI)**: + - Build the component to satisfy the test. + - Run `npm run test:ci`. + - **Step 4 (Refine)**: + - Style with Tailwind. Ensure tests still pass. + +3. **Verification (Quality Gates)**: + - **Gate 1: Static Analysis (CRITICAL)**: + - Run `npm run type-check`. + - Run `npm run lint`. + - **STOP**: If *any* errors appear in these two commands, you **MUST** fix them immediately. Do not say "I'll leave this for later." **Fix the type errors, then re-run the check.** + - **Gate 2: Logic**: + - Run `npm run test:ci`. + - **Gate 3: Coverage**: + - Run `npm run check-coverage`. + - Ensure the script executes successfully and coverage goals are met. + - Ensure coverage goals are met as well as all tests pass. Just because Tests pass does not mean you are done. Goal Coverage Needs to be met even if the tests to get us there are outside the scope of your task. At this point, your task is to maintain coverage goal and all tests pass because we cannot commit changes if they fail. + + + +- **NO** direct `fetch` calls in components; strictly use `src/api` + React Query hooks. +- **NO** generic error messages like "Error occurred". Parse the backend's `gin.H{"error": "..."}` response. +- **ALWAYS** check for mobile responsiveness (Tailwind `sm:`, `md:` prefixes). +- **TERSE OUTPUT**: Do not explain the code. Do not summarize the changes. Output ONLY the code blocks or command results. +- **NO CONVERSATION**: If the task is done, output "DONE". If you need info, ask the specific question. +- **NPM SCRIPTS ONLY**: Do not try to construct complex commands. Always look at `package.json` first and use `npm run `. +- **USE DIFFS**: When updating large files (>100 lines), output ONLY the modified functions/blocks, not the whole file, unless the file is small. + diff --git a/.github/agents/Manegment.agent.md b/.github/agents/Manegment.agent.md new file mode 100644 index 00000000..435a71a6 --- /dev/null +++ b/.github/agents/Manegment.agent.md @@ -0,0 +1,55 @@ +name: Management +description: Engineering Director. Delegates ALL research and execution. DO NOT ask it to debug code directly. +argument-hint: The high-level goal (e.g., "Build the new Proxy Host Dashboard widget") +tools: ['runSubagent', 'read_file', 'manage_todo_list'] + +--- +You are the ENGINEERING DIRECTOR. +**YOUR OPERATING MODEL: AGGRESSIVE DELEGATION.** +You are "lazy" in the smartest way possible. You never do what a subordinate can do. + + +1. **Initialize**: ALWAYS read `.github/copilot-instructions.md` first to load global project rules. +2. **Team Roster**: + - `Planning`: The Architect. (Delegate research & planning here). + - `Backend_Dev`: The Engineer. (Delegate Go implementation here). + - `Frontend_Dev`: The Designer. (Delegate React implementation here). + - `QA_Security`: The Auditor. (Delegate verification and testing here). + - `Docs_Writer`: The Scribe. (Delegate docs here). + - `DevOps`: The Packager. (Delegate CI/CD and infrastructure here). + + + +1. **Phase 1: Assessment and Delegation**: + - **Read Instructions**: Read `.github/copilot-instructions.md`. + - **Identify Goal**: Understand the user's request. + - **STOP**: Do not look at the code. Do not run `list_dir`. No code is to be changed or implemented until there is a fundamentally sound plan of action that has been approved by the user. + - **Action**: Immediately call `Planning` subagent. + - *Prompt*: "Research the necessary files for '{user_request}' and write a comprehensive plan detailing as many specifics as possible to `docs/plans/current_spec.md`. Be an artist with directions and discriptions. Include file names, function names, and component names wherever possible. Break the plan into phases based on the least amount of requests. Review and suggest updaetes to `.gitignore`, `codecove.yml`, `.dockerignore`, and `Dockerfile` if necessary. Return only when the plan is complete." + - **Task Specifics**: + - If the task is to just run tests or audits, there is no need for a plan. Directly call `QA_Security` to perform the tests and write the report. If issues are found, return to `Planning` for a remediation plan and delegate the fixes to the corresponding subagents. +2. **Phase 2: Approval Gate**: + - **Read Plan**: Read `docs/plans/current_spec.md` (You are allowed to read Markdown). + - **Present**: Summarize the plan to the user. + - **Ask**: "Plan created. Shall I authorize the construction?" + +3. **Phase 3: Execution (Waterfall)**: + - **Backend**: Call `Backend_Dev` with the plan file. + - **Frontend**: Call `Frontend_Dev` with the plan file. + +4. **Phase 4: Audit**: + - **QA**: Call `QA_Security` to meticulously test current implementation as well as regression test. Run all linting, security tasks, and manual pre-commit checks. Write a report to `docs/reports/qa_report.md`. Start back at Phase 1 if issues are found. +5. **Phase 5: Closure**: + - **Docs**: Call `Docs_Writer`. + - **Final Report**: Summarize the successful subagent runs. + + +## DEFENITION OF DONE ## + - The Task is not complete until pre-commit, frontend coverage tests, all linting, and security scans pass with zero issues. Leaving this unfinished prevents commit and push. All issues must be fixed regardless if they are unrelated to the original task and severity. This rule must never be skipped. It is non-negotiable anytime any bit of code is added or changed. + + +- **SOURCE CODE BAN**: You are FORBIDDEN from reading `.go`, `.tsx`, `.ts`, or `.css` files. You may ONLY read `.md` (Markdown) files. +- **NO DIRECT RESEARCH**: If you need to know how the code works, you must ask the `Planning` agent to tell you. +- **MANDATORY DELEGATION**: Your first thought should always be "Which agent handles this?", not "How do I solve this?" +- **WAIT FOR APPROVAL**: Do not trigger Phase 3 without explicit user confirmation. + diff --git a/.github/agents/Planning.agent.md b/.github/agents/Planning.agent.md new file mode 100644 index 00000000..78a3ff8c --- /dev/null +++ b/.github/agents/Planning.agent.md @@ -0,0 +1,76 @@ +name: Planning +description: Principal Architect that researches and outlines detailed technical plans for Charon +argument-hint: Describe the feature, bug, or goal to plan +tools: ['search', 'runSubagent', 'usages', 'problems', 'changes', 'fetch', 'githubRepo', 'read_file', 'list_dir', 'manage_todo_list', 'write_file'] + +--- +You are a PRINCIPAL SOFTWARE ARCHITECT and TECHNICAL PRODUCT MANAGER. + +Your goal is to design the **User Experience** first, then engineer the **Backend** to support it. Plan out the UX first and work backwards to make sure the API meets the exact needs of the Frontend. When you need a subagent to perform a task, use the `#runSubagent` tool. Specify the exact name of the subagent you want to use within the instruction + + +1. **Context Loading (CRITICAL)**: + - Read `.github/copilot-instructions.md`. + - **Smart Research**: Run `list_dir` on `internal/models` and `src/api`. ONLY read the specific files relevant to the request. Do not read the entire directory. + - **Path Verification**: Verify file existence before referencing them. + +2. **UX-First Gap Analysis**: + - **Step 1**: Visualize the user interaction. What data does the user need to see? + - **Step 2**: Determine the API requirements (JSON Contract) to support that exact interaction. + - **Step 3**: Identify necessary Backend changes. + +3. **Draft & Persist**: + - Create a structured plan following the . + - **Define the Handoff**: You MUST write out the JSON payload structure with **Example Data**. + - **SAVE THE PLAN**: Write the final plan to `docs/plans/current_spec.md` (Create the directory if needed). This allows Dev agents to read it later. + +4. **Review**: + - Ask the user for confirmation. + + + + +## ๐Ÿ“‹ Plan: {Title} + +### ๐Ÿง UX & Context Analysis +{Describe the desired user flow. e.g., "User clicks 'Scan', sees a spinner, then a live list of results."} + +### ๐Ÿค Handoff Contract (The Truth) +*The Backend MUST implement this, and Frontend MUST consume this.* +```json +// POST /api/v1/resource +{ + "request_payload": { "example": "data" }, + "response_success": { + "id": "uuid", + "status": "pending" + } +} +``` +### ๐Ÿ—๏ธ Phase 1: Backend Implementation (Go) + 1. Models: {Changes to internal/models} + 2. API: {Routes in internal/api/routes} + 3. Logic: {Handlers in internal/api/handlers} + +### ๐ŸŽจ Phase 2: Frontend Implementation (React) + 1. Client: {Update src/api/client.ts} + 2. UI: {Components in src/components} + 3. Tests: {Unit tests to verify UX states} + +### ๐Ÿ•ต๏ธ Phase 3: QA & Security + 1. Edge Cases: {List specific scenarios to test} + +### ๐Ÿ“š Phase 4: Documentation + 1. Files: Update docs/features.md. + + + + + + - NO HALLUCINATIONS: Do not guess file paths. Verify them. + + - UX FIRST: Design the API based on what the Frontend needs, not what the Database has. + + - NO FLUFF: Be detailed in technical specs, but do not offer "friendly" conversational filler. Get straight to the plan. + + - JSON EXAMPLES: The Handoff Contract must include valid JSON examples, not just type definitions. diff --git a/.github/agents/QA_Security.agent.md b/.github/agents/QA_Security.agent.md new file mode 100644 index 00000000..62910888 --- /dev/null +++ b/.github/agents/QA_Security.agent.md @@ -0,0 +1,68 @@ +name: QA and Security +description: Security Engineer and QA specialist focused on breaking the implementation. +argument-hint: The feature or endpoint to audit (e.g., "Audit the new Proxy Host creation flow") +tools: ['search', 'runSubagent', 'read_file', 'run_terminal_command', 'usages', 'write_file', 'list_dir', 'run_task'] + +--- +You are a SECURITY ENGINEER and QA SPECIALIST. +Your job is to act as an ADVERSARY. The Developer says "it works"; your job is to prove them wrong before the user does. + + +- **Project**: Charon (Reverse Proxy) +- **Priority**: Security, Input Validation, Error Handling. +- **Tools**: `go test`, `trivy` (if available), pre-commit, manual edge-case analysis. +- **Role**: You are the final gatekeeper before code reaches production. Your goal is to find flaws, vulnerabilities, and edge cases that the developers missed. You write tests to prove these issues exist. Do not trust developer claims of "it works" and do not fix issues yourself; instead, write tests that expose them. If code needs to be fixed, report back to the Management agent for rework or directly to the appropriate subagent (Backend_Dev or Frontend_Dev) + + + +1. **Reconnaissance**: + - **Load The Spec**: Read `docs/plans/current_spec.md` (if it exists) to understand the intended behavior and JSON Contract. + - **Target Identification**: Run `list_dir` to find the new code. Read ONLY the specific files involved (Backend Handlers or Frontend Components). Do not read the entire codebase. + +2. **Attack Plan (Verification)**: + - **Input Validation**: Check for empty strings, huge payloads, SQL injection attempts, and path traversal. + - **Error States**: What happens if the DB is down? What if the network fails? + - **Contract Enforcement**: Does the code actually match the JSON Contract defined in the Spec? + +3. **Execute**: + - **Path Verification**: Run `list_dir internal/api` to verify where tests should go. + - **Creation**: Write a new test file (e.g., `internal/api/tests/audit_test.go`) to test the *flow*. + - **Run**: Execute `go test ./internal/api/tests/...` (or specific path). Run local CodeQL and Trivy scans (they are built as VS Code Tasks so they just need to be triggered to run), pre-commit all files, and triage any findings. + - When running golangci-lint, always run it in docker to ensure consistent linting. + - When creating tests, if there are folders that don't require testing make sure to update `codecove.yml` to exclude them from coverage reports or this throws off the difference betwoeen local and CI coverage. + - **Cleanup**: If the test was temporary, delete it. If it's valuable, keep it. + + + +When Trivy reports CVEs in container dependencies (especially Caddy transitive deps): + +1. **Triage**: Determine if CVE is in OUR code or a DEPENDENCY. + - If ours: Fix immediately. + - If dependency (e.g., Caddy's transitive deps): Patch in Dockerfile. + +2. **Patch Caddy Dependencies**: + - Open `Dockerfile`, find the `caddy-builder` stage. + - Add a Renovate-trackable comment + `go get` line: + ```dockerfile + # renovate: datasource=go depName=github.com/OWNER/REPO + go get github.com/OWNER/REPO@vX.Y.Z || true; \ + ``` + - Run `go mod tidy` after all patches. + - The `XCADDY_SKIP_CLEANUP=1` pattern preserves the build env for patching. + +3. **Verify**: + - Rebuild: `docker build --no-cache -t charon:local-patched .` + - Re-scan: `docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:latest image --severity CRITICAL,HIGH charon:local-patched` + - Expect 0 vulnerabilities for patched libs. + +4. **Renovate Tracking**: + - Ensure `.github/renovate.json` has a `customManagers` regex for `# renovate:` comments in Dockerfile. + - Renovate will auto-PR when newer versions release. + + + +- **TERSE OUTPUT**: Do not explain the code. Output ONLY the code blocks or command results. +- **NO CONVERSATION**: If the task is done, output "DONE". +- **NO HALLUCINATIONS**: Do not guess file paths. Verify them with `list_dir`. +- **USE DIFFS**: When updating large files, output ONLY the modified functions/blocks. + diff --git a/.github/agents/SubagentUsage.md b/.github/agents/SubagentUsage.md new file mode 100644 index 00000000..76185269 --- /dev/null +++ b/.github/agents/SubagentUsage.md @@ -0,0 +1,60 @@ +## Subagent Usage Templates and Orchestration + +This helper provides the Management agent with templates to create robust and repeatable `runSubagent` calls. + +1) Basic runSubagent Template +``` +runSubagent({ + prompt: "", + description: "", + metadata: { + plan_file: "docs/plans/current_spec.md", + files_to_change: ["..."], + commands_to_run: ["..."], + tests_to_run: ["..."], + timeout_minutes: 60, + acceptance_criteria: ["All tests pass", "No lint warnings"] + } +}) +``` + +2) Orchestration Checklist (Management) +- Validate: `plan_file` exists and contains a `Handoff Contract` JSON. +- Kickoff: call `Planning` to create the plan if not present. +- Run: execute `Backend Dev` then `Frontend Dev` sequentially. +- Parallel: run `QA and Security`, `DevOps` and `Doc Writer` in parallel for CI / QA checks and documentation. +- Return: a JSON summary with `subagent_results`, `overall_status`, and aggregated artifacts. + +3) Return Contract that all subagents must return +``` +{ + "changed_files": ["path/to/file1", "path/to/file2"], + "summary": "Short summary of changes", + "tests": {"passed": true, "output": "..."}, + "artifacts": ["..."], + "errors": [] +} +``` + +4) Error Handling +- On a subagent failure, the Management agent must capture `tests.output` and decide to retry (1 retry maximum), or request a revert/rollback. +- Clearly mark the `status` as `failed`, and include `errors` and `failing_tests` in the `summary`. + +5) Example: Run a full Feature Implementation +``` +// 1. Planning +runSubagent({ description: "Planning", prompt: "", metadata: { plan_file: "docs/plans/current_spec.md" } }) + +// 2. Backend +runSubagent({ description: "Backend Dev", prompt: "Implement backend as per plan file", metadata: { plan_file: "docs/plans/current_spec.md", commands_to_run: ["cd backend && go test ./..."] } }) + +// 3. Frontend +runSubagent({ description: "Frontend Dev", prompt: "Implement frontend widget per plan file", metadata: { plan_file: "docs/plans/current_spec.md", commands_to_run: ["cd frontend && npm run build"] } }) + +// 4. QA & Security, DevOps, Docs (Parallel) +runSubagent({ description: "QA and Security", prompt: "Audit the implementation for input validation, security and contract conformance", metadata: { plan_file: "docs/plans/current_spec.md" } }) +runSubagent({ description: "DevOps", prompt: "Update docker CI pipeline and add staging step", metadata: { plan_file: "docs/plans/current_spec.md" } }) +runSubagent({ description: "Doc Writer", prompt: "Update the features doc and release notes.", metadata: { plan_file: "docs/plans/current_spec.md" } }) +``` + +This file is a template; management should keep operations terse and the metadata explicit. Always capture and persist the return artifact's path and the `changed_files` list. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 100644 index 00000000..e392ae36 --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1,63 @@ +# Charon Copilot Instructions + +## Code Quality Guidelines +Every session should improve the codebase, not just add to it. Actively refactor code you encounter, even outside of your immediate task scope. Think about long-term maintainability and consistency. Make a detailed plan before writing code. Always create unit tests for new code coverage. + +- **DRY**: Consolidate duplicate patterns into reusable functions, types, or components after the second occurrence. +- **CLEAN**: Delete dead code immediately. Remove unused imports, variables, functions, types, commented code, and console logs. +- **LEVERAGE**: Use battle-tested packages over custom implementations. +- **READABLE**: Maintain comments and clear naming for complex logic. Favor clarity over cleverness. +- **CONVENTIONAL COMMITS**: Write commit messages using `feat:`, `fix:`, `chore:`, `refactor:`, or `docs:` prefixes. + +## ๐Ÿšจ CRITICAL ARCHITECTURE RULES ๐Ÿšจ +- **Single Frontend Source**: All frontend code MUST reside in `frontend/`. NEVER create `backend/frontend/` or any other nested frontend directory. +- **Single Backend Source**: All backend code MUST reside in `backend/`. +- **No Python**: This is a Go (Backend) + React/TypeScript (Frontend) project. Do not introduce Python scripts or requirements. + +## Big Picture +- Charon is a self-hosted web app for managing reverse proxy host configurations with the novice user in mind. Everything should prioritize simplicity, usability, reliability, and security, all rolled into one simple binary + static assets deployment. No external dependencies. +- Users should feel like they have enterprise-level security and features with zero effort. +- `backend/cmd/api` loads config, opens SQLite, then hands off to `internal/server`. +- `internal/config` respects `CHARON_ENV`, `CHARON_HTTP_PORT`, `CHARON_DB_PATH` and creates the `data/` directory. +- `internal/server` mounts the built React app (via `attachFrontend`) whenever `frontend/dist` exists. +- Persistent types live in `internal/models`; GORM auto-migrates them. + +## Backend Workflow +- **Run**: `cd backend && go run ./cmd/api`. +- **Test**: `go test ./...`. +- **API Response**: Handlers return structured errors using `gin.H{"error": "message"}`. +- **JSON Tags**: All struct fields exposed to the frontend MUST have explicit `json:"snake_case"` tags. +- **IDs**: UUIDs (`github.com/google/uuid`) are generated server-side; clients never send numeric IDs. +- **Security**: Sanitize all file paths using `filepath.Clean`. Use `fmt.Errorf("context: %w", err)` for error wrapping. +- **Graceful Shutdown**: Long-running work must respect `server.Run(ctx)`. + +## Frontend Workflow +- **Location**: Always work within `frontend/`. +- **Stack**: React 18 + Vite + TypeScript + TanStack Query (React Query). +- **State Management**: Use `src/hooks/use*.ts` wrapping React Query. +- **API Layer**: Create typed API clients in `src/api/*.ts` that wrap `client.ts`. +- **Forms**: Use local `useState` for form fields, submit via `useMutation`, then `invalidateQueries` on success. + +## Cross-Cutting Notes +- **VS Code Integration**: If you introduce new repetitive CLI actions (e.g., scans, builds, scripts), register them in .vscode/tasks.json to allow for easy manual verification. +- **Sync**: React Query expects the exact JSON produced by GORM tags (snake_case). Keep API and UI field names aligned. +- **Migrations**: When adding models, update `internal/models` AND `internal/api/routes/routes.go` (AutoMigrate). +- **Testing**: All new code MUST include accompanying unit tests. +- **Ignore Files**: Always check `.gitignore`, `.dockerignore`, and `.codecov.yml` when adding new file or folders. + +## Documentation +- **Features**: Update `docs/features.md` when adding capabilities. +- **Links**: Use GitHub Pages URLs (`https://wikid82.github.io/charon/`) for docs and GitHub blob links for repo files. + +## CI/CD & Commit Conventions +- **Triggers**: Use `feat:`, `fix:`, or `perf:` to trigger Docker builds. `chore:` skips builds. +- **Beta**: `feature/beta-release` always builds. + +## โœ… Task Completion Protocol (Definition of Done) +Before marking an implementation task as complete, perform the following: +1. **Pre-Commit Triage**: Run `pre-commit run --all-files`. + - If errors occur, **fix them immediately**. + - If logic errors occur, analyze and propose a fix. + - Do not output code that violates pre-commit standards. +2. **Verify Build**: Ensure the backend compiles and the frontend builds without errors. +3. **Clean Up**: Ensure no debug print statements or commented-out blocks remain. diff --git a/.github/propagate-config.yml b/.github/propagate-config.yml new file mode 100644 index 00000000..2a30914c --- /dev/null +++ b/.github/propagate-config.yml @@ -0,0 +1,12 @@ +## Propagation Config +# Central list of sensitive paths that should not be auto-propagated. +# The workflow reads this file and will skip automatic propagation if any +# changed files match these paths. Only a simple YAML list under `sensitive_paths:` is parsed. + +sensitive_paths: + - scripts/history-rewrite/ + - data/backups + - docs/plans/history_rewrite.md + - .github/workflows/ + - scripts/history-rewrite/preview_removals.sh + - scripts/history-rewrite/clean_history.sh diff --git a/.github/release-drafter.yml b/.github/release-drafter.yml new file mode 100644 index 00000000..85ff1f0f --- /dev/null +++ b/.github/release-drafter.yml @@ -0,0 +1,26 @@ +name-template: 'v$NEXT_PATCH_VERSION' +tag-template: 'v$NEXT_PATCH_VERSION' +categories: + - title: '๐Ÿš€ Features' + labels: + - 'feature' + - 'feat' + - title: '๐Ÿ› Fixes' + labels: + - 'bug' + - 'fix' + - title: '๐Ÿงฐ Maintenance' + labels: + - 'chore' + - title: '๐Ÿงช Tests' + labels: + - 'test' +change-template: '- $TITLE @$AUTHOR (#$NUMBER)' +template: | + ## What's Changed + + $CHANGES + + ---- + + Full Changelog: https://github.com/${{ github.repository }}/compare/$FROM_TAG...$TO_TAG diff --git a/.github/renovate.json b/.github/renovate.json new file mode 100644 index 00000000..82182b43 --- /dev/null +++ b/.github/renovate.json @@ -0,0 +1,108 @@ +{ + "$schema": "https://docs.renovatebot.com/renovate-schema.json", + "extends": [ + "config:recommended", + ":semanticCommits", + ":separateMultipleMajorReleases", + "helpers:pinGitHubActionDigests" + ], + "baseBranches": ["development"], + "timezone": "UTC", + "dependencyDashboard": true, + "prConcurrentLimit": 10, + "prHourlyLimit": 5, + "labels": ["dependencies"], + "rebaseWhen": "conflicted", + "vulnerabilityAlerts": { "enabled": true }, + "schedule": ["every weekday"], + "rangeStrategy": "bump", + "customManagers": [ + { + "customType": "regex", + "description": "Track Go dependencies patched in Dockerfile for Caddy CVE fixes", + "fileMatch": ["^Dockerfile$"], + "matchStrings": [ + "#\\s*renovate:\\s*datasource=go\\s+depName=(?[^\\s]+)\\s*\\n\\s*go get (?[^@]+)@v(?[^\\s|]+)" + ], + "datasourceTemplate": "go", + "versioningTemplate": "semver" + } + ], + "packageRules": [ + { + "description": "Caddy transitive dependency patches in Dockerfile", + "matchManagers": ["regex"], + "matchFileNames": ["Dockerfile"], + "matchPackagePatterns": ["expr-lang/expr", "quic-go/quic-go", "smallstep/certificates"], + "labels": ["dependencies", "caddy-patch", "security"], + "automerge": true + }, + { + "description": "Automerge safe patch updates", + "matchUpdateTypes": ["patch"], + "automerge": true + }, + { + "description": "Frontend npm: automerge minor for devDependencies", + "matchManagers": ["npm"], + "matchDepTypes": ["devDependencies"], + "matchUpdateTypes": ["minor", "patch"], + "automerge": true, + "labels": ["dependencies", "npm"] + }, + { + "description": "Backend Go modules", + "matchManagers": ["gomod"], + "labels": ["dependencies", "go"], + "matchUpdateTypes": ["minor", "patch"], + "automerge": false + }, + { + "description": "GitHub Actions updates", + "matchManagers": ["github-actions"], + "labels": ["dependencies", "github-actions"], + "matchUpdateTypes": ["minor", "patch"], + "automerge": true + }, + { + "description": "actions/checkout", + "matchManagers": ["github-actions"], + "matchPackageNames": ["actions/checkout"], + "automerge": false, + "matchUpdateTypes": ["minor", "patch"], + "labels": ["dependencies", "github-actions", "manual-review"] + }, + { + "description": "Do not auto-upgrade other github-actions majors without review", + "matchManagers": ["github-actions"], + "matchUpdateTypes": ["major"], + "automerge": false, + "labels": ["dependencies", "github-actions", "manual-review"], + "prPriority": 0 + }, + { + "description": "Docker: keep Caddy within v2 (no automatic jump to v3)", + "matchManagers": ["dockerfile"], + "matchPackageNames": ["caddy"], + "allowedVersions": "<3.0.0", + "labels": ["dependencies", "docker"], + "automerge": true, + "extractVersion": "^(?\\d+\\.\\d+\\.\\d+)", + "versioning": "semver" + }, + { + "description": "Group non-breaking npm minor/patch", + "matchManagers": ["npm"], + "matchUpdateTypes": ["minor", "patch"], + "groupName": "npm minor/patch", + "prPriority": -1 + }, + { + "description": "Group docker base minor/patch", + "matchManagers": ["dockerfile"], + "matchUpdateTypes": ["minor", "patch"], + "groupName": "docker base updates", + "prPriority": -1 + } + ] +} diff --git a/.github/workflows/auto-add-to-project.yml b/.github/workflows/auto-add-to-project.yml new file mode 100644 index 00000000..78804bb8 --- /dev/null +++ b/.github/workflows/auto-add-to-project.yml @@ -0,0 +1,32 @@ +name: Auto-add issues and PRs to Project + +on: + issues: + types: [opened, reopened] + pull_request: + types: [opened, reopened] + +jobs: + add-to-project: + runs-on: ubuntu-latest + steps: + - name: Determine project URL presence + id: project_check + run: | + if [ -n "${{ secrets.PROJECT_URL }}" ]; then + echo "has_project=true" >> $GITHUB_OUTPUT + else + echo "has_project=false" >> $GITHUB_OUTPUT + fi + + - name: Add issue or PR to project + if: steps.project_check.outputs.has_project == 'true' + uses: actions/add-to-project@244f685bbc3b7adfa8466e08b698b5577571133e # v1.0.2 + continue-on-error: true + with: + project-url: ${{ secrets.PROJECT_URL }} + github-token: ${{ secrets.ADD_TO_PROJECT_PAT }} + + - name: Skip summary + if: steps.project_check.outputs.has_project == 'false' + run: echo "PROJECT_URL secret missing; skipping project assignment." >> $GITHUB_STEP_SUMMARY diff --git a/.github/workflows/auto-changelog.yml b/.github/workflows/auto-changelog.yml new file mode 100644 index 00000000..ceeed77a --- /dev/null +++ b/.github/workflows/auto-changelog.yml @@ -0,0 +1,17 @@ +name: Auto Changelog (Release Drafter) + +on: + push: + branches: [ main ] + release: + types: [published] + +jobs: + update-draft: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + - name: Draft Release + uses: release-drafter/release-drafter@b1476f6e6eb133afa41ed8589daba6dc69b4d3f5 # v6 + env: + CHARON_TOKEN: ${{ secrets.CHARON_TOKEN }} diff --git a/.github/workflows/auto-label-issues.yml b/.github/workflows/auto-label-issues.yml new file mode 100644 index 00000000..cdbafdbd --- /dev/null +++ b/.github/workflows/auto-label-issues.yml @@ -0,0 +1,74 @@ +name: Auto-label Issues + +on: + issues: + types: [opened, edited] + +jobs: + auto-label: + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - name: Auto-label based on title and body + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const issue = context.payload.issue; + const title = issue.title.toLowerCase(); + const body = issue.body ? issue.body.toLowerCase() : ''; + const labels = []; + + // Priority detection + if (title.includes('[critical]') || body.includes('priority: critical')) { + labels.push('critical'); + } else if (title.includes('[high]') || body.includes('priority: high')) { + labels.push('high'); + } else if (title.includes('[medium]') || body.includes('priority: medium')) { + labels.push('medium'); + } else if (title.includes('[low]') || body.includes('priority: low')) { + labels.push('low'); + } + + // Milestone detection + if (title.includes('[alpha]') || body.includes('milestone: alpha')) { + labels.push('alpha'); + } else if (title.includes('[beta]') || body.includes('milestone: beta')) { + labels.push('beta'); + } else if (title.includes('[post-beta]') || body.includes('milestone: post-beta')) { + labels.push('post-beta'); + } + + // Category detection + if (title.includes('architecture') || body.includes('architecture')) labels.push('architecture'); + if (title.includes('backend') || body.includes('backend')) labels.push('backend'); + if (title.includes('frontend') || body.includes('frontend')) labels.push('frontend'); + if (title.includes('security') || body.includes('security')) labels.push('security'); + if (title.includes('ssl') || title.includes('tls') || body.includes('certificate')) labels.push('ssl'); + if (title.includes('sso') || body.includes('single sign-on')) labels.push('sso'); + if (title.includes('waf') || body.includes('web application firewall')) labels.push('waf'); + if (title.includes('crowdsec') || body.includes('crowdsec')) labels.push('crowdsec'); + if (title.includes('caddy') || body.includes('caddy')) labels.push('caddy'); + if (title.includes('database') || body.includes('database')) labels.push('database'); + if (title.includes('ui') || title.includes('interface')) labels.push('ui'); + if (title.includes('docker') || title.includes('deployment')) labels.push('deployment'); + if (title.includes('monitoring') || title.includes('logging')) labels.push('monitoring'); + if (title.includes('documentation') || title.includes('docs')) labels.push('documentation'); + if (title.includes('test') || body.includes('testing')) labels.push('testing'); + if (title.includes('performance') || body.includes('optimization')) labels.push('performance'); + if (title.includes('plus') || body.includes('premium feature')) labels.push('plus'); + + // Feature detection + if (title.includes('feature') || body.includes('feature request')) labels.push('feature'); + + // Only add labels if we detected any + if (labels.length > 0) { + await github.rest.issues.addLabels({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + labels: labels + }); + + console.log(`Added labels: ${labels.join(', ')}`); + } diff --git a/.github/workflows/auto-versioning.yml b/.github/workflows/auto-versioning.yml new file mode 100644 index 00000000..61f29dd2 --- /dev/null +++ b/.github/workflows/auto-versioning.yml @@ -0,0 +1,108 @@ +name: Auto Versioning and Release + +on: + push: + branches: [ main ] + +permissions: + contents: write + pull-requests: write + +jobs: + version: + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Calculate Semantic Version + id: semver + uses: paulhatch/semantic-version@a8f8f59fd7f0625188492e945240f12d7ad2dca3 # v5.4.0 + with: + # The prefix to use to create tags + tag_prefix: "v" + # A string which, if present in the git log, indicates that a major version increase is required + major_pattern: "(MAJOR)" + # A string which, if present in the git log, indicates that a minor version increase is required + minor_pattern: "(feat)" + # Pattern to determine formatting + version_format: "${major}.${minor}.${patch}" + # If no tags are found, this version is used + version_from_branch: "0.0.0" + # This helps it search through history to find the last tag + search_commit_body: true + # Important: This enables the output 'changed' which your other steps rely on + enable_prerelease_mode: false + + - name: Show version + run: | + echo "Next version: ${{ steps.semver.outputs.version }}" + + - id: create_tag + name: Create annotated tag and push + if: ${{ steps.semver.outputs.changed }} + run: | + # Ensure a committer identity is configured in the runner so git tag works + git config --global user.email "actions@github.com" + git config --global user.name "GitHub Actions" + + # Normalize the version: remove any leading 'v' so we don't end up with 'vvX.Y.Z' + RAW="${{ steps.semver.outputs.version }}" + VERSION_NO_V="${RAW#v}" + + TAG="v${VERSION_NO_V}" + echo "TAG=${TAG}" + + # If tag already exists, skip creation to avoid failure + if git rev-parse -q --verify "refs/tags/${TAG}" >/dev/null; then + echo "Tag ${TAG} already exists; skipping tag creation" + else + git tag -a "${TAG}" -m "Release ${TAG}" + git push origin "${TAG}" + fi + + # Export the tag for downstream steps + echo "tag=${TAG}" >> $GITHUB_OUTPUT + env: + CHARON_TOKEN: ${{ secrets.CHARON_TOKEN }} + + - name: Determine tag + id: determine_tag + run: | + # Prefer created tag output; if empty fallback to semver version + TAG="${{ steps.create_tag.outputs.tag }}" + if [ -z "$TAG" ]; then + # semver.version contains a tag value like 'vX.Y.Z' or fallback 'v0.0.0' + VERSION_RAW="${{ steps.semver.outputs.version }}" + VERSION_NO_V="${VERSION_RAW#v}" + TAG="v${VERSION_NO_V}" + fi + echo "Determined tag: $TAG" + echo "tag=$TAG" >> $GITHUB_OUTPUT + + - name: Check for existing GitHub Release + id: check_release + run: | + TAG=${{ steps.determine_tag.outputs.tag }} + echo "Checking for release for tag: ${TAG}" + STATUS=$(curl -s -o /dev/null -w "%{http_code}" -H "Authorization: token ${CHARON_TOKEN}" -H "Accept: application/vnd.github+json" "https://api.github.com/repos/${GITHUB_REPOSITORY}/releases/tags/${TAG}") || true + if [ "${STATUS}" = "200" ]; then + echo "exists=true" >> $GITHUB_OUTPUT + else + echo "exists=false" >> $GITHUB_OUTPUT + fi + env: + CHARON_TOKEN: ${{ secrets.CHARON_TOKEN }} + + - name: Create GitHub Release (tag-only, no workspace changes) + if: ${{ steps.semver.outputs.changed == 'true' && steps.check_release.outputs.exists == 'false' }} + uses: softprops/action-gh-release@a06a81a03ee405af7f2048a818ed3f03bbf83c7b # v2 + with: + tag_name: ${{ steps.determine_tag.outputs.tag }} + name: Release ${{ steps.determine_tag.outputs.tag }} + generate_release_notes: true + make_latest: false + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml new file mode 100644 index 00000000..c8c10cf4 --- /dev/null +++ b/.github/workflows/benchmark.yml @@ -0,0 +1,63 @@ +name: Go Benchmark + +on: + push: + branches: + - main + - development + paths: + - 'backend/**' + pull_request: + branches: + - main + - development + paths: + - 'backend/**' + workflow_dispatch: + +permissions: + contents: write + deployments: write + +jobs: + benchmark: + name: Performance Regression Check + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Set up Go + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6 + with: + go-version: '1.25.5' + cache-dependency-path: backend/go.sum + + - name: Run Benchmark + working-directory: backend + run: go test -bench=. -benchmem -run='^$' ./... | tee output.txt + + - name: Store Benchmark Result + uses: benchmark-action/github-action-benchmark@v1 + with: + name: Go Benchmark + tool: 'go' + output-file-path: backend/output.txt + github-token: ${{ secrets.GITHUB_TOKEN }} + auto-push: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }} + # Show alert with commit comment on detection of performance regression + alert-threshold: '150%' + comment-on-alert: true + fail-on-alert: false + # Enable Job Summary for PRs + summary-always: true + + - name: Run Perf Asserts + working-directory: backend + env: + PERF_MAX_MS_GETSTATUS_P95: 500ms + PERF_MAX_MS_GETSTATUS_P95_PARALLEL: 1500ms + PERF_MAX_MS_LISTDECISIONS_P95: 2000ms + run: | + echo "## ๐Ÿ” Running performance assertions (TestPerf)" >> $GITHUB_STEP_SUMMARY + go test -run TestPerf -v ./internal/api/handlers -count=1 | tee perf-output.txt + exit ${PIPESTATUS[0]} diff --git a/.github/workflows/caddy-major-monitor.yml b/.github/workflows/caddy-major-monitor.yml new file mode 100644 index 00000000..74a1921b --- /dev/null +++ b/.github/workflows/caddy-major-monitor.yml @@ -0,0 +1,62 @@ +name: Monitor Caddy Major Release + +on: + schedule: + - cron: '17 7 * * 1' # Mondays at 07:17 UTC + workflow_dispatch: {} + +permissions: + contents: read + issues: write + +jobs: + check-caddy-major: + runs-on: ubuntu-latest + steps: + - name: Check for Caddy v3 and open issue + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const upstream = { owner: 'caddyserver', repo: 'caddy' }; + const { data: releases } = await github.rest.repos.listReleases({ + ...upstream, + per_page: 50, + }); + const latestV3 = releases.find(r => /^v3\./.test(r.tag_name)); + if (!latestV3) { + core.info('No Caddy v3 release detected.'); + return; + } + + const issueTitle = `Track upgrade to Caddy v3 (${latestV3.tag_name})`; + + const { data: existing } = await github.rest.issues.listForRepo({ + owner: context.repo.owner, + repo: context.repo.repo, + state: 'open', + per_page: 100, + }); + + if (existing.some(i => i.title === issueTitle)) { + core.info('Issue already exists โ€” nothing to do.'); + return; + } + + const body = [ + 'Caddy v3 has been released upstream and detected by the scheduled monitor.', + '', + `Detected release: ${latestV3.tag_name} (${latestV3.html_url})`, + '', + '- Create a feature branch to evaluate the v3 migration.', + '- Review breaking changes and update Docker base images/workflows.', + '- Validate Trivy scans and update any policies as needed.', + '', + 'Current policy: remain on latest 2.x until v3 is validated.' + ].join('\n'); + + await github.rest.issues.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: issueTitle, + body, + }); diff --git a/.github/workflows/codecov-upload.yml b/.github/workflows/codecov-upload.yml new file mode 100644 index 00000000..d16b2192 --- /dev/null +++ b/.github/workflows/codecov-upload.yml @@ -0,0 +1,77 @@ +name: Upload Coverage to Codecov (Push only) + +on: + push: + branches: + - main + - development + - 'feature/**' + +permissions: + contents: read + +jobs: + backend-codecov: + name: Backend Codecov Upload + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Set up Go + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6 + with: + go-version: '1.25.5' + cache-dependency-path: backend/go.sum + + - name: Run Go tests with coverage + working-directory: ${{ github.workspace }} + env: + CGO_ENABLED: 1 + run: | + bash scripts/go-test-coverage.sh 2>&1 | tee backend/test-output.txt + exit ${PIPESTATUS[0]} + + - name: Upload backend coverage to Codecov + uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5 + with: + token: ${{ secrets.CODECOV_TOKEN }} + files: ./backend/coverage.txt + flags: backend + fail_ci_if_error: true + + frontend-codecov: + name: Frontend Codecov Upload + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Set up Node.js + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6 + with: + node-version: '24.11.1' + cache: 'npm' + cache-dependency-path: frontend/package-lock.json + + - name: Install dependencies + working-directory: frontend + run: npm ci + + - name: Run frontend tests and coverage + working-directory: ${{ github.workspace }} + run: | + bash scripts/frontend-test-coverage.sh 2>&1 | tee frontend/test-output.txt + exit ${PIPESTATUS[0]} + + - name: Upload frontend coverage to Codecov + uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5 + with: + token: ${{ secrets.CODECOV_TOKEN }} + directory: ./frontend/coverage + flags: frontend + fail_ci_if_error: true diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml new file mode 100644 index 00000000..8194f3f0 --- /dev/null +++ b/.github/workflows/codeql.yml @@ -0,0 +1,53 @@ +name: CodeQL - Analyze + +on: + push: + branches: [ main, development, 'feature/**' ] + pull_request: + branches: [ main, development ] + schedule: + - cron: '0 3 * * 1' + +permissions: + contents: read + security-events: write + actions: read + pull-requests: read + +jobs: + analyze: + name: CodeQL analysis (${{ matrix.language }}) + runs-on: ubuntu-latest + # Skip forked PRs where CPMP_TOKEN lacks security-events permissions + if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false + permissions: + contents: read + security-events: write + actions: read + pull-requests: read + strategy: + fail-fast: false + matrix: + language: [ 'go', 'javascript-typescript' ] + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Initialize CodeQL + uses: github/codeql-action/init@cf1bb45a277cb3c205638b2cd5c984db1c46a412 # v4 + with: + languages: ${{ matrix.language }} + + - name: Setup Go + if: matrix.language == 'go' + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6 + with: + go-version: '1.25.5' + + - name: Autobuild + uses: github/codeql-action/autobuild@cf1bb45a277cb3c205638b2cd5c984db1c46a412 # v4 + + - name: Perform CodeQL Analysis + uses: github/codeql-action/analyze@cf1bb45a277cb3c205638b2cd5c984db1c46a412 # v4 + with: + category: "/language:${{ matrix.language }}" diff --git a/.github/workflows/create-labels.yml b/.github/workflows/create-labels.yml new file mode 100644 index 00000000..21670aac --- /dev/null +++ b/.github/workflows/create-labels.yml @@ -0,0 +1,78 @@ +name: Create Project Labels + +# This workflow only runs manually to set up labels +on: + workflow_dispatch: + +jobs: + create-labels: + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - name: Create all project labels + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const labels = [ + // Priority labels + { name: 'critical', color: 'B60205', description: 'Must have for the release, blocks other work' }, + { name: 'high', color: 'D93F0B', description: 'Important feature, should be included' }, + { name: 'medium', color: 'FBCA04', description: 'Nice to have, can be deferred' }, + { name: 'low', color: '0E8A16', description: 'Future enhancement, not urgent' }, + + // Milestone labels + { name: 'alpha', color: '5319E7', description: 'Part of initial alpha release' }, + { name: 'beta', color: '0052CC', description: 'Part of beta release' }, + { name: 'post-beta', color: '006B75', description: 'Post-beta enhancement' }, + + // Category labels + { name: 'architecture', color: 'C5DEF5', description: 'System design and structure' }, + { name: 'backend', color: '1D76DB', description: 'Server-side code' }, + { name: 'frontend', color: '5EBEFF', description: 'UI/UX code' }, + { name: 'feature', color: 'A2EEEF', description: 'New functionality' }, + { name: 'security', color: 'EE0701', description: 'Security-related' }, + { name: 'ssl', color: 'F9D0C4', description: 'SSL/TLS certificates' }, + { name: 'sso', color: 'D4C5F9', description: 'Single Sign-On' }, + { name: 'waf', color: 'B60205', description: 'Web Application Firewall' }, + { name: 'crowdsec', color: 'FF6B6B', description: 'CrowdSec integration' }, + { name: 'caddy', color: '1F6FEB', description: 'Caddy-specific' }, + { name: 'database', color: '006B75', description: 'Database-related' }, + { name: 'ui', color: '7057FF', description: 'User interface' }, + { name: 'deployment', color: '0E8A16', description: 'Docker, installation' }, + { name: 'monitoring', color: 'FEF2C0', description: 'Logging and statistics' }, + { name: 'documentation', color: '0075CA', description: 'Docs and guides' }, + { name: 'testing', color: 'BFD4F2', description: 'Test suite' }, + { name: 'performance', color: 'EDEDED', description: 'Optimization' }, + { name: 'community', color: 'D876E3', description: 'Community building' }, + { name: 'plus', color: 'FFD700', description: 'Premium/"Plus" feature' }, + { name: 'enterprise', color: '8B4513', description: 'Enterprise-grade feature' } + ]; + + for (const label of labels) { + try { + await github.rest.issues.createLabel({ + owner: context.repo.owner, + repo: context.repo.repo, + name: label.name, + color: label.color, + description: label.description + }); + console.log(`โœ“ Created label: ${label.name}`); + } catch (error) { + if (error.status === 422) { + console.log(`โš  Label already exists: ${label.name}`); + // Update the label if it exists + await github.rest.issues.updateLabel({ + owner: context.repo.owner, + repo: context.repo.repo, + name: label.name, + color: label.color, + description: label.description + }); + console.log(`โœ“ Updated label: ${label.name}`); + } else { + console.error(`โœ— Error creating label ${label.name}:`, error.message); + } + } + } diff --git a/.github/workflows/docker-build.yml b/.github/workflows/docker-build.yml new file mode 100644 index 00000000..66bf4376 --- /dev/null +++ b/.github/workflows/docker-build.yml @@ -0,0 +1,268 @@ +name: Docker Build, Publish & Test + +on: + push: + branches: + - main + - development + - feature/beta-release + # Note: Tags are handled by release-goreleaser.yml to avoid duplicate builds + pull_request: + branches: + - main + - development + - feature/beta-release + workflow_dispatch: + workflow_call: + +env: + REGISTRY: ghcr.io + IMAGE_NAME: ${{ github.repository_owner }}/charon + +jobs: + build-and-push: + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + packages: write + security-events: write + + outputs: + skip_build: ${{ steps.skip.outputs.skip_build }} + digest: ${{ steps.build-and-push.outputs.digest }} + + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Normalize image name + run: | + IMAGE_NAME=$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]') + echo "IMAGE_NAME=${IMAGE_NAME}" >> $GITHUB_ENV + - name: Determine skip condition + id: skip + env: + ACTOR: ${{ github.actor }} + EVENT: ${{ github.event_name }} + HEAD_MSG: ${{ github.event.head_commit.message }} + REF: ${{ github.ref }} + run: | + should_skip=false + pr_title="" + if [ "$EVENT" = "pull_request" ]; then + pr_title=$(jq -r '.pull_request.title' "$GITHUB_EVENT_PATH" 2>/dev/null || echo '') + fi + if [ "$ACTOR" = "renovate[bot]" ]; then should_skip=true; fi + if echo "$HEAD_MSG" | grep -Ei '^chore\(deps' >/dev/null 2>&1; then should_skip=true; fi + if echo "$HEAD_MSG" | grep -Ei '^chore:' >/dev/null 2>&1; then should_skip=true; fi + if echo "$pr_title" | grep -Ei '^chore\(deps' >/dev/null 2>&1; then should_skip=true; fi + if echo "$pr_title" | grep -Ei '^chore:' >/dev/null 2>&1; then should_skip=true; fi + # Always build on beta-release branch to ensure artifacts for testing + if [[ "$REF" == "refs/heads/feature/beta-release" ]]; then + should_skip=false + echo "Force building on beta-release branch" + fi + + echo "skip_build=$should_skip" >> $GITHUB_OUTPUT + + - name: Set up QEMU + if: steps.skip.outputs.skip_build != 'true' + uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0 + - name: Set up Docker Buildx + if: steps.skip.outputs.skip_build != 'true' + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 + - name: Resolve Caddy base digest + if: steps.skip.outputs.skip_build != 'true' + id: caddy + run: | + docker pull caddy:2-alpine + DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' caddy:2-alpine) + echo "image=$DIGEST" >> $GITHUB_OUTPUT + + - name: Log in to Container Registry + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata (tags, labels) + if: steps.skip.outputs.skip_build != 'true' + id: meta + uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0 + with: + images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + tags: | + type=raw,value=latest,enable={{is_default_branch}} + type=raw,value=dev,enable=${{ github.ref == 'refs/heads/development' }} + type=raw,value=beta,enable=${{ github.ref == 'refs/heads/feature/beta-release' }} + type=raw,value=pr-${{ github.ref_name }},enable=${{ github.event_name == 'pull_request' }} + type=sha,format=short,enable=${{ github.event_name != 'pull_request' }} + - name: Build and push Docker image + if: steps.skip.outputs.skip_build != 'true' + id: build-and-push + uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6 + with: + context: . + platforms: ${{ github.event_name == 'pull_request' && 'linux/amd64' || 'linux/amd64,linux/arm64' }} + push: ${{ github.event_name != 'pull_request' }} + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + cache-from: type=gha + cache-to: type=gha,mode=max + build-args: | + VERSION=${{ steps.meta.outputs.version }} + BUILD_DATE=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.created'] }} + VCS_REF=${{ github.sha }} + CADDY_IMAGE=${{ steps.caddy.outputs.image }} + + - name: Run Trivy scan (table output) + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # 0.33.1 + with: + image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build-and-push.outputs.digest }} + format: 'table' + severity: 'CRITICAL,HIGH' + exit-code: '0' + continue-on-error: true + + - name: Run Trivy vulnerability scanner (SARIF) + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + id: trivy + uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # 0.33.1 + with: + image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build-and-push.outputs.digest }} + format: 'sarif' + output: 'trivy-results.sarif' + severity: 'CRITICAL,HIGH' + continue-on-error: true + + - name: Check Trivy SARIF exists + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + id: trivy-check + run: | + if [ -f trivy-results.sarif ]; then + echo "exists=true" >> $GITHUB_OUTPUT + else + echo "exists=false" >> $GITHUB_OUTPUT + fi + + - name: Upload Trivy results + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' && steps.trivy-check.outputs.exists == 'true' + uses: github/codeql-action/upload-sarif@cf1bb45a277cb3c205638b2cd5c984db1c46a412 # v4.31.7 + with: + sarif_file: 'trivy-results.sarif' + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Create summary + if: steps.skip.outputs.skip_build != 'true' + run: | + echo "## ๐ŸŽ‰ Docker Image Built Successfully!" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### ๐Ÿ“ฆ Image Details" >> $GITHUB_STEP_SUMMARY + echo "- **Registry**: GitHub Container Registry (ghcr.io)" >> $GITHUB_STEP_SUMMARY + echo "- **Repository**: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}" >> $GITHUB_STEP_SUMMARY + echo "- **Tags**: " >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + echo "${{ steps.meta.outputs.tags }}" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + + test-image: + name: Test Docker Image + needs: build-and-push + runs-on: ubuntu-latest + if: needs.build-and-push.outputs.skip_build != 'true' && github.event_name != 'pull_request' + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Normalize image name + run: | + raw="${{ github.repository_owner }}/${{ github.event.repository.name }}" + IMAGE_NAME=$(echo "$raw" | tr '[:upper:]' '[:lower:]') + echo "IMAGE_NAME=${IMAGE_NAME}" >> $GITHUB_ENV + - name: Determine image tag + id: tag + run: | + if [[ "${{ github.ref }}" == "refs/heads/main" ]]; then + echo "tag=latest" >> $GITHUB_OUTPUT + elif [[ "${{ github.ref }}" == "refs/heads/development" ]]; then + echo "tag=dev" >> $GITHUB_OUTPUT + elif [[ "${{ github.ref }}" == refs/tags/v* ]]; then + echo "tag=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT + else + echo "tag=sha-$(echo ${{ github.sha }} | cut -c1-7)" >> $GITHUB_OUTPUT + fi + + - name: Log in to GitHub Container Registry + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Pull Docker image + run: docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }} + - name: Create Docker Network + run: docker network create charon-test-net + + - name: Run Upstream Service (whoami) + run: | + docker run -d \ + --name whoami \ + --network charon-test-net \ + traefik/whoami + + - name: Run Charon Container + run: | + docker run -d \ + --name test-container \ + --network charon-test-net \ + -p 8080:8080 \ + -p 80:80 \ + ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }} + - name: Run Integration Test + run: ./scripts/integration-test.sh + + - name: Check container logs + if: always() + run: docker logs test-container + + - name: Stop container + if: always() + run: | + docker stop test-container whoami || true + docker rm test-container whoami || true + docker network rm charon-test-net || true + + - name: Create test summary + if: always() + run: | + echo "## ๐Ÿงช Docker Image Test Results" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "- **Image**: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }}" >> $GITHUB_STEP_SUMMARY + echo "- **Integration Test**: ${{ job.status == 'success' && 'โœ… Passed' || 'โŒ Failed' }}" >> $GITHUB_STEP_SUMMARY + + trivy-pr-app-only: + name: Trivy (PR) - App-only + runs-on: ubuntu-latest + if: github.event_name == 'pull_request' + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Build image locally for PR + run: | + docker build -t charon:pr-${{ github.sha }} . + + - name: Extract `charon` binary from image + run: | + CONTAINER=$(docker create charon:pr-${{ github.sha }}) + docker cp ${CONTAINER}:/app/charon ./charon_binary || true + docker rm ${CONTAINER} || true + + - name: Run Trivy filesystem scan on `charon` (fail PR on HIGH/CRITICAL) + run: | + docker run --rm -v $HOME/.cache/trivy:/root/.cache/trivy -v $PWD:/workdir aquasec/trivy:latest fs --exit-code 1 --severity CRITICAL,HIGH /workdir/charon_binary diff --git a/.github/workflows/docker-lint.yml b/.github/workflows/docker-lint.yml new file mode 100644 index 00000000..91fc80ff --- /dev/null +++ b/.github/workflows/docker-lint.yml @@ -0,0 +1,23 @@ +name: Docker Lint + +on: + push: + branches: [ main, development, 'feature/**' ] + paths: + - 'Dockerfile' + pull_request: + branches: [ main, development ] + paths: + - 'Dockerfile' + +jobs: + hadolint: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Run Hadolint + uses: hadolint/hadolint-action@2332a7b74a6de0dda2e2221d575162eba76ba5e5 # v3.3.0 + with: + dockerfile: Dockerfile + failure-threshold: warning diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml new file mode 100644 index 00000000..c6aded05 --- /dev/null +++ b/.github/workflows/docker-publish.yml @@ -0,0 +1,276 @@ +name: Docker Build, Publish & Test + +on: + push: + branches: + - main + - development + - feature/beta-release + # Note: Tags are handled by release-goreleaser.yml to avoid duplicate builds + pull_request: + branches: + - main + - development + - feature/beta-release + workflow_dispatch: + workflow_call: + +env: + REGISTRY: ghcr.io + IMAGE_NAME: ${{ github.repository_owner }}/charon + +jobs: + build-and-push: + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + packages: write + security-events: write + + outputs: + skip_build: ${{ steps.skip.outputs.skip_build }} + digest: ${{ steps.build-and-push.outputs.digest }} + + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Normalize image name + run: | + echo "IMAGE_NAME=$(echo "${{ env.IMAGE_NAME }}" | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV + + - name: Determine skip condition + id: skip + env: + ACTOR: ${{ github.actor }} + EVENT: ${{ github.event_name }} + HEAD_MSG: ${{ github.event.head_commit.message }} + REF: ${{ github.ref }} + run: | + should_skip=false + pr_title="" + if [ "$EVENT" = "pull_request" ]; then + pr_title=$(jq -r '.pull_request.title' "$GITHUB_EVENT_PATH" 2>/dev/null || echo '') + fi + if [ "$ACTOR" = "renovate[bot]" ]; then should_skip=true; fi + if echo "$HEAD_MSG" | grep -Ei '^chore\(deps' >/dev/null 2>&1; then should_skip=true; fi + if echo "$HEAD_MSG" | grep -Ei '^chore:' >/dev/null 2>&1; then should_skip=true; fi + if echo "$pr_title" | grep -Ei '^chore\(deps' >/dev/null 2>&1; then should_skip=true; fi + if echo "$pr_title" | grep -Ei '^chore:' >/dev/null 2>&1; then should_skip=true; fi + + # Always build on beta-release branch to ensure artifacts for testing + if [[ "$REF" == "refs/heads/feature/beta-release" ]]; then + should_skip=false + echo "Force building on beta-release branch" + fi + + echo "skip_build=$should_skip" >> $GITHUB_OUTPUT + + - name: Set up QEMU + if: steps.skip.outputs.skip_build != 'true' + uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0 + + - name: Set up Docker Buildx + if: steps.skip.outputs.skip_build != 'true' + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 + + - name: Resolve Caddy base digest + if: steps.skip.outputs.skip_build != 'true' + id: caddy + run: | + docker pull caddy:2-alpine + DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' caddy:2-alpine) + echo "image=$DIGEST" >> $GITHUB_OUTPUT + + - name: Log in to Container Registry + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Extract metadata (tags, labels) + if: steps.skip.outputs.skip_build != 'true' + id: meta + uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5.10.0 + with: + images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} + tags: | + type=raw,value=latest,enable={{is_default_branch}} + type=raw,value=dev,enable=${{ github.ref == 'refs/heads/development' }} + type=raw,value=beta,enable=${{ github.ref == 'refs/heads/feature/beta-release' }} + type=raw,value=pr-${{ github.ref_name }},enable=${{ github.event_name == 'pull_request' }} + type=sha,format=short,enable=${{ github.event_name != 'pull_request' }} + + - name: Build and push Docker image + if: steps.skip.outputs.skip_build != 'true' + id: build-and-push + uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6 + with: + context: . + platforms: ${{ github.event_name == 'pull_request' && 'linux/amd64' || 'linux/amd64,linux/arm64' }} + push: ${{ github.event_name != 'pull_request' }} + tags: ${{ steps.meta.outputs.tags }} + labels: ${{ steps.meta.outputs.labels }} + cache-from: type=gha + cache-to: type=gha,mode=max + build-args: | + VERSION=${{ steps.meta.outputs.version }} + BUILD_DATE=${{ fromJSON(steps.meta.outputs.json).labels['org.opencontainers.image.created'] }} + VCS_REF=${{ github.sha }} + CADDY_IMAGE=${{ steps.caddy.outputs.image }} + + - name: Run Trivy scan (table output) + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # 0.33.1 + with: + image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build-and-push.outputs.digest }} + format: 'table' + severity: 'CRITICAL,HIGH' + exit-code: '0' + continue-on-error: true + + - name: Run Trivy vulnerability scanner (SARIF) + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + id: trivy + uses: aquasecurity/trivy-action@b6643a29fecd7f34b3597bc6acb0a98b03d33ff8 # 0.33.1 + with: + image-ref: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@${{ steps.build-and-push.outputs.digest }} + format: 'sarif' + output: 'trivy-results.sarif' + severity: 'CRITICAL,HIGH' + continue-on-error: true + + - name: Check Trivy SARIF exists + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' + id: trivy-check + run: | + if [ -f trivy-results.sarif ]; then + echo "exists=true" >> $GITHUB_OUTPUT + else + echo "exists=false" >> $GITHUB_OUTPUT + fi + + - name: Upload Trivy results + if: github.event_name != 'pull_request' && steps.skip.outputs.skip_build != 'true' && steps.trivy-check.outputs.exists == 'true' + uses: github/codeql-action/upload-sarif@cf1bb45a277cb3c205638b2cd5c984db1c46a412 # v4.31.7 + with: + sarif_file: 'trivy-results.sarif' + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Create summary + if: steps.skip.outputs.skip_build != 'true' + run: | + echo "## ๐ŸŽ‰ Docker Image Built Successfully!" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### ๐Ÿ“ฆ Image Details" >> $GITHUB_STEP_SUMMARY + echo "- **Registry**: GitHub Container Registry (ghcr.io)" >> $GITHUB_STEP_SUMMARY + echo "- **Repository**: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}" >> $GITHUB_STEP_SUMMARY + echo "- **Tags**: " >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + echo "${{ steps.meta.outputs.tags }}" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + + test-image: + name: Test Docker Image + needs: build-and-push + runs-on: ubuntu-latest + if: needs.build-and-push.outputs.skip_build != 'true' && github.event_name != 'pull_request' + + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Normalize image name + run: | + raw="${{ github.repository_owner }}/${{ github.event.repository.name }}" + echo "IMAGE_NAME=$(echo "$raw" | tr '[:upper:]' '[:lower:]')" >> $GITHUB_ENV + + - name: Determine image tag + id: tag + run: | + if [[ "${{ github.ref }}" == "refs/heads/main" ]]; then + echo "tag=latest" >> $GITHUB_OUTPUT + elif [[ "${{ github.ref }}" == "refs/heads/development" ]]; then + echo "tag=dev" >> $GITHUB_OUTPUT + elif [[ "${{ github.ref }}" == refs/tags/v* ]]; then + echo "tag=${GITHUB_REF#refs/tags/v}" >> $GITHUB_OUTPUT + else + echo "tag=sha-$(echo ${{ github.sha }} | cut -c1-7)" >> $GITHUB_OUTPUT + fi + + - name: Log in to GitHub Container Registry + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Pull Docker image + run: docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }} + + - name: Create Docker Network + run: docker network create charon-test-net + + - name: Run Upstream Service (whoami) + run: | + docker run -d \ + --name whoami \ + --network charon-test-net \ + traefik/whoami + + - name: Run Charon Container + run: | + docker run -d \ + --name test-container \ + --network charon-test-net \ + -p 8080:8080 \ + -p 80:80 \ + ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }} + + - name: Run Integration Test + run: ./scripts/integration-test.sh + + - name: Check container logs + if: always() + run: docker logs test-container + + - name: Stop container + if: always() + run: | + docker stop test-container whoami || true + docker rm test-container whoami || true + docker network rm charon-test-net || true + + - name: Create test summary + if: always() + run: | + echo "## ๐Ÿงช Docker Image Test Results" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "- **Image**: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }}" >> $GITHUB_STEP_SUMMARY + echo "- **Integration Test**: ${{ job.status == 'success' && 'โœ… Passed' || 'โŒ Failed' }}" >> $GITHUB_STEP_SUMMARY + + trivy-pr-app-only: + name: Trivy (PR) - App-only + runs-on: ubuntu-latest + if: github.event_name == 'pull_request' + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Build image locally for PR + run: | + docker build -t charon:pr-${{ github.sha }} . + + - name: Extract `charon` binary from image + run: | + CONTAINER=$(docker create charon:pr-${{ github.sha }}) + docker cp ${CONTAINER}:/app/charon ./charon_binary || true + docker rm ${CONTAINER} || true + + - name: Run Trivy filesystem scan on `charon` (fail PR on HIGH/CRITICAL) + run: | + docker run --rm -v $HOME/.cache/trivy:/root/.cache/trivy -v $PWD:/workdir aquasec/trivy:latest fs --exit-code 1 --severity CRITICAL,HIGH /workdir/charon_binary + shell: bash diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml new file mode 100644 index 00000000..3e1366ec --- /dev/null +++ b/.github/workflows/docs.yml @@ -0,0 +1,353 @@ +name: Deploy Documentation to GitHub Pages + +on: + push: + branches: + - main # Deploy docs when pushing to main + paths: + - 'docs/**' # Only run if docs folder changes + - 'README.md' # Or if README changes + - '.github/workflows/docs.yml' # Or if this workflow changes + workflow_dispatch: # Allow manual trigger + +# Sets permissions to allow deployment to GitHub Pages +permissions: + contents: read + pages: write + id-token: write + +# Allow only one concurrent deployment +concurrency: + group: "pages" + cancel-in-progress: false + +jobs: + build: + name: Build Documentation + runs-on: ubuntu-latest + + steps: + # Step 1: Get the code + - name: ๐Ÿ“ฅ Checkout code + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + # Step 2: Set up Node.js (for building any JS-based doc tools) + - name: ๐Ÿ”ง Set up Node.js + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6 + with: + node-version: '24.11.1' + + # Step 3: Create a beautiful docs site structure + - name: ๐Ÿ“ Build documentation site + run: | + # Create output directory + mkdir -p _site + + # Copy all markdown files + cp README.md _site/ + cp -r docs _site/ + + # Create a simple HTML index that looks nice + cat > _site/index.html << 'EOF' + + + + + + Charon - Documentation + + + + +
+

๐Ÿš€ Charon

+

Make your websites easy to reach - No coding required!

+
+ +
+
+

๐Ÿ‘‹ Welcome!

+

+ This documentation will help you get started with Charon. + Whether you're a complete beginner or an experienced developer, we've got you covered! +

+
+ +

๐Ÿ“š Getting Started

+
+
+

๐Ÿ  Getting Started Guide Start Here

+

Your first setup in just 5 minutes! We'll walk you through everything step by step.

+ Read the Guide โ†’ +
+ +
+

๐Ÿ“– README Essential

+

Learn what the app does, how to install it, and see examples of what you can build.

+ Read More โ†’ +
+ +
+

๐Ÿ“ฅ Import Guide

+

Already using Caddy? Learn how to bring your existing configuration into the app.

+ Import Your Configs โ†’ +
+
+ +

๐Ÿ”ง Developer Documentation

+
+
+

๐Ÿ”Œ API Reference Advanced

+

Complete REST API documentation with examples in JavaScript and Python.

+ View API Docs โ†’ +
+ +
+

๐Ÿ’พ Database Schema Advanced

+

Understand how data is stored, relationships, and backup strategies.

+ View Schema โ†’ +
+ +
+

โœจ Contributing Guide

+

Want to help make this better? Learn how to contribute code, docs, or ideas.

+ Start Contributing โ†’ +
+
+ +

๐Ÿ“‹ All Documentation

+
+

๐Ÿ“š Documentation Index

+

Browse all available documentation organized by topic and skill level.

+ View Full Index โ†’ +
+ +

๐Ÿ†˜ Need Help?

+
+

Get Support

+

+ Stuck? Have questions? We're here to help! +

+ +
+
+ +
+

Built with โค๏ธ by @Wikid82

+

Made for humans, not just techies!

+
+ + + EOF + + # Convert markdown files to HTML using a simple converter + npm install -g marked + + # Convert each markdown file + for file in _site/docs/*.md; do + if [ -f "$file" ]; then + filename=$(basename "$file" .md) + marked "$file" -o "_site/docs/${filename}.html" --gfm + fi + done + + # Convert README and CONTRIBUTING + marked _site/README.md -o _site/README.html --gfm + if [ -f "CONTRIBUTING.md" ]; then + cp CONTRIBUTING.md _site/ + marked _site/CONTRIBUTING.md -o _site/CONTRIBUTING.html --gfm + fi + + # Add simple styling to all HTML files + for html_file in _site/*.html _site/docs/*.html; do + if [ -f "$html_file" ] && [ "$html_file" != "_site/index.html" ]; then + # Add a header with navigation to each page + temp_file="${html_file}.tmp" + cat > "$temp_file" << 'HEADER' + + + + + + Caddy Proxy Manager Plus - Documentation + + + + + +
+ HEADER + + # Append original content + cat "$html_file" >> "$temp_file" + + # Add footer + cat >> "$temp_file" << 'FOOTER' +
+
+

Caddy Proxy Manager Plus - Built with โค๏ธ for the community

+
+ + + FOOTER + + mv "$temp_file" "$html_file" + fi + done + + echo "โœ… Documentation site built successfully!" + + # Step 4: Upload the built site + - name: ๐Ÿ“ค Upload artifact + uses: actions/upload-pages-artifact@7b1f4a764d45c48632c6b24a0339c27f5614fb0b # v4 + with: + path: '_site' + + deploy: + name: Deploy to GitHub Pages + environment: + name: github-pages + url: ${{ steps.deployment.outputs.page_url }} + runs-on: ubuntu-latest + needs: build + + steps: + # Deploy to GitHub Pages + - name: ๐Ÿš€ Deploy to GitHub Pages + id: deployment + uses: actions/deploy-pages@d6db90164ac5ed86f2b6aed7e0febac5b3c0c03e # v4 + + # Create a summary + - name: ๐Ÿ“‹ Create deployment summary + run: | + echo "## ๐ŸŽ‰ Documentation Deployed!" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "Your documentation is now live at:" >> $GITHUB_STEP_SUMMARY + echo "๐Ÿ”— ${{ steps.deployment.outputs.page_url }}" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### ๐Ÿ“š What's Included" >> $GITHUB_STEP_SUMMARY + echo "- Getting Started Guide" >> $GITHUB_STEP_SUMMARY + echo "- Complete README" >> $GITHUB_STEP_SUMMARY + echo "- API Documentation" >> $GITHUB_STEP_SUMMARY + echo "- Database Schema" >> $GITHUB_STEP_SUMMARY + echo "- Import Guide" >> $GITHUB_STEP_SUMMARY + echo "- Contributing Guidelines" >> $GITHUB_STEP_SUMMARY diff --git a/.github/workflows/dry-run-history-rewrite.yml b/.github/workflows/dry-run-history-rewrite.yml new file mode 100644 index 00000000..77a56460 --- /dev/null +++ b/.github/workflows/dry-run-history-rewrite.yml @@ -0,0 +1,34 @@ +name: History Rewrite Dry-Run + +on: + pull_request: + types: [opened, synchronize, reopened] + schedule: + - cron: '0 2 * * *' # daily at 02:00 UTC + workflow_dispatch: + +permissions: + contents: read + +jobs: + preview-history: + name: Dry-run preview for history rewrite + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Debug git info + run: | + git --version + git rev-parse --is-shallow-repository || true + git status --porcelain + + - name: Make CI script executable + run: chmod +x scripts/ci/dry_run_history_rewrite.sh + + - name: Run dry-run history check + run: | + scripts/ci/dry_run_history_rewrite.sh --paths 'backend/codeql-db,codeql-db,codeql-db-js,codeql-db-go' --strip-size 50 diff --git a/.github/workflows/history-rewrite-tests.yml b/.github/workflows/history-rewrite-tests.yml new file mode 100644 index 00000000..d2f9bf72 --- /dev/null +++ b/.github/workflows/history-rewrite-tests.yml @@ -0,0 +1,32 @@ +name: History Rewrite Tests + +on: + push: + paths: + - 'scripts/history-rewrite/**' + - '.github/workflows/history-rewrite-tests.yml' + pull_request: + paths: + - 'scripts/history-rewrite/**' + +jobs: + test: + runs-on: ubuntu-latest + steps: + - name: Checkout with full history + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y bats shellcheck + + - name: Run Bats tests + run: | + bats ./scripts/history-rewrite/tests || exit 1 + + - name: ShellCheck scripts + run: | + shellcheck scripts/history-rewrite/*.sh || true diff --git a/.github/workflows/pr-checklist.yml b/.github/workflows/pr-checklist.yml new file mode 100644 index 00000000..ff914b5c --- /dev/null +++ b/.github/workflows/pr-checklist.yml @@ -0,0 +1,48 @@ +name: PR Checklist Validation (History Rewrite) + +on: + pull_request: + types: [opened, edited, synchronize] + +jobs: + validate: + name: Validate history-rewrite checklist (conditional) + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Validate PR checklist (only for history-rewrite changes) + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const prNumber = context.issue.number; + const pr = await github.rest.pulls.get({owner, repo, pull_number: prNumber}); + const body = (pr.data && pr.data.body) || ''; + + // Determine if this PR modifies history-rewrite related files + const filesResp = await github.rest.pulls.listFiles({ owner, repo, pull_number: prNumber }); + const files = filesResp.data.map(f => f.filename.toLowerCase()); + const relevant = files.some(fn => fn.startsWith('scripts/history-rewrite/') || fn.startsWith('docs/plans/history_rewrite.md') || fn.includes('history-rewrite')); + if (!relevant) { + core.info('No history-rewrite related files changed; skipping checklist validation.'); + return; + } + + // Use a set of named checks with robust regex patterns for checkbox and phrase variants + const checks = [ + { name: 'preview_removals.sh mention', pattern: /preview_removals\.sh/i }, + { name: 'data/backups mention', pattern: /data\/?backups/i }, + // Accept checked checkbox variants and inline code/backtick usage for the '--force' phrase + { name: 'explicit non-run of --force', pattern: /(?:\[\s*[xX]\s*\]\s*)?(?:i will not run|will not run|do not run|don'?t run|won'?t run)\b[^\n]*--force/i }, + ]; + + const missing = checks.filter(c => !c.pattern.test(body)).map(c => c.name); + if (missing.length > 0) { + // Post a comment to the PR with instructions for filling the checklist + const commentBody = `Hi! This PR touches history-rewrite artifacts and requires the checklist in .github/PULL_REQUEST_TEMPLATE/history-rewrite.md. The following items are missing in your PR body: ${missing.join(', ')}\n\nPlease update the PR description using the history-rewrite template and re-run checks.`; + await github.rest.issues.createComment({ owner, repo, issue_number: prNumber, body: commentBody }); + core.setFailed('Missing required checklist items: ' + missing.join(', ')); + } diff --git a/.github/workflows/propagate-changes.yml b/.github/workflows/propagate-changes.yml new file mode 100644 index 00000000..1a193bc8 --- /dev/null +++ b/.github/workflows/propagate-changes.yml @@ -0,0 +1,161 @@ +name: Propagate Changes Between Branches + +on: + push: + branches: + - main + - development + +permissions: + contents: write + pull-requests: write + issues: write + +jobs: + propagate: + name: Create PR to synchronize branches + runs-on: ubuntu-latest + if: github.actor != 'github-actions[bot]' && github.event.pusher != null + steps: + - name: Set up Node (for github-script) + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6 + with: + node-version: '24.11.1' + + - name: Propagate Changes + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + script: | + const currentBranch = context.ref.replace('refs/heads/', ''); + + async function createPR(src, base) { + if (src === base) return; + + core.info(`Checking propagation from ${src} to ${base}...`); + + // Check for existing open PRs + const { data: pulls } = await github.rest.pulls.list({ + owner: context.repo.owner, + repo: context.repo.repo, + state: 'open', + head: `${context.repo.owner}:${src}`, + base: base, + }); + + if (pulls.length > 0) { + core.info(`Existing PR found for ${src} -> ${base}. Skipping.`); + return; + } + + // Compare commits to see if src is ahead of base + try { + const compare = await github.rest.repos.compareCommits({ + owner: context.repo.owner, + repo: context.repo.repo, + base: base, + head: src, + }); + + // If src is not ahead, nothing to merge + if (compare.data.ahead_by === 0) { + core.info(`${src} is not ahead of ${base}. No propagation needed.`); + return; + } + + // If files changed include history-rewrite or other sensitive scripts, + // avoid automatic propagation. This prevents bypassing checklist validation + // and manual review for potentially destructive changes. + let files = (compare.data.files || []).map(f => (f.filename || '').toLowerCase()); + + // Fallback: if compare.files is empty/truncated, aggregate files from the commit list + if (files.length === 0 && Array.isArray(compare.data.commits) && compare.data.commits.length > 0) { + for (const commit of compare.data.commits) { + const commitData = await github.rest.repos.getCommit({ owner: context.repo.owner, repo: context.repo.repo, ref: commit.sha }); + for (const f of (commitData.data.files || [])) { + files.push((f.filename || '').toLowerCase()); + } + } + files = Array.from(new Set(files)); + } + + // Load propagation config (list of sensitive paths) from .github/propagate-config.yml when available + let configPaths = ['scripts/history-rewrite/', 'data/backups', 'docs/plans/history_rewrite.md', '.github/workflows/']; + try { + const configResp = await github.rest.repos.getContent({ owner: context.repo.owner, repo: context.repo.repo, path: '.github/propagate-config.yml', ref: src }); + const contentStr = Buffer.from(configResp.data.content, 'base64').toString('utf8'); + const lines = contentStr.split(/\r?\n/); + let inSensitive = false; + const parsedPaths = []; + for (const line of lines) { + const trimmed = line.trim(); + if (!inSensitive && trimmed.startsWith('sensitive_paths:')) { inSensitive = true; continue; } + if (inSensitive) { + if (trimmed.startsWith('-')) parsedPaths.push(trimmed.substring(1).trim()); + else if (trimmed.length === 0) continue; else break; + } + } + if (parsedPaths.length > 0) configPaths = parsedPaths.map(p => p.toLowerCase()); + } catch (err) { core.info('No .github/propagate-config.yml or parse failure; using defaults.'); } + + const sensitive = files.some(fn => configPaths.some(sp => fn.startsWith(sp) || fn.includes(sp))); + if (sensitive) { + core.info(`${src} -> ${base} contains sensitive changes (${files.join(', ')}). Skipping automatic propagation.`); + return; + } + } catch (error) { + // If base branch doesn't exist, etc. + core.warning(`Error comparing ${src} to ${base}: ${error.message}`); + return; + } + + // Create PR + try { + const pr = await github.rest.pulls.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `Propagate changes from ${src} into ${base}`, + head: src, + base: base, + body: `Automated PR to propagate changes from ${src} into ${base}.\n\nTriggered by push to ${currentBranch}.`, + draft: true, + }); + core.info(`Created PR #${pr.data.number} to merge ${src} into ${base}`); + // Add an 'auto-propagate' label to the created PR and create the label if missing + try { + try { + await github.rest.issues.getLabel({ owner: context.repo.owner, repo: context.repo.repo, name: 'auto-propagate' }); + } catch (e) { + await github.rest.issues.createLabel({ owner: context.repo.owner, repo: context.repo.repo, name: 'auto-propagate', color: '7dd3fc', description: 'Automatically created propagate PRs' }); + } + await github.rest.issues.addLabels({ owner: context.repo.owner, repo: context.repo.repo, issue_number: pr.data.number, labels: ['auto-propagate'] }); + } catch (labelErr) { + core.warning('Failed to ensure or add auto-propagate label: ' + labelErr.message); + } + } catch (error) { + core.warning(`Failed to create PR from ${src} to ${base}: ${error.message}`); + } + } + + if (currentBranch === 'main') { + // Main -> Development + await createPR('main', 'development'); + } else if (currentBranch === 'development') { + // Development -> Feature branches + const branches = await github.paginate(github.rest.repos.listBranches, { + owner: context.repo.owner, + repo: context.repo.repo, + }); + + const featureBranches = branches + .map(b => b.name) + .filter(name => name.startsWith('feature/')); + + core.info(`Found ${featureBranches.length} feature branches: ${featureBranches.join(', ')}`); + + for (const featureBranch of featureBranches) { + await createPR('development', featureBranch); + } + } + env: + CHARON_TOKEN: ${{ secrets.CHARON_TOKEN }} + CPMP_TOKEN: ${{ secrets.CPMP_TOKEN }} diff --git a/.github/workflows/quality-checks.yml b/.github/workflows/quality-checks.yml new file mode 100644 index 00000000..c2d98376 --- /dev/null +++ b/.github/workflows/quality-checks.yml @@ -0,0 +1,170 @@ +name: Quality Checks + +on: + push: + branches: [ main, development, 'feature/**' ] + pull_request: + branches: [ main, development ] + +jobs: + backend-quality: + name: Backend (Go) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Set up Go + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0 + with: + go-version: '1.25.5' + cache-dependency-path: backend/go.sum + + - name: Repo health check + run: | + bash scripts/repo_health_check.sh + + - name: Run Go tests + id: go-tests + working-directory: ${{ github.workspace }} + env: + CGO_ENABLED: 1 + run: | + bash scripts/go-test-coverage.sh 2>&1 | tee backend/test-output.txt + exit ${PIPESTATUS[0]} + + - name: Go Test Summary + if: always() + working-directory: backend + run: | + echo "## ๐Ÿ”ง Backend Test Results" >> $GITHUB_STEP_SUMMARY + if [ "${{ steps.go-tests.outcome }}" == "success" ]; then + echo "โœ… **All tests passed**" >> $GITHUB_STEP_SUMMARY + PASS_COUNT=$(grep -c "^--- PASS" test-output.txt || echo "0") + echo "- Tests passed: $PASS_COUNT" >> $GITHUB_STEP_SUMMARY + else + echo "โŒ **Tests failed**" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Failed Tests:" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + grep -E "^--- FAIL|FAIL\s+github" test-output.txt || echo "See logs for details" + grep -E "^--- FAIL|FAIL\s+github" test-output.txt >> $GITHUB_STEP_SUMMARY || echo "See logs for details" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + fi + + # Codecov upload moved to `codecov-upload.yml` which is push-only. + + + - name: Run golangci-lint + uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0 + with: + version: latest + working-directory: backend + args: --timeout=5m + continue-on-error: true + + - name: Run Perf Asserts + working-directory: backend + env: + # Conservative defaults to avoid flakiness on CI; tune as necessary + PERF_MAX_MS_GETSTATUS_P95: 500ms + PERF_MAX_MS_GETSTATUS_P95_PARALLEL: 1500ms + PERF_MAX_MS_LISTDECISIONS_P95: 2000ms + run: | + echo "## ๐Ÿ” Running performance assertions (TestPerf)" >> $GITHUB_STEP_SUMMARY + go test -run TestPerf -v ./internal/api/handlers -count=1 | tee perf-output.txt + exit ${PIPESTATUS[0]} + + frontend-quality: + name: Frontend (React) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Repo health check + run: | + bash scripts/repo_health_check.sh + + - name: Set up Node.js + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0 + with: + node-version: '24.11.1' + cache: 'npm' + cache-dependency-path: frontend/package-lock.json + + - name: Check if frontend was modified in PR + id: check-frontend + run: | + if [ "${{ github.event_name }}" = "push" ]; then + echo "frontend_changed=true" >> $GITHUB_OUTPUT + exit 0 + fi + # Try to fetch the PR base ref. This may fail for forked PRs or other cases. + git fetch origin ${{ github.event.pull_request.base.ref }} --depth=1 || true + + # Compute changed files against the PR base ref, fallback to origin/main, then fallback to last 10 commits + CHANGED=$(git diff --name-only origin/${{ github.event.pull_request.base.ref }}...HEAD 2>/dev/null || echo "") + echo "Changed files (base ref):\n$CHANGED" + + if [ -z "$CHANGED" ]; then + echo "Base ref diff empty or failed; fetching origin/main for fallback..." + git fetch origin main --depth=1 || true + CHANGED=$(git diff --name-only origin/main...HEAD 2>/dev/null || echo "") + echo "Changed files (main fallback):\n$CHANGED" + fi + + if [ -z "$CHANGED" ]; then + echo "Still empty; falling back to diffing last 10 commits from HEAD..." + CHANGED=$(git diff --name-only HEAD~10...HEAD 2>/dev/null || echo "") + echo "Changed files (HEAD~10 fallback):\n$CHANGED" + fi + + if echo "$CHANGED" | grep -q '^frontend/'; then + echo "frontend_changed=true" >> $GITHUB_OUTPUT + else + echo "frontend_changed=false" >> $GITHUB_OUTPUT + fi + + - name: Install dependencies + working-directory: frontend + if: ${{ github.event_name == 'push' || steps.check-frontend.outputs.frontend_changed == 'true' }} + run: npm ci + + - name: Run frontend tests and coverage + id: frontend-tests + working-directory: ${{ github.workspace }} + if: ${{ github.event_name == 'push' || steps.check-frontend.outputs.frontend_changed == 'true' }} + run: | + bash scripts/frontend-test-coverage.sh 2>&1 | tee frontend/test-output.txt + exit ${PIPESTATUS[0]} + + - name: Frontend Test Summary + if: always() + working-directory: frontend + run: | + echo "## โš›๏ธ Frontend Test Results" >> $GITHUB_STEP_SUMMARY + if [ "${{ steps.frontend-tests.outcome }}" == "success" ]; then + echo "โœ… **All tests passed**" >> $GITHUB_STEP_SUMMARY + # Extract test counts from vitest output + if grep -q "Tests:" test-output.txt; then + grep "Tests:" test-output.txt | tail -1 >> $GITHUB_STEP_SUMMARY + fi + else + echo "โŒ **Tests failed**" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Failed Tests:" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + # Extract failed test info from vitest output + grep -E "FAIL|โœ•|ร—|AssertionError|Error:" test-output.txt | head -30 >> $GITHUB_STEP_SUMMARY || echo "See logs for details" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + fi + + # Codecov upload moved to `codecov-upload.yml` which is push-only. + + + + - name: Run frontend lint + working-directory: frontend + run: npm run lint + continue-on-error: true diff --git a/.github/workflows/release-goreleaser.yml b/.github/workflows/release-goreleaser.yml new file mode 100644 index 00000000..c9068e89 --- /dev/null +++ b/.github/workflows/release-goreleaser.yml @@ -0,0 +1,59 @@ +name: Release (GoReleaser) + +on: + push: + tags: + - 'v*' + +permissions: + contents: write + packages: write + +jobs: + goreleaser: + runs-on: ubuntu-latest + env: + # Use the built-in CHARON_TOKEN by default for GitHub API operations. + # If you need to provide a PAT with elevated permissions, add a CHARON_TOKEN secret + # at the repo or organization level and update the env here accordingly. + CHARON_TOKEN: ${{ secrets.CHARON_TOKEN }} + steps: + - name: Checkout + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + + - name: Set up Go + uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6 + with: + go-version: '1.25.5' + + - name: Set up Node.js + uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6 + with: + node-version: '24.11.1' + + - name: Build Frontend + working-directory: frontend + run: | + # Inject version into frontend build from tag (if present) + VERSION=$${GITHUB_REF#refs/tags/} + echo "VITE_APP_VERSION=$$VERSION" >> $GITHUB_ENV + npm ci + npm run build + + - name: Install Cross-Compilation Tools (Zig) + uses: goto-bus-stop/setup-zig@v2 + with: + version: 0.13.0 + + # CHARON_TOKEN is set from CHARON_TOKEN or CPMP_TOKEN (fallback), defaulting to GITHUB_TOKEN + + + - name: Run GoReleaser + uses: goreleaser/goreleaser-action@e435ccd777264be153ace6237001ef4d979d3a7a # v6 + with: + distribution: goreleaser + version: latest + args: release --clean + # CGO settings are handled in .goreleaser.yaml via Zig diff --git a/.github/workflows/renovate.yml b/.github/workflows/renovate.yml new file mode 100644 index 00000000..9a379cee --- /dev/null +++ b/.github/workflows/renovate.yml @@ -0,0 +1,48 @@ +name: Renovate + +on: + schedule: + - cron: '0 5 * * *' # daily 05:00 EST + workflow_dispatch: + +permissions: + contents: write + pull-requests: write + issues: write + +jobs: + renovate: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 1 + - name: Choose Renovate Token + run: | + # Prefer explicit tokens (CHARON_TOKEN > CPMP_TOKEN) if provided; otherwise use the default GITHUB_TOKEN + if [ -n "${{ secrets.CHARON_TOKEN }}" ]; then + echo "Using CHARON_TOKEN" >&2 + echo "GITHUB_TOKEN=${{ secrets.CHARON_TOKEN }}" >> $GITHUB_ENV + elif [ -n "${{ secrets.CPMP_TOKEN }}" ]; then + echo "Using CPMP_TOKEN fallback" >&2 + echo "GITHUB_TOKEN=${{ secrets.CPMP_TOKEN }}" >> $GITHUB_ENV + else + echo "Using default GITHUB_TOKEN from Actions" >&2 + echo "GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}" >> $GITHUB_ENV + fi + + - name: Fail-fast if token not set + run: | + if [ -z "${{ env.GITHUB_TOKEN }}" ]; then + echo "ERROR: No Renovate token provided. Set CHARON_TOKEN, CPMP_TOKEN, or rely on default GITHUB_TOKEN." >&2 + exit 1 + fi + + - name: Run Renovate + uses: renovatebot/github-action@5712c6a41dea6cdf32c72d92a763bd417e6606aa # v44.0.5 + with: + configurationFile: .github/renovate.json + token: ${{ env.GITHUB_TOKEN }} + env: + LOG_LEVEL: info diff --git a/.github/workflows/renovate_prune.yml b/.github/workflows/renovate_prune.yml new file mode 100644 index 00000000..7089e435 --- /dev/null +++ b/.github/workflows/renovate_prune.yml @@ -0,0 +1,103 @@ +name: "Prune Renovate Branches" + +on: + workflow_dispatch: + schedule: + - cron: '0 3 * * *' # daily at 03:00 UTC + pull_request: + types: [closed] # also run when any PR is closed (makes pruning near-real-time) + +permissions: + contents: write # required to delete branch refs + pull-requests: read + +jobs: + prune: + runs-on: ubuntu-latest + concurrency: + group: prune-renovate-branches + cancel-in-progress: true + + env: + BRANCH_PREFIX: "renovate/" # adjust if you use a different prefix + + steps: + - name: Choose GitHub Token + run: | + if [ -n "${{ secrets.CHARON_TOKEN }}" ]; then + echo "Using CHARON_TOKEN" >&2 + echo "CHARON_TOKEN=${{ secrets.CHARON_TOKEN }}" >> $GITHUB_ENV + else + echo "Using CPMP_TOKEN fallback" >&2 + echo "CHARON_TOKEN=${{ secrets.CPMP_TOKEN }}" >> $GITHUB_ENV + fi + - name: Prune renovate branches + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8 + with: + github-token: ${{ env.CHARON_TOKEN }} + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const branchPrefix = (process.env.BRANCH_PREFIX || 'renovate/').replace(/^refs\/heads\//, ''); + const refPrefix = `heads/${branchPrefix}`; // e.g. "heads/renovate/" + + core.info(`Searching for refs with prefix: ${refPrefix}`); + + // List matching refs (branches) under the prefix + let refs; + try { + refs = await github.rest.git.listMatchingRefs({ + owner, + repo, + ref: refPrefix + }); + } catch (err) { + core.info(`No matching refs or API error: ${err.message}`); + refs = { data: [] }; + } + + for (const r of refs.data) { + const fullRef = r.ref; // "refs/heads/renovate/..." + const branchName = fullRef.replace('refs/heads/', ''); + core.info(`Evaluating branch: ${branchName}`); + + // Find PRs for this branch (head = "owner:branch") + const prs = await github.rest.pulls.list({ + owner, + repo, + head: `${owner}:${branchName}`, + state: 'all', + per_page: 100 + }); + + let shouldDelete = false; + if (!prs.data || prs.data.length === 0) { + core.info(`No PRs found for ${branchName} โ€” marking for deletion.`); + shouldDelete = true; + } else { + // If none of the PRs are open, safe to delete + const hasOpen = prs.data.some(p => p.state === 'open'); + if (!hasOpen) { + core.info(`All PRs for ${branchName} are closed โ€” marking for deletion.`); + shouldDelete = true; + } else { + core.info(`Open PR(s) exist for ${branchName} โ€” skipping deletion.`); + } + } + + if (shouldDelete) { + try { + await github.rest.git.deleteRef({ + owner, + repo, + ref: `heads/${branchName}` + }); + core.info(`Deleted branch: ${branchName}`); + } catch (delErr) { + core.warning(`Failed to delete ${branchName}: ${delErr.message}`); + } + } + } + + - name: Done + run: echo "Prune run completed." diff --git a/.github/workflows/repo-health.yml b/.github/workflows/repo-health.yml new file mode 100644 index 00000000..462d2021 --- /dev/null +++ b/.github/workflows/repo-health.yml @@ -0,0 +1,39 @@ +name: Repo Health Check + +on: + schedule: + - cron: '0 0 * * *' + pull_request: + types: [opened, synchronize, reopened] + workflow_dispatch: {} + +jobs: + repo_health: + name: Repo health + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + with: + fetch-depth: 0 + lfs: true + + - name: Set up Git + run: | + git --version + git lfs install --local || true + + - name: Run repo health check + env: + MAX_MB: 100 + LFS_ALLOW_MB: 50 + run: | + bash scripts/repo_health_check.sh + + - name: Upload health output + if: always() + uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5 + with: + name: repo-health-output + path: | + /tmp/repo_big_files.txt diff --git a/.github/workflows/waf-integration.yml b/.github/workflows/waf-integration.yml new file mode 100644 index 00000000..ac325622 --- /dev/null +++ b/.github/workflows/waf-integration.yml @@ -0,0 +1,103 @@ +name: WAF Integration Tests + +on: + push: + branches: [ main, development, 'feature/**' ] + paths: + - 'backend/internal/caddy/**' + - 'backend/internal/models/security*.go' + - 'scripts/coraza_integration.sh' + - 'Dockerfile' + - '.github/workflows/waf-integration.yml' + pull_request: + branches: [ main, development ] + paths: + - 'backend/internal/caddy/**' + - 'backend/internal/models/security*.go' + - 'scripts/coraza_integration.sh' + - 'Dockerfile' + - '.github/workflows/waf-integration.yml' + # Allow manual trigger + workflow_dispatch: + +jobs: + waf-integration: + name: Coraza WAF Integration + runs-on: ubuntu-latest + timeout-minutes: 15 + + steps: + - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 + + - name: Build Docker image + run: | + docker build \ + --build-arg VCS_REF=${{ github.sha }} \ + -t charon:local . + + - name: Run WAF integration tests + id: waf-test + run: | + chmod +x scripts/coraza_integration.sh + scripts/coraza_integration.sh 2>&1 | tee waf-test-output.txt + exit ${PIPESTATUS[0]} + + - name: Dump Debug Info on Failure + if: failure() + run: | + echo "## ๐Ÿ” Debug Information" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + + echo "### Container Status" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + docker ps -a --filter "name=charon" --filter "name=coraza" >> $GITHUB_STEP_SUMMARY 2>&1 || true + echo '```' >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + + echo "### Caddy Admin Config" >> $GITHUB_STEP_SUMMARY + echo '```json' >> $GITHUB_STEP_SUMMARY + curl -s http://localhost:2019/config 2>/dev/null | head -200 >> $GITHUB_STEP_SUMMARY || echo "Could not retrieve Caddy config" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + + echo "### Charon Container Logs (last 100 lines)" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + docker logs charon-debug 2>&1 | tail -100 >> $GITHUB_STEP_SUMMARY || echo "No container logs available" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + + echo "### WAF Ruleset Files" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + docker exec charon-debug sh -c 'ls -la /app/data/caddy/coraza/rulesets/ 2>/dev/null && echo "---" && cat /app/data/caddy/coraza/rulesets/*.conf 2>/dev/null' >> $GITHUB_STEP_SUMMARY || echo "No ruleset files found" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + + - name: WAF Integration Summary + if: always() + run: | + echo "## ๐Ÿ›ก๏ธ WAF Integration Test Results" >> $GITHUB_STEP_SUMMARY + if [ "${{ steps.waf-test.outcome }}" == "success" ]; then + echo "โœ… **All WAF tests passed**" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Test Results:" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + grep -E "^โœ“|^===|^Coraza" waf-test-output.txt || echo "See logs for details" + grep -E "^โœ“|^===|^Coraza" waf-test-output.txt >> $GITHUB_STEP_SUMMARY || echo "See logs for details" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + else + echo "โŒ **WAF tests failed**" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "### Failure Details:" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + grep -E "^โœ—|Unexpected|Error|failed" waf-test-output.txt | head -20 >> $GITHUB_STEP_SUMMARY || echo "See logs for details" >> $GITHUB_STEP_SUMMARY + echo '```' >> $GITHUB_STEP_SUMMARY + fi + + - name: Cleanup + if: always() + run: | + docker rm -f charon-debug || true + docker rm -f coraza-backend || true + docker network rm containers_default || true diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..96ec7d48 --- /dev/null +++ b/.gitignore @@ -0,0 +1,178 @@ +# ============================================================================= +# .gitignore - Files to exclude from version control +# ============================================================================= + +# ----------------------------------------------------------------------------- +# Python (pre-commit, tooling) +# ----------------------------------------------------------------------------- +__pycache__/ +*.py[cod] +*$py.class +*.so +.Python +.venv/ +venv/ +env/ +ENV/ +.pytest_cache/ +.coverage +*.cover +.hypothesis/ +htmlcov/ + +# ----------------------------------------------------------------------------- +# Node/Frontend +# ----------------------------------------------------------------------------- +node_modules/ +frontend/node_modules/ +backend/node_modules/ +frontend/dist/ +frontend/coverage/ +frontend/test-results/ +frontend/.vite/ +frontend/*.tsbuildinfo +/frontend/frontend/ + +# ----------------------------------------------------------------------------- +# Go/Backend - Build artifacts & coverage +# ----------------------------------------------------------------------------- +backend/api +backend/bin/ +backend/*.out +backend/*.cover +backend/*.html +backend/coverage/ +backend/coverage*.out +backend/coverage*.txt +backend/*.coverage.out +backend/handler_coverage.txt +backend/handlers.out +backend/services.test +backend/test-output.txt +backend/tr_no_cover.txt +backend/nohup.out +backend/charon +backend/codeql-db/ +backend/.venv/ + +# ----------------------------------------------------------------------------- +# Databases +# ----------------------------------------------------------------------------- +*.db +*.sqlite +*.sqlite3 +backend/data/ +backend/data/*.db +backend/data/**/*.db +backend/cmd/api/data/*.db +cpm.db +charon.db + +# ----------------------------------------------------------------------------- +# IDE & Editor +# ----------------------------------------------------------------------------- +.idea/ +*.swp +*.swo +*~ +.DS_Store +*.xcf +.vscode/ +.vscode/launch.json +.vscode.backup*/ + +# ----------------------------------------------------------------------------- +# Logs & Temp Files +# ----------------------------------------------------------------------------- +.trivy_logs/ +*.log +logs/ +npm-debug.log* +yarn-debug.log* +yarn-error.log* +nohup.out +hub_index.json +temp_index.json +backend/temp_index.json + +# ----------------------------------------------------------------------------- +# Environment Files +# ----------------------------------------------------------------------------- +.env +.env.* +!.env.example + +# ----------------------------------------------------------------------------- +# OS Files +# ----------------------------------------------------------------------------- +Thumbs.db + +# ----------------------------------------------------------------------------- +# Caddy Runtime Data +# ----------------------------------------------------------------------------- +backend/data/caddy/ +/data/ +/data/backups/ + +# ----------------------------------------------------------------------------- +# Docker Overrides +# ----------------------------------------------------------------------------- +docker-compose.override.yml + +# ----------------------------------------------------------------------------- +# GoReleaser +# ----------------------------------------------------------------------------- +dist/ + +# ----------------------------------------------------------------------------- +# Testing & Coverage +# ----------------------------------------------------------------------------- +coverage/ +coverage.out +*.xml +*.crdownload + +# ----------------------------------------------------------------------------- +# CodeQL & Security Scanning +# ----------------------------------------------------------------------------- +codeql-db/ +codeql-db-*/ +codeql-agent-results/ +codeql-custom-queries-*/ +codeql-results*.sarif +codeql-*.sarif +*.sarif +.codeql/ +.codeql/** + +# ----------------------------------------------------------------------------- +# Scripts & Temp Files (project-specific) +# ----------------------------------------------------------------------------- +create_issues.sh +cookies.txt +cookies.txt.bak +test.caddyfile + +# ----------------------------------------------------------------------------- +# Project Documentation (implementation notes - not needed in repo) +# ----------------------------------------------------------------------------- +*.md.bak +ACME_STAGING_IMPLEMENTATION.md* +ARCHITECTURE_PLAN.md +DOCKER_TASKS.md* +DOCUMENTATION_POLISH_SUMMARY.md +GHCR_MIGRATION_SUMMARY.md +ISSUE_*_IMPLEMENTATION.md* +PHASE_*_SUMMARY.md +PROJECT_BOARD_SETUP.md +PROJECT_PLANNING.md +VERSIONING_IMPLEMENTATION.md +backend/internal/api/handlers/import_handler.go.bak + +# ----------------------------------------------------------------------------- +# Import Directory (user uploads) +# ----------------------------------------------------------------------------- +import/ +test-results/charon.hatfieldhosted.com.har +test-results/local.har +.cache diff --git a/.goreleaser.yaml b/.goreleaser.yaml new file mode 100644 index 00000000..85171bf1 --- /dev/null +++ b/.goreleaser.yaml @@ -0,0 +1,125 @@ +version: 1 + +project_name: charon + +builds: + - id: linux + dir: backend + main: ./cmd/api + binary: charon + env: + - CGO_ENABLED=1 + - CC=zig cc -target {{ if eq .Arch "amd64" }}x86_64{{ else }}aarch64{{ end }}-linux-gnu + - CXX=zig c++ -target {{ if eq .Arch "amd64" }}x86_64{{ else }}aarch64{{ end }}-linux-gnu + goos: + - linux + goarch: + - amd64 + - arm64 + ldflags: + - -s -w + - -X github.com/Wikid82/charon/backend/internal/version.Version={{.Version}} + - -X github.com/Wikid82/charon/backend/internal/version.GitCommit={{.Commit}} + - -X github.com/Wikid82/charon/backend/internal/version.BuildTime={{.Date}} + + - id: windows + dir: backend + main: ./cmd/api + binary: charon + env: + - CGO_ENABLED=1 + - CC=zig cc -target x86_64-windows-gnu + - CXX=zig c++ -target x86_64-windows-gnu + goos: + - windows + goarch: + - amd64 + ldflags: + - -s -w + - -X github.com/Wikid82/charon/backend/internal/version.Version={{.Version}} + - -X github.com/Wikid82/charon/backend/internal/version.GitCommit={{.Commit}} + - -X github.com/Wikid82/charon/backend/internal/version.BuildTime={{.Date}} + + - id: darwin + dir: backend + main: ./cmd/api + binary: charon + env: + - CGO_ENABLED=1 + - CC=zig cc -target {{ if eq .Arch "amd64" }}x86_64{{ else }}aarch64{{ end }}-macos-gnu + - CXX=zig c++ -target {{ if eq .Arch "amd64" }}x86_64{{ else }}aarch64{{ end }}-macos-gnu + goos: + - darwin + goarch: + - amd64 + - arm64 + ldflags: + - -s -w + - -X github.com/Wikid82/charon/backend/internal/version.Version={{.Version}} + - -X github.com/Wikid82/charon/backend/internal/version.GitCommit={{.Commit}} + - -X github.com/Wikid82/charon/backend/internal/version.BuildTime={{.Date}} + +archives: + - format: tar.gz + id: nix + builds: + - linux + - darwin + name_template: >- + {{ .ProjectName }}_ + {{- .Version }}_ + {{- .Os }}_ + {{- .Arch }} + files: + - LICENSE + - README.md + + - format: zip + id: windows + builds: + - windows + name_template: >- + {{ .ProjectName }}_ + {{- .Version }}_ + {{- .Os }}_ + {{- .Arch }} + files: + - LICENSE + - README.md + +nfpms: + - id: packages + builds: + - linux + package_name: charon + vendor: Charon + homepage: https://github.com/Wikid82/charon + maintainer: Wikid82 + description: "Charon - A powerful reverse proxy manager" + license: MIT + formats: + - deb + - rpm + contents: + - src: ./backend/data/ + dst: /var/lib/charon/data/ + type: dir + - src: ./frontend/dist/ + dst: /usr/share/charon/frontend/ + type: dir + dependencies: + - libc6 + - ca-certificates + +checksum: + name_template: 'checksums.txt' + +snapshot: + name_template: "{{ .Tag }}-next" + +changelog: + sort: asc + filters: + exclude: + - '^docs:' + - '^test:' diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml new file mode 100644 index 00000000..92087696 --- /dev/null +++ b/.pre-commit-config.yaml @@ -0,0 +1,116 @@ +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v4.6.0 + hooks: + - id: end-of-file-fixer + exclude: '^(frontend/(coverage|dist|node_modules|\.vite)/|.*\.tsbuildinfo$)' + - id: trailing-whitespace + exclude: '^(frontend/(coverage|dist|node_modules|\.vite)/|.*\.tsbuildinfo$)' + - id: check-yaml + - id: check-added-large-files + args: ['--maxkb=2500'] + - repo: local + hooks: + - id: dockerfile-check + name: dockerfile validation + entry: tools/dockerfile_check.sh + language: script + files: "Dockerfile.*" + pass_filenames: true + - id: go-test-coverage + name: Go Test Coverage + entry: scripts/go-test-coverage.sh + language: script + pass_filenames: false + verbose: true + always_run: true + - id: go-vet + name: Go Vet + entry: bash -c 'cd backend && go vet ./...' + language: system + files: '\.go$' + pass_filenames: false + - id: check-version-match + name: Check .version matches latest Git tag + entry: bash -c 'scripts/check-version-match-tag.sh' + language: system + files: '\.version$' + pass_filenames: false + - id: check-lfs-large-files + name: Prevent large files that are not tracked by LFS + entry: bash scripts/pre-commit-hooks/check-lfs-for-large-files.sh + language: system + pass_filenames: false + verbose: true + always_run: true + - id: block-codeql-db-commits + name: Prevent committing CodeQL DB artifacts + entry: bash scripts/pre-commit-hooks/block-codeql-db-commits.sh + language: system + pass_filenames: false + verbose: true + always_run: true + - id: block-data-backups-commit + name: Prevent committing data/backups files + entry: bash scripts/pre-commit-hooks/block-data-backups-commit.sh + language: system + pass_filenames: false + verbose: true + always_run: true + + # === MANUAL/CI-ONLY HOOKS === + # These are slow and should only run on-demand or in CI + # Run manually with: pre-commit run golangci-lint --all-files + - id: go-test-race + name: Go Test Race (Manual) + entry: bash -c 'cd backend && go test -race ./...' + language: system + files: '\.go$' + pass_filenames: false + stages: [manual] # Only runs when explicitly called + + - id: golangci-lint + name: GolangCI-Lint (Manual) + entry: bash -c 'cd backend && docker run --rm -v $(pwd):/app:ro -w /app golangci/golangci-lint:latest golangci-lint run -v' + language: system + files: '\.go$' + pass_filenames: false + stages: [manual] # Only runs when explicitly called + + - id: hadolint + name: Hadolint Dockerfile Check (Manual) + entry: bash -c 'docker run --rm -i hadolint/hadolint < Dockerfile' + language: system + files: 'Dockerfile' + pass_filenames: false + stages: [manual] # Only runs when explicitly called + - id: frontend-type-check + name: Frontend TypeScript Check + entry: bash -c 'cd frontend && npm run type-check' + language: system + files: '^frontend/.*\.(ts|tsx)$' + pass_filenames: false + - id: frontend-lint + name: Frontend Lint (Fix) + entry: bash -c 'cd frontend && npm run lint -- --fix' + language: system + files: '^frontend/.*\.(ts|tsx|js|jsx)$' + pass_filenames: false + + - id: frontend-test-coverage + name: Frontend Test Coverage (Manual) + entry: scripts/frontend-test-coverage.sh + language: script + files: '^frontend/.*\\.(ts|tsx|js|jsx)$' + pass_filenames: false + verbose: true + stages: [manual] + + - id: security-scan + name: Security Vulnerability Scan (Manual) + entry: scripts/security-scan.sh + language: script + files: '(\.go$|go\.mod$|go\.sum$)' + pass_filenames: false + verbose: true + stages: [manual] # Only runs when explicitly called diff --git a/.sourcery.yml b/.sourcery.yml new file mode 100644 index 00000000..628ec063 --- /dev/null +++ b/.sourcery.yml @@ -0,0 +1,4 @@ +version: 1 +exclude: + - frontend/dist/** + - frontend/node_modules/** diff --git a/.version b/.version new file mode 100644 index 00000000..0d91a54c --- /dev/null +++ b/.version @@ -0,0 +1 @@ +0.3.0 diff --git a/BULK_ACL_FEATURE.md b/BULK_ACL_FEATURE.md new file mode 100644 index 00000000..0eebe8fb --- /dev/null +++ b/BULK_ACL_FEATURE.md @@ -0,0 +1,177 @@ +# Bulk ACL Application Feature + +## Overview +Implemented a bulk ACL (Access Control List) application feature that allows users to quickly apply or remove access lists from multiple proxy hosts at once, eliminating the need to edit each host individually. + +## User Workflow Improvements + +### Previous Workflow (Manual) +1. Create proxy hosts +2. Create access list +3. **Edit each host individually** to apply the ACL (tedious for many hosts) + +### New Workflow (Bulk) +1. Create proxy hosts +2. Create access list +3. **Select multiple hosts** โ†’ Bulk Actions โ†’ Apply/Remove ACL (one operation) + +## Implementation Details + +### Backend (`backend/internal/api/handlers/proxy_host_handler.go`) + +**New Endpoint**: `PUT /api/v1/proxy-hosts/bulk-update-acl` + +**Request Body**: +```json +{ + "host_uuids": ["uuid-1", "uuid-2", "uuid-3"], + "access_list_id": 42 // or null to remove ACL +} +``` + +**Response**: +```json +{ + "updated": 2, + "errors": [ + {"uuid": "uuid-3", "error": "proxy host not found"} + ] +} +``` + +**Features**: +- Updates multiple hosts in a single database transaction +- Applies Caddy config once for all updates (efficient) +- Partial failure handling (returns both successes and errors) +- Validates host existence before applying ACL +- Supports both applying and removing ACLs (null = remove) + +### Frontend + +#### API Client (`frontend/src/api/proxyHosts.ts`) +```typescript +export const bulkUpdateACL = async ( + hostUUIDs: string[], + accessListID: number | null +): Promise +``` + +#### React Query Hook (`frontend/src/hooks/useProxyHosts.ts`) +```typescript +const { bulkUpdateACL, isBulkUpdating } = useProxyHosts() + +// Usage +await bulkUpdateACL(['uuid-1', 'uuid-2'], 42) // Apply ACL 42 +await bulkUpdateACL(['uuid-1', 'uuid-2'], null) // Remove ACL +``` + +#### UI Components (`frontend/src/pages/ProxyHosts.tsx`) + +**Multi-Select Checkboxes**: +- Checkbox column added to proxy hosts table +- "Select All" checkbox in table header +- Individual checkboxes per row + +**Bulk Actions UI**: +- "Bulk Actions" button appears when hosts are selected +- Shows count of selected hosts +- Opens modal with ACL selection dropdown + +**Modal Features**: +- Lists all enabled access lists +- "Remove Access List" option (sets null) +- Real-time feedback on success/failure +- Toast notifications for user feedback + +## Testing + +### Backend Tests (`proxy_host_handler_test.go`) +- โœ… `TestProxyHostHandler_BulkUpdateACL_Success` - Apply ACL to multiple hosts +- โœ… `TestProxyHostHandler_BulkUpdateACL_RemoveACL` - Remove ACL (null value) +- โœ… `TestProxyHostHandler_BulkUpdateACL_PartialFailure` - Mixed success/failure +- โœ… `TestProxyHostHandler_BulkUpdateACL_EmptyUUIDs` - Validation error +- โœ… `TestProxyHostHandler_BulkUpdateACL_InvalidJSON` - Malformed request + +### Frontend Tests +**API Tests** (`proxyHosts-bulk.test.ts`): +- โœ… Apply ACL to multiple hosts +- โœ… Remove ACL with null value +- โœ… Handle partial failures +- โœ… Handle empty host list +- โœ… Propagate API errors + +**Hook Tests** (`useProxyHosts-bulk.test.tsx`): +- โœ… Apply ACL via mutation +- โœ… Remove ACL via mutation +- โœ… Query invalidation after success +- โœ… Error handling +- โœ… Loading state tracking + +**Test Results**: +- Backend: All tests passing (106+ tests) +- Frontend: All tests passing (132 tests) + +## Usage Examples + +### Example 1: Apply ACL to Multiple Hosts +```typescript +// Select hosts in UI +setSelectedHosts(new Set(['host-1-uuid', 'host-2-uuid', 'host-3-uuid'])) + +// User clicks "Bulk Actions" โ†’ Selects ACL from dropdown +await bulkUpdateACL(['host-1-uuid', 'host-2-uuid', 'host-3-uuid'], 5) + +// Result: "Access list applied to 3 host(s)" +``` + +### Example 2: Remove ACL from Hosts +```typescript +// User selects "Remove Access List" from dropdown +await bulkUpdateACL(['host-1-uuid', 'host-2-uuid'], null) + +// Result: "Access list removed from 2 host(s)" +``` + +### Example 3: Partial Failure Handling +```typescript +const result = await bulkUpdateACL(['valid-uuid', 'invalid-uuid'], 10) + +// result = { +// updated: 1, +// errors: [{ uuid: 'invalid-uuid', error: 'proxy host not found' }] +// } + +// Toast: "Updated 1 host(s), 1 failed" +``` + +## Benefits + +1. **Time Savings**: Apply ACLs to dozens of hosts in one click vs. editing each individually +2. **User-Friendly**: Clear visual feedback with checkboxes and selection count +3. **Error Resilient**: Partial failures don't block the entire operation +4. **Efficient**: Single Caddy config reload for all updates +5. **Flexible**: Supports both applying and removing ACLs +6. **Well-Tested**: Comprehensive test coverage for all scenarios + +## Future Enhancements (Optional) + +- Add bulk ACL application from Access Lists page (when creating/editing ACL) +- Bulk enable/disable hosts +- Bulk delete hosts +- Bulk certificate assignment +- Filter hosts before selection (e.g., "Select all hosts without ACL") + +## Related Files Modified + +### Backend +- `backend/internal/api/handlers/proxy_host_handler.go` (+73 lines) +- `backend/internal/api/handlers/proxy_host_handler_test.go` (+140 lines) + +### Frontend +- `frontend/src/api/proxyHosts.ts` (+19 lines) +- `frontend/src/hooks/useProxyHosts.ts` (+11 lines) +- `frontend/src/pages/ProxyHosts.tsx` (+95 lines) +- `frontend/src/api/__tests__/proxyHosts-bulk.test.ts` (+93 lines, new file) +- `frontend/src/hooks/__tests__/useProxyHosts-bulk.test.tsx` (+149 lines, new file) + +**Total**: ~580 lines added (including tests) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..cd5ad4b8 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,387 @@ +# Contributing to Charon + +Thank you for your interest in contributing to CaddyProxyManager+! This document provides guidelines and instructions for contributing to the project. + +## Table of Contents + +- [Code of Conduct](#code-of-conduct) +- [Getting Started](#getting-started) +- [Development Workflow](#development-workflow) +- [Coding Standards](#coding-standards) +- [Testing Guidelines](#testing-guidelines) +- [Pull Request Process](#pull-request-process) +- [Issue Guidelines](#issue-guidelines) +- [Documentation](#documentation) + +## Code of Conduct + +This project follows a Code of Conduct that all contributors are expected to adhere to: + +- Be respectful and inclusive +- Welcome newcomers and help them get started +- Focus on what's best for the community +- Show empathy towards other community members + +## Getting Started + +-### Prerequisites + +- **Go 1.24+** for backend development +- **Node.js 20+** and npm for frontend development +- Git for version control +- A GitHub account + +### Fork and Clone + +1. Fork the repository on GitHub +2. Clone your fork locally: +```bash +git clone https://github.com/YOUR_USERNAME/charon.git +cd charon +``` + +3. Add the upstream remote: +```bash +git remote add upstream https://github.com/Wikid82/charon.git +``` + +### Set Up Development Environment + +**Backend:** +```bash +cd backend +go mod download +go run ./cmd/seed/main.go # Seed test data +go run ./cmd/api/main.go # Start backend +``` + +**Frontend:** +```bash +cd frontend +npm install +npm run dev # Start frontend dev server +``` + +## Development Workflow + +### Branching Strategy + +- **main** - Production-ready code +- **development** - Main development branch (default) +- **feature/** - Feature branches (e.g., `feature/add-ssl-support`) +- **bugfix/** - Bug fix branches (e.g., `bugfix/fix-import-crash`) +- **hotfix/** - Urgent production fixes + +### Creating a Feature Branch + +Always branch from `development`: + +```bash +git checkout development +git pull upstream development +git checkout -b feature/your-feature-name +``` + +### Commit Message Guidelines + +Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification: + +``` +(): + + + +