Files
Charon/docs/plans/structure.md
GitHub Actions 3169b05156 fix: skip incomplete system log viewer tests
- Marked 12 tests as skip pending feature implementation
- Features tracked in GitHub issue #686 (system log viewer feature completion)
- Tests cover sorting by timestamp/level/method/URI/status, pagination controls, filtering by text/level, download functionality
- Unblocks Phase 2 at 91.7% pass rate to proceed to Phase 3 security enforcement validation
- TODO comments in code reference GitHub #686 for feature completion tracking
- Tests skipped: Pagination (3), Search/Filter (2), Download (2), Sorting (1), Log Display (4)
2026-02-09 21:55:55 +00:00

919 lines
30 KiB
Markdown

# Repository Structure Reorganization Plan
**Date**: December 21, 2025 (Revised)
**Status**: Proposed
**Risk Level**: Medium (requires CI/CD updates, Docker path changes)
---
## Executive Summary
The repository root level currently contains **81 items**, making it difficult to navigate and maintain. This plan proposes moving files into logical directories to achieve a cleaner, more organized structure with only **~15 essential items** at the root level.
**Key Benefits**:
- Easier navigation for contributors
- Clearer separation of concerns
- Reduced cognitive load when browsing repository
- Better .gitignore and .dockerignore maintenance
- Improved CI/CD workflow clarity
---
## Current Root-Level Analysis
### Category Breakdown
| Category | Count | Examples | Status |
|----------|-------|----------|--------|
| **Docker Compose Files** | 5 | `docker-compose.yml`, `docker-compose.dev.yml`, etc. | 🔴 Scattered |
| **CodeQL SARIF Files** | 6 | `codeql-go.sarif`, `codeql-results-*.sarif` | 🔴 Build artifacts at root |
| **Implementation Docs** | 16 | `BULK_ACL_FEATURE.md`, `IMPLEMENTATION_SUMMARY.md`, etc. | 🔴 Should be in docs/ |
| **Config Files** | 8 | `eslint.config.js`, `.pre-commit-config.yaml`, `Makefile`, etc. | 🟡 Mixed - some stay, some move |
| **Docker Files** | 3 | `Dockerfile`, `docker-entrypoint.sh`, `DOCKER.md` | 🟡 Could group |
| **Core Docs** | 4 | `README.md`, `CONTRIBUTING.md`, `LICENSE`, `VERSION.md` | 🟢 Stay at root |
| **Hidden Config** | 15+ | `.github/`, `.vscode/`, `.gitignore`, `.dockerignore`, etc. | 🟢 Stay at root |
| **Source Directories** | 7 | `backend/`, `frontend/`, `docs/`, `scripts/`, etc. | 🟢 Stay at root |
| **Workspace File** | 1 | `Chiron.code-workspace` | 🟢 Stay at root |
| **Build Artifacts** | 3 | `codeql-db/`, `codeql-agent-results/`, `.trivy_logs/` | 🔴 Gitignored but present |
**Total Root Items**: ~60 items (files + directories)
### Problem Areas
1. **Docker Compose Sprawl**: 5 files at root when they should be grouped
2. **SARIF Pollution**: 6 CodeQL SARIF files are build artifacts (should be .gitignored)
3. **Documentation Chaos**: 16 implementation/feature docs scattered at root instead of `docs/`
4. **Mixed Purposes**: Docker files, configs, docs, code all at same level
---
## Proposed New Structure
### Root Level (Clean)
```
/projects/Charon/
├── .github/ # GitHub workflows, templates, agents
├── .vscode/ # VS Code workspace settings
├── backend/ # Go backend source
├── configs/ # Runtime configs (CrowdSec, etc.)
├── data/ # Runtime data (gitignored)
├── docs/ # Documentation (enhanced)
├── frontend/ # React frontend source
├── logs/ # Runtime logs (gitignored)
├── scripts/ # Build/test/integration scripts
├── test-results/ # Test outputs (gitignored)
├── tools/ # Development tools
├── .codecov.yml # Codecov configuration
├── .dockerignore # Docker build exclusions
├── .gitattributes # Git attributes
├── .gitignore # Git exclusions
├── .goreleaser.yaml # GoReleaser config
├── .markdownlint.json # Markdown lint config
├── .markdownlintrc # Markdown lint config
├── .pre-commit-config.yaml # Pre-commit hooks
├── CHANGELOG.md # Project changelog
├── Chiron.code-workspace # VS Code workspace
├── CONTRIBUTING.md # Contribution guidelines
├── CONTRIBUTING_TRANSLATIONS.md # Translation guidelines
├── LICENSE # License file
├── Makefile # Build automation
├── README.md # Project readme
├── VERSION.md # Version documentation
├── eslint.config.js # ESLint config
├── go.work # Go workspace
├── go.work.sum # Go workspace checksums
└── package.json # Root package.json (pre-commit, etc.)
```
### New Directory: `.docker/`
**Purpose**: Consolidate all Docker-related files except the primary Dockerfile
```
.docker/
├── compose/
│ ├── docker-compose.yml # Main compose (moved from root)
│ ├── docker-compose.dev.yml # Dev override (moved from root)
│ ├── docker-compose.local.yml # Local override (moved from root)
│ ├── docker-compose.remote.yml # Remote override (moved from root)
│ ├── docker-compose.override.yml # Remote override (moved from root)
│ └── README.md # Compose file documentation
├── docker-entrypoint.sh # Entrypoint script (moved from root)
└── README.md # Docker documentation (DOCKER.md renamed)
```
**Why `.docker/` with a dot?**
- Keeps it close to root-level Dockerfile (co-location)
- Hidden by default in file browsers (reduces clutter)
- Common pattern in monorepos (`.github/`, `.vscode/`)
**Alternative**: Could use `docker/` without dot, but `.docker/` is preferred for consistency
### Enhanced: `docs/`
**New subdirectory**: `docs/implementation/`
**Purpose**: Archive completed implementation documents that shouldn't be at root
```
docs/
├── implementation/ # NEW: Implementation documents
│ ├── BULK_ACL_FEATURE.md # Moved from root
│ ├── IMPLEMENTATION_SUMMARY.md # Moved from root
│ ├── ISSUE_16_ACL_IMPLEMENTATION.md # Moved from root
│ ├── QA_AUDIT_REPORT_LOADING_OVERLAYS.md # Moved from root
│ ├── QA_MIGRATION_COMPLETE.md # Moved from root
│ ├── SECURITY_CONFIG_PRIORITY.md # Moved from root
│ ├── SECURITY_IMPLEMENTATION_PLAN.md # Moved from root
│ ├── WEBSOCKET_FIX_SUMMARY.md # Moved from root
│ └── README.md # Index of implementation docs
├── issues/ # Existing: Issue templates
├── plans/ # Existing: Planning documents
│ ├── structure.md # THIS FILE
│ └── ...
├── reports/ # Existing: Reports
├── troubleshooting/ # Existing: Troubleshooting guides
├── acme-staging.md
├── api.md
├── ...
└── index.md
```
### Enhanced: `.gitignore`
**New entries** to prevent SARIF files at root:
> **Note**: The `*.sarif` pattern may already exist in `.gitignore`. Verify before adding to avoid duplication. The explicit patterns below ensure comprehensive coverage.
```gitignore
# Add to "CodeQL & Security Scanning" section:
# -----------------------------------------------------------------------------
# CodeQL & Security Scanning
# -----------------------------------------------------------------------------
# ... existing entries ...
# Prevent SARIF files at root level
/*.sarif
/codeql-*.sarif
# Explicit gitignore for scattered SARIF files
/codeql-go.sarif
/codeql-js.sarif
/codeql-results-go.sarif
/codeql-results-go-backend.sarif
/codeql-results-go-new.sarif
/codeql-results-js.sarif
# Test artifacts at root
/block*.txt
/final_block_test.txt
# Debug/temp config files at root
/caddy_*.json
!package*.json
# Trivy scan outputs at root
/trivy-*.txt
```
#### Local Override Migration
**Important**: With the move to `.docker/compose/`, the standard `docker-compose.override.yml` behavior changes:
- **Previous behavior**: `docker-compose.override.yml` at repository root was auto-applied by Docker Compose
- **New behavior**: Override files at `.docker/compose/docker-compose.override.yml` must be explicitly referenced with `-f` flag
**Updated .gitignore entry**:
```gitignore
# Local docker-compose override (new location)
.docker/compose/docker-compose.override.yml
```
**Usage with new location**:
```bash
# Development with override
docker compose -f .docker/compose/docker-compose.yml -f .docker/compose/docker-compose.override.yml up -d
```
**Note**: Users with existing `docker-compose.override.yml` at root should move it to `.docker/compose/` and update their workflow scripts accordingly.
---
## File Migration Table
### Docker Compose Files → `.docker/compose/`
| Current Path | New Path | Type |
|-------------|----------|------|
| `/docker-compose.yml` | `/.docker/compose/docker-compose.yml` | Move |
| `/docker-compose.dev.yml` | `/.docker/compose/docker-compose.dev.yml` | Move |
| `/docker-compose.local.yml` | `/.docker/compose/docker-compose.local.yml` | Move |
| `/docker-compose.remote.yml` | `/.docker/compose/docker-compose.remote.yml` | Move |
| `/docker-compose.override.yml` | `/.docker/compose/docker-compose.override.yml` | Move (if exists) |
**Note**: `docker-compose.override.yml` is gitignored. Include in .gitignore update.
### Docker Support Files → `.docker/`
| Current Path | New Path | Type |
|-------------|----------|------|
| `/docker-entrypoint.sh` | `/.docker/docker-entrypoint.sh` | Move |
| `/DOCKER.md` | `/.docker/README.md` | Move + Rename |
### Implementation Docs → `docs/implementation/`
| Current Path | New Path | Type |
|-------------|----------|------|
| `/BULK_ACL_FEATURE.md` | `/docs/implementation/BULK_ACL_FEATURE.md` | Move |
| `/IMPLEMENTATION_SUMMARY.md` | `/docs/implementation/IMPLEMENTATION_SUMMARY.md` | Move |
| `/ISSUE_16_ACL_IMPLEMENTATION.md` | `/docs/implementation/ISSUE_16_ACL_IMPLEMENTATION.md` | Move |
| `/QA_AUDIT_REPORT_LOADING_OVERLAYS.md` | `/docs/implementation/QA_AUDIT_REPORT_LOADING_OVERLAYS.md` | Move |
| `/QA_MIGRATION_COMPLETE.md` | `/docs/implementation/QA_MIGRATION_COMPLETE.md` | Move |
| `/SECURITY_CONFIG_PRIORITY.md` | `/docs/implementation/SECURITY_CONFIG_PRIORITY.md` | Move |
| `/SECURITY_IMPLEMENTATION_PLAN.md` | `/docs/implementation/SECURITY_IMPLEMENTATION_PLAN.md` | Move |
| `/WEBSOCKET_FIX_SUMMARY.md` | `/docs/implementation/WEBSOCKET_FIX_SUMMARY.md` | Move |
| `/AGENT_SKILLS_MIGRATION_SUMMARY.md` | `/docs/implementation/AGENT_SKILLS_MIGRATION_SUMMARY.md` | Move |
| `/I18N_IMPLEMENTATION_SUMMARY.md` | `/docs/implementation/I18N_IMPLEMENTATION_SUMMARY.md` | Move |
| `/INVESTIGATION_SUMMARY.md` | `/docs/implementation/INVESTIGATION_SUMMARY.md` | Move |
| `/PHASE_0_COMPLETE.md` | `/docs/implementation/PHASE_0_COMPLETE.md` | Move |
| `/PHASE_3_COMPLETE.md` | `/docs/implementation/PHASE_3_COMPLETE.md` | Move |
| `/PHASE_4_COMPLETE.md` | `/docs/implementation/PHASE_4_COMPLETE.md` | Move |
| `/PHASE_5_COMPLETE.md` | `/docs/implementation/PHASE_5_COMPLETE.md` | Move |
| `/QA_PHASE5_VERIFICATION_REPORT.md` | `/docs/implementation/QA_PHASE5_VERIFICATION_REPORT.md` | Move |
| `/SECURITY_HEADERS_IMPLEMENTATION_SUMMARY.md` | `/docs/implementation/SECURITY_HEADERS_IMPLEMENTATION_SUMMARY.md` | Move |
### CodeQL SARIF Files → Delete (Add to .gitignore)
| Current Path | Action | Reason |
|-------------|--------|--------|
| `/codeql-go.sarif` | Delete + gitignore | Build artifact |
| `/codeql-js.sarif` | Delete + gitignore | Build artifact |
| `/codeql-results-go.sarif` | Delete + gitignore | Build artifact |
| `/codeql-results-go-backend.sarif` | Delete + gitignore | Build artifact |
| `/codeql-results-go-new.sarif` | Delete + gitignore | Build artifact |
| `/codeql-results-js.sarif` | Delete + gitignore | Build artifact |
**Note**: These are generated by CodeQL and should never be committed.
### Test/Debug Files → Delete + Gitignore
| Current Path | Action | Reason |
|-------------|--------|--------|
| `/block_test.txt` | Delete + gitignore | Test artifact |
| `/blocking_test.txt` | Delete + gitignore | Test artifact |
| `/final_block_test.txt` | Delete + gitignore | Test artifact |
| `/caddy_config_qa.json` | Delete + gitignore | Debug config |
| `/caddy_crowdsec_config.json` | Delete + gitignore | Debug config |
| `/trivy-image-scan.txt` | Delete + gitignore | Scan output |
| `/trivy-scan-output.txt` | Delete + gitignore | Scan output |
**Note**: These are test/debug artifacts that should never be committed.
### Files Staying at Root
| File | Reason |
|------|--------|
| `Dockerfile` | Primary Docker build file - standard location |
| `Makefile` | Build automation - standard location |
| `README.md` | Project entry point - standard location |
| `CONTRIBUTING.md` | Contributor guidelines - standard location |
| `CONTRIBUTING_TRANSLATIONS.md` | Translation contribution guidelines - standard location |
| `LICENSE` | License file - standard location |
| `VERSION.md` | Version documentation - standard location |
| `CHANGELOG.md` | Project changelog - standard location |
| `Chiron.code-workspace` | VS Code workspace - standard location |
| `go.work` | Go workspace - required at root |
| `go.work.sum` | Go workspace checksums - required at root |
| `package.json` | Root package (pre-commit, etc.) - required at root |
| `eslint.config.js` | ESLint config - required at root |
| `.codecov.yml` | Codecov config - required at root |
| `.goreleaser.yaml` | GoReleaser config - required at root |
| `.markdownlint.json` | Markdown lint config - required at root |
| `.pre-commit-config.yaml` | Pre-commit config - required at root |
| All `.git*` files | Git configuration - required at root |
| All hidden directories | Standard locations |
---
## Impact Analysis
### Files Requiring Updates
#### 1. GitHub Workflows (`.github/workflows/*.yml`)
**Files to Update**: 25+ workflow files
**Changes Needed**:
```yaml
# OLD (scattered references):
- 'Dockerfile'
- 'docker-compose.yml'
- 'docker-entrypoint.sh'
- 'DOCKER.md'
# NEW (centralized references):
- 'Dockerfile' # Stays at root
- '.docker/compose/docker-compose.yml'
- '.docker/compose/docker-compose.*.yml'
- '.docker/docker-entrypoint.sh'
- '.docker/README.md'
```
**Specific Files**:
- `.github/workflows/docker-lint.yml` - References Dockerfile (no change needed)
- `.github/workflows/docker-build.yml` - May reference docker-compose
- `.github/workflows/docker-publish.yml` - May reference docker-compose
- `.github/workflows/waf-integration.yml` - References Dockerfile (no change needed)
**Search Pattern**: `grep -r "docker-compose" .github/workflows/`
#### 2. Scripts (`scripts/*.sh`)
**Files to Update**: ~5 scripts
**Changes Needed**:
```bash
# OLD:
docker-compose -f docker-compose.local.yml up -d
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
# NEW:
docker-compose -f .docker/compose/docker-compose.local.yml up -d
docker compose -f .docker/compose/docker-compose.yml -f .docker/compose/docker-compose.dev.yml up
```
**Specific Files**:
- `scripts/coraza_integration.sh` - Uses docker-compose.local.yml
- `scripts/crowdsec_integration.sh` - Uses docker-compose files
- `scripts/crowdsec_startup_test.sh` - Uses docker-compose files
- `scripts/integration-test.sh` - Uses docker-compose files
**Search Pattern**: `grep -r "docker-compose" scripts/`
#### 3. VS Code Tasks (`.vscode/tasks.json`)
**Changes Needed**:
```json
// OLD:
"docker compose -f docker-compose.override.yml up -d"
"docker compose -f docker-compose.local.yml up -d"
// NEW:
"docker compose -f .docker/compose/docker-compose.override.yml up -d"
"docker compose -f .docker/compose/docker-compose.local.yml up -d"
```
**Affected Tasks**:
- "Build & Run: Local Docker Image"
- "Build & Run: Local Docker Image No-Cache"
- "Docker: Start Dev Environment"
- "Docker: Stop Dev Environment"
- "Docker: Start Local Environment"
- "Docker: Stop Local Environment"
#### 4. Makefile
**Changes Needed**:
```makefile
# OLD:
docker-compose build
docker-compose up -d
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up
docker-compose down
docker-compose logs -f
# NEW:
docker-compose -f .docker/compose/docker-compose.yml build
docker-compose -f .docker/compose/docker-compose.yml up -d
docker-compose -f .docker/compose/docker-compose.yml -f .docker/compose/docker-compose.dev.yml up
docker-compose -f .docker/compose/docker-compose.yml down
docker-compose -f .docker/compose/docker-compose.yml logs -f
```
#### 5. Dockerfile
**Changes Needed**:
```dockerfile
# OLD:
COPY docker-entrypoint.sh /usr/local/bin/
# NEW:
COPY .docker/docker-entrypoint.sh /usr/local/bin/
```
**Line**: Search for `docker-entrypoint.sh` in Dockerfile
#### 6. Documentation Files
**Files to Update**:
- `README.md` - May reference docker-compose files or DOCKER.md
- `CONTRIBUTING.md` - May reference docker-compose files
- `docs/getting-started.md` - Likely references docker-compose
- `docs/debugging-local-container.md` - Likely references docker-compose
- Any docs referencing implementation files moved to `docs/implementation/`
**Search Pattern**:
- `grep -r "docker-compose" docs/`
- `grep -r "DOCKER.md" docs/`
- `grep -r "BULK_ACL_FEATURE\|IMPLEMENTATION_SUMMARY" docs/`
#### 7. .dockerignore
**Changes Needed**:
```dockerignore
# Add to "Documentation" section:
docs/implementation/
# Update Docker Compose exclusion:
.docker/
```
#### 8. .gitignore
**Changes Needed**:
```gitignore
# Add explicit SARIF exclusions at root:
/*.sarif
/codeql-*.sarif
# Update docker-compose.override.yml path:
.docker/compose/docker-compose.override.yml
```
---
## Migration Steps
### Phase 1: Preparation (No Breaking Changes)
1. **Create new directories**:
```bash
mkdir -p .docker/compose
mkdir -p docs/implementation
```
2. **Create README files**:
- `.docker/README.md` (content from DOCKER.md + compose guide)
- `.docker/compose/README.md` (compose file documentation)
- `docs/implementation/README.md` (index of implementation docs)
3. **Update .gitignore** (add SARIF exclusions):
```bash
# Add to .gitignore:
/*.sarif
/codeql-*.sarif
.docker/compose/docker-compose.override.yml
```
4. **Commit preparation**:
```bash
git add .docker/ docs/implementation/ .gitignore
git commit -m "chore: prepare directory structure for reorganization"
```
### Phase 2: Move Files (Breaking Changes)
**⚠️ WARNING**: This phase will break existing workflows until all references are updated.
1. **Move Docker Compose files**:
```bash
git mv docker-compose.yml .docker/compose/
git mv docker-compose.dev.yml .docker/compose/
git mv docker-compose.local.yml .docker/compose/
git mv docker-compose.remote.yml .docker/compose/
# docker-compose.override.yml is gitignored, no need to move
```
2. **Move Docker support files**:
```bash
git mv docker-entrypoint.sh .docker/
git mv DOCKER.md .docker/README.md
```
3. **Move implementation docs**:
```bash
git mv BULK_ACL_FEATURE.md docs/implementation/
git mv IMPLEMENTATION_SUMMARY.md docs/implementation/
git mv ISSUE_16_ACL_IMPLEMENTATION.md docs/implementation/
git mv QA_AUDIT_REPORT_LOADING_OVERLAYS.md docs/implementation/
git mv QA_MIGRATION_COMPLETE.md docs/implementation/
git mv SECURITY_CONFIG_PRIORITY.md docs/implementation/
git mv SECURITY_IMPLEMENTATION_PLAN.md docs/implementation/
git mv WEBSOCKET_FIX_SUMMARY.md docs/implementation/
git mv AGENT_SKILLS_MIGRATION_SUMMARY.md docs/implementation/
git mv I18N_IMPLEMENTATION_SUMMARY.md docs/implementation/
git mv INVESTIGATION_SUMMARY.md docs/implementation/
git mv PHASE_0_COMPLETE.md docs/implementation/
git mv PHASE_3_COMPLETE.md docs/implementation/
git mv PHASE_4_COMPLETE.md docs/implementation/
git mv PHASE_5_COMPLETE.md docs/implementation/
git mv QA_PHASE5_VERIFICATION_REPORT.md docs/implementation/
git mv SECURITY_HEADERS_IMPLEMENTATION_SUMMARY.md docs/implementation/
```
4. **Delete test/debug files**:
```bash
git rm block_test.txt
git rm blocking_test.txt
git rm final_block_test.txt
git rm caddy_config_qa.json
git rm caddy_crowdsec_config.json
git rm trivy-image-scan.txt
git rm trivy-scan-output.txt
```
5. **Delete SARIF files**:
```bash
git rm codeql-go.sarif
git rm codeql-js.sarif
git rm codeql-results-go.sarif
git rm codeql-results-go-backend.sarif
git rm codeql-results-go-new.sarif
git rm codeql-results-js.sarif
```
6. **Commit file moves**:
```bash
git commit -m "chore: reorganize repository structure
- Move docker-compose files to .docker/compose/
- Move docker-entrypoint.sh to .docker/
- Move DOCKER.md to .docker/README.md
- Move implementation docs to docs/implementation/
- Delete committed SARIF files (should be gitignored)
- Delete test/debug artifacts (should be gitignored)
"
```
### Phase 3: Update References (Fix Breaking Changes)
**Order matters**: Update in this sequence to minimize build failures.
1. **Update Dockerfile**:
- Change `docker-entrypoint.sh` → `.docker/docker-entrypoint.sh`
- Test: `docker build -t charon:test .`
2. **Update Makefile**:
- Change all `docker-compose` commands to use `.docker/compose/docker-compose.yml`
- Test: `make docker-build`, `make docker-up`
3. **Update .vscode/tasks.json**:
- Change docker-compose paths in all tasks
- Test: Run "Docker: Start Local Environment" task
4. **Update scripts/**:
- Update `scripts/coraza_integration.sh`
- Update `scripts/crowdsec_integration.sh`
- Update `scripts/crowdsec_startup_test.sh`
- Update `scripts/integration-test.sh`
- Test: Run each script
5. **Update .github/workflows/**:
- Update all workflows referencing docker-compose files
- Test: Trigger workflows or dry-run locally
6. **Update .dockerignore**:
- Add `.docker/` exclusion
- Add `docs/implementation/` exclusion
7. **Update documentation**:
- Update `README.md`
- Update `CONTRIBUTING.md`
- Update `docs/getting-started.md`
- Update `docs/debugging-local-container.md`
- Update any docs referencing moved files
8. **Commit all reference updates**:
```bash
git add -A
git commit -m "chore: update all references to reorganized files
- Update Dockerfile to reference .docker/docker-entrypoint.sh
- Update Makefile docker-compose paths
- Update VS Code tasks with new compose paths
- Update scripts with new compose paths
- Update GitHub workflows with new paths
- Update documentation references
- Update .dockerignore and .gitignore
"
```
### Phase 4: Verification
1. **Local build test**:
```bash
docker build -t charon:test .
docker compose -f .docker/compose/docker-compose.yml build
```
2. **Local run test**:
```bash
docker compose -f .docker/compose/docker-compose.local.yml up -d
# Verify Charon starts correctly
docker compose -f .docker/compose/docker-compose.local.yml down
```
3. **Backend tests**:
```bash
cd backend && go test ./...
```
4. **Frontend tests**:
```bash
cd frontend && npm run test
```
5. **Integration tests**:
```bash
scripts/integration-test.sh
```
6. **Pre-commit checks**:
```bash
pre-commit run --all-files
```
7. **VS Code tasks**:
- Test "Build & Run: Local Docker Image"
- Test "Docker: Start Local Environment"
### Phase 5: CI/CD Monitoring
1. **Push to feature branch**:
```bash
git checkout -b chore/reorganize-structure
git push origin chore/reorganize-structure
```
2. **Create PR** with detailed description:
- Link to this plan
- List all changed files
- Note breaking changes
- Request review from maintainers
3. **Monitor CI/CD**:
- Watch all workflow runs
- Fix any failures immediately
- Update this plan if new issues discovered
4. **After merge**:
- Announce in project channels
- Update any external documentation
- Monitor for issues in next few days
### Phase 6: Documentation & Enforcement
1. **Create structure enforcement instructions**:
Create `.github/instructions/structure.instructions.md` to enforce the clean structure going forward:
```markdown
---
applyTo: '*'
description: 'Repository structure guidelines to maintain organized file placement'
---
# Repository Structure Guidelines
## Root Level Rules
The repository root should contain ONLY:
- Essential config files (`.gitignore`, `.pre-commit-config.yaml`, `Makefile`, etc.)
- Standard project files (`README.md`, `CONTRIBUTING.md`, `LICENSE`, `CHANGELOG.md`)
- Go workspace files (`go.work`, `go.work.sum`)
- VS Code workspace (`Chiron.code-workspace`)
- Primary `Dockerfile` (entrypoint and compose files live in `.docker/`)
## File Placement Rules
### Implementation/Feature Documentation
- **Location**: `docs/implementation/`
- **Pattern**: `*_SUMMARY.md`, `*_IMPLEMENTATION.md`, `*_COMPLETE.md`, `*_FEATURE.md`
- **Never** place implementation docs at root
### Docker Compose Files
- **Location**: `.docker/compose/`
- **Files**: `docker-compose.yml`, `docker-compose.*.yml`
- **Override**: Local overrides go in `.docker/compose/docker-compose.override.yml`
### Docker Support Files
- **Location**: `.docker/`
- **Files**: `docker-entrypoint.sh`, Docker documentation
### Test Artifacts
- **Never commit**: `*.sarif`, `*_test.txt`, `*.cover` files at root
- **Location**: Test outputs should go to `test-results/` or be gitignored
### Debug/Temp Config Files
- **Never commit**: Temporary JSON configs like `caddy_*.json` at root
- **Location**: Use `configs/` for persistent configs, gitignore temp files
## Before Creating New Files
Ask yourself:
1. Is this a standard project file? → Root is OK
2. Is this implementation documentation? → `docs/implementation/`
3. Is this Docker-related? → `.docker/` or `.docker/compose/`
4. Is this a test artifact? → `test-results/` or gitignore
5. Is this a script? → `scripts/`
6. Is this runtime config? → `configs/`
## Enforcement
Pre-commit hooks and CI will flag files placed incorrectly at root level.
```
2. **Update .pre-commit-config.yaml** (optional future enhancement):
Consider adding a hook to detect new files at root that don't match allowed patterns.
3. **Announce changes**:
- Update `CHANGELOG.md` with structure reorganization entry
- Notify contributors of new file placement guidelines
---
## Risk Assessment
### High Risk Changes
| Change | Risk | Mitigation |
|--------|------|------------|
| **Docker Compose Paths** | CI/CD workflows may break | Test all workflows locally before merge |
| **Dockerfile COPY** | Docker build may fail | Test build immediately after change |
| **VS Code Tasks** | Local development disrupted | Update tasks before file moves |
| **Script References** | Integration tests may fail | Test all scripts after updates |
### Medium Risk Changes
| Change | Risk | Mitigation |
|--------|------|------------|
| **Documentation References** | Broken links | Use find-and-replace, verify all links |
| **Makefile Commands** | Local builds may fail | Test all make targets |
| **.dockerignore** | Docker image size may change | Compare before/after image sizes |
### Low Risk Changes
| Change | Risk | Mitigation |
|--------|------|------------|
| **Implementation Docs Move** | Internal docs, low impact | Update any cross-references |
| **SARIF Deletion** | Already gitignored | None needed |
| **.gitignore Updates** | Prevents future pollution | None needed |
### Rollback Plan
If critical issues arise after merge:
1. **Immediate**: Revert the merge commit
2. **Analysis**: Identify what was missed in testing
3. **Fix**: Update this plan with new requirements
4. **Re-attempt**: Create new PR with fixes
---
## Success Criteria
✅ **Before Merge**:
- [ ] All file moves completed
- [ ] All references updated
- [ ] Local Docker build succeeds
- [ ] Local Docker run succeeds
- [ ] Backend tests pass
- [ ] Frontend tests pass
- [ ] Integration tests pass
- [ ] Pre-commit checks pass
- [ ] All VS Code tasks work
- [ ] Documentation updated
- [ ] Structure instructions file created at `.github/instructions/structure.instructions.md`
- [ ] Test/debug files cleaned up and gitignored
- [ ] PR reviewed by maintainers
✅ **After Merge**:
- [ ] All CI/CD workflows pass
- [ ] Docker images build successfully
- [ ] No broken links in documentation
- [ ] No regressions reported
- [ ] Root level has ~15 items (down from 81)
---
## Alternative Approaches Considered
### Alternative 1: Keep Docker Files at Root
**Pros**: No breaking changes, familiar location
**Cons**: Doesn't solve the clutter problem
**Decision**: Rejected - doesn't meet goal of cleaning up root
### Alternative 2: Use `docker/` Instead of `.docker/`
**Pros**: More visible, no hidden directory
**Cons**: Less consistent with `.github/`, `.vscode/` pattern
**Decision**: Rejected - prefer hidden directory for consistency
### Alternative 3: Keep Implementation Docs at Root
**Pros**: Easier to find for contributors
**Cons**: Continues root-level clutter
**Decision**: Rejected - docs belong in `docs/`, can add index
### Alternative 4: Move All Config Files to `.config/`
**Pros**: Maximum organization
**Cons**: Many tools expect configs at root (eslint, pre-commit, etc.)
**Decision**: Rejected - tool requirements win over organization
### Alternative 5: Delete Old Implementation Docs
**Pros**: Maximum cleanup
**Cons**: Loses historical context, implementation notes
**Decision**: Rejected - prefer archiving to deletion
---
## Future Enhancements
After this reorganization, consider:
1. **`.config/` Directory**: For configs that don't need to be at root
2. **`build/` Directory**: For build artifacts and temporary files
3. **`deployments/` Directory**: For deployment configurations (Kubernetes, etc.)
4. **Submodule for Configs**: If `configs/` grows too large
5. **Documentation Site**: Consider moving docs to dedicated site structure
---
## References
- [Twelve-Factor App](https://12factor.net/) - Config management
- [GitHub's .github Directory](https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/creating-a-default-community-health-file)
- [VS Code Workspace](https://code.visualstudio.com/docs/editor/workspaces)
- [Docker Best Practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/)
---
## Appendix: Search Commands
For agents implementing this plan, use these commands to find all references:
```bash
# Find docker-compose references:
grep -r "docker-compose\.yml" . --exclude-dir=node_modules --exclude-dir=.git
# Find docker-entrypoint.sh references:
grep -r "docker-entrypoint\.sh" . --exclude-dir=node_modules --exclude-dir=.git
# Find DOCKER.md references:
grep -r "DOCKER\.md" . --exclude-dir=node_modules --exclude-dir=.git
# Find implementation doc references:
grep -r "BULK_ACL_FEATURE\|IMPLEMENTATION_SUMMARY\|ISSUE_16_ACL" . --exclude-dir=node_modules --exclude-dir=.git
# Find SARIF references:
grep -r "\.sarif" . --exclude-dir=node_modules --exclude-dir=.git
```
---
**End of Plan**