231 lines
9.1 KiB
Markdown
231 lines
9.1 KiB
Markdown
# Issue #5, #43, and Caddyfile Import Implementation
|
|
|
|
## Summary
|
|
Implemented comprehensive data persistence layer (Issue #5), remote server management (Issue #43), and Caddyfile import functionality with UI confirmation workflow.
|
|
|
|
## Components Implemented
|
|
|
|
### Data Models (Issue #5)
|
|
**Location**: `backend/internal/models/`
|
|
|
|
- **RemoteServer** (`remote_server.go`): Backend server registry with provider, host, port, scheme, tags, enabled status, and reachability tracking
|
|
- **SSLCertificate** (`ssl_certificate.go`): TLS certificate management (Let's Encrypt, custom, self-signed) with auto-renew support
|
|
- **AccessList** (`access_list.go`): IP-based and auth-based access control rules (allow/deny/basic_auth/forward_auth)
|
|
- **User** (`user.go`): Authenticated users with role-based access (admin/user/viewer), password hash, last login
|
|
- **Setting** (`setting.go`): Global key-value configuration store with type and category
|
|
- **ImportSession** (`import_session.go`): Caddyfile import workflow tracking with pending/reviewing/committed/rejected states
|
|
|
|
### Service Layer
|
|
**Location**: `backend/internal/services/`
|
|
|
|
- **ProxyHostService** (`proxyhost_service.go`): Business logic for proxy hosts with domain uniqueness validation
|
|
- **RemoteServerService** (`remoteserver_service.go`): Remote server management with name/host:port uniqueness checks
|
|
|
|
### API Handlers (Issue #43)
|
|
**Location**: `backend/internal/api/handlers/`
|
|
|
|
- **RemoteServerHandler** (`remote_server_handler.go`): Full CRUD endpoints for remote server management
|
|
- `GET /api/v1/remote-servers` - List all (with optional ?enabled=true filter)
|
|
- `POST /api/v1/remote-servers` - Create new server
|
|
- `GET /api/v1/remote-servers/:uuid` - Get by UUID
|
|
- `PUT /api/v1/remote-servers/:uuid` - Update existing
|
|
- `DELETE /api/v1/remote-servers/:uuid` - Delete server
|
|
|
|
### Caddyfile Import
|
|
**Location**: `backend/internal/caddy/`
|
|
|
|
- **Importer** (`importer.go`): Comprehensive Caddyfile parsing and conversion
|
|
- `ParseCaddyfile()`: Executes `caddy adapt` to convert Caddyfile → JSON
|
|
- `ExtractHosts()`: Parses Caddy JSON and extracts proxy host information
|
|
- `ConvertToProxyHosts()`: Transforms parsed data to CPM+ models
|
|
- Conflict detection for duplicate domains
|
|
- Unsupported directive warnings (rewrites, file_server, etc.)
|
|
- Automatic Caddyfile backup to timestamped files
|
|
|
|
- **ImportHandler** (`backend/internal/api/handlers/import_handler.go`): Import workflow API
|
|
- `GET /api/v1/import/status` - Check for pending import sessions
|
|
- `GET /api/v1/import/preview` - Get parsed hosts + conflicts for review
|
|
- `POST /api/v1/import/upload` - Manual Caddyfile paste/upload
|
|
- `POST /api/v1/import/commit` - Finalize import with conflict resolutions
|
|
- `DELETE /api/v1/import/cancel` - Discard pending import
|
|
- `CheckMountedImport()`: Startup function to detect `/import/Caddyfile`
|
|
|
|
### Configuration Updates
|
|
**Location**: `backend/internal/config/config.go`
|
|
|
|
Added environment variables:
|
|
- `CPM_CADDY_BINARY`: Path to Caddy executable (default: `caddy`)
|
|
- `CPM_IMPORT_CADDYFILE`: Mount point for existing Caddyfile (default: `/import/Caddyfile`)
|
|
- `CPM_IMPORT_DIR`: Directory for import artifacts (default: `data/imports`)
|
|
|
|
### Application Entrypoint
|
|
**Location**: `backend/cmd/api/main.go`
|
|
|
|
- Initializes all services and handlers
|
|
- Registers import routes with config dependencies
|
|
- Checks for mounted Caddyfile on startup
|
|
- Logs warnings if import processing fails (non-fatal)
|
|
|
|
### Docker Integration
|
|
**Location**: `docker-compose.yml`
|
|
|
|
Added environment variables and volume mount comment:
|
|
```yaml
|
|
environment:
|
|
- CPM_CADDY_BINARY=caddy
|
|
- CPM_IMPORT_CADDYFILE=/import/Caddyfile
|
|
- CPM_IMPORT_DIR=/app/data/imports
|
|
|
|
volumes:
|
|
# Mount your existing Caddyfile for automatic import (optional)
|
|
# - ./my-existing-Caddyfile:/import/Caddyfile:ro
|
|
```
|
|
|
|
### Database Migrations
|
|
**Location**: `backend/internal/api/routes/routes.go`
|
|
|
|
Updated `AutoMigrate` to include all new models:
|
|
- ProxyHost, CaddyConfig (existing)
|
|
- RemoteServer, SSLCertificate, AccessList, User, Setting, ImportSession (new)
|
|
|
|
## Import Workflow
|
|
|
|
### Docker Mount Scenario
|
|
1. User bind-mounts existing Caddyfile: `-v ./Caddyfile:/import/Caddyfile:ro`
|
|
2. CPM+ detects file on startup via `CheckMountedImport()`
|
|
3. Parses Caddyfile → Caddy JSON → extracts hosts
|
|
4. Creates `ImportSession` with status='pending'
|
|
5. Frontend shows banner: "Import detected: X hosts found, Y conflicts"
|
|
6. User clicks to review → sees table with detected hosts, conflicts, actions
|
|
7. User resolves conflicts (skip/rename/merge) and clicks "Import"
|
|
8. Backend commits approved hosts to database
|
|
9. Generates per-host JSON files in `data/caddy/sites/`
|
|
10. Archives original Caddyfile to `data/imports/backups/<timestamp>.backup`
|
|
|
|
### Manual Upload Scenario
|
|
1. User clicks "Import Caddyfile" in UI
|
|
2. Pastes Caddyfile content or uploads file
|
|
3. POST to `/api/v1/import/upload` processes content
|
|
4. Same review flow as mount scenario (steps 5-10)
|
|
|
|
## Conflict Resolution
|
|
When importing, system detects:
|
|
- Duplicate domains (within Caddyfile or vs existing CPM+ hosts)
|
|
- Unsupported directives (rewrite, file_server, custom handlers)
|
|
|
|
User actions:
|
|
- **Skip**: Don't import this host
|
|
- **Rename**: Auto-append `-imported` suffix to domain
|
|
- **Merge**: Replace existing host with imported config (future enhancement)
|
|
|
|
## Security Considerations
|
|
- Import APIs require authentication (admin role from Issue #5 User model)
|
|
- Caddyfile parsing sandboxed via `exec.Command()` with timeout
|
|
- Original files backed up before any modifications
|
|
- Import session stores audit trail (who imported, when, what resolutions)
|
|
|
|
## Next Steps (Remaining Work)
|
|
|
|
### Frontend Components
|
|
1. **RemoteServers Page** (`frontend/src/pages/RemoteServers.tsx`)
|
|
- List/grid view with enable/disable toggle
|
|
- Create/edit form with provider dropdown
|
|
- Reachability status indicators
|
|
- Integration into ProxyHosts form as dropdown
|
|
|
|
2. **Import Review UI** (`frontend/src/pages/ImportCaddy.tsx`)
|
|
- Banner/modal for pending imports
|
|
- Table showing detected hosts with conflict warnings
|
|
- Action buttons (Skip, Rename) per host
|
|
- Diff preview of changes
|
|
- Commit/Cancel buttons
|
|
|
|
3. **Hooks**
|
|
- `frontend/src/hooks/useRemoteServers.ts`: CRUD operations
|
|
- `frontend/src/hooks/useImport.ts`: Import workflow state management
|
|
|
|
### Testing
|
|
1. **Handler Tests** (`backend/internal/api/handlers/*_test.go`)
|
|
- RemoteServer CRUD tests mirroring `proxy_host_handler_test.go`
|
|
- Import workflow tests (upload, preview, commit, cancel)
|
|
|
|
2. **Service Tests** (`backend/internal/services/*_test.go`)
|
|
- Uniqueness validation tests
|
|
- Domain conflict detection
|
|
|
|
3. **Importer Tests** (`backend/internal/caddy/importer_test.go`)
|
|
- Caddyfile parsing with fixtures in `testdata/`
|
|
- Host extraction edge cases
|
|
- Conflict detection scenarios
|
|
|
|
### Per-Host JSON Files
|
|
Currently `caddy/manager.go` generates monolithic config. Enhance:
|
|
1. `GenerateConfig()`: Create per-host JSON files in `data/caddy/sites/<uuid>.json`
|
|
2. `ApplyConfig()`: Compose aggregate from individual files
|
|
3. Rollback: Revert specific host file without affecting others
|
|
|
|
### Documentation
|
|
1. Update `README.md`: Import workflow instructions
|
|
2. Create `docs/import-guide.md`: Detailed import process, conflict resolution examples
|
|
3. Update `VERSION.md`: Document import feature as part of v0.2.0
|
|
4. Update `DOCKER.md`: Volume mount examples, environment variables
|
|
|
|
## Known Limitations
|
|
- Unsupported Caddyfile directives stored as warnings, not imported
|
|
- Single-upstream only (multi-upstream load balancing planned for later)
|
|
- No authentication/authorization yet (depends on Issue #5 User/Auth implementation)
|
|
- Per-host JSON files not yet implemented (monolithic config still used)
|
|
- Frontend components not yet implemented
|
|
|
|
## Testing Notes
|
|
- Go module initialized (`backend/go.mod`)
|
|
- Dependencies require `go mod tidy` or `go get` (network issues during implementation)
|
|
- Compilation verified structurally sound
|
|
- Integration tests require actual Caddy binary in PATH
|
|
|
|
## Files Modified
|
|
- `backend/internal/api/routes/routes.go`: Added migrations, import handler registration
|
|
- `backend/internal/config/config.go`: Added import-related env vars
|
|
- `docker-compose.yml`: Added import env vars and volume mount comment
|
|
|
|
## Files Created
|
|
### Models
|
|
- `backend/internal/models/remote_server.go`
|
|
- `backend/internal/models/ssl_certificate.go`
|
|
- `backend/internal/models/access_list.go`
|
|
- `backend/internal/models/user.go`
|
|
- `backend/internal/models/setting.go`
|
|
- `backend/internal/models/import_session.go`
|
|
|
|
### Services
|
|
- `backend/internal/services/proxyhost_service.go`
|
|
- `backend/internal/services/remoteserver_service.go`
|
|
|
|
### Handlers
|
|
- `backend/internal/api/handlers/remote_server_handler.go`
|
|
- `backend/internal/api/handlers/import_handler.go`
|
|
|
|
### Caddy Integration
|
|
- `backend/internal/caddy/importer.go`
|
|
|
|
### Application
|
|
- `backend/cmd/api/main.go`
|
|
- `backend/go.mod`
|
|
|
|
## Dependencies Required
|
|
```go
|
|
// go.mod
|
|
module github.com/Wikid82/CaddyProxyManagerPlus/backend
|
|
|
|
go 1.22
|
|
|
|
require (
|
|
github.com/gin-gonic/gin v1.11.0
|
|
github.com/google/uuid v1.6.0
|
|
gorm.io/gorm v1.31.1
|
|
gorm.io/driver/sqlite v1.6.0
|
|
)
|
|
```
|
|
|
|
Run `go mod tidy` to fetch dependencies when network is stable.
|