feat: add nightly branch workflow

2026-01-13 22:11:28 +00:00
parent d27c925ba5
commit 4adcd9eda1
187 changed files with 8897 additions and 1614 deletions
--- a/docs/implementation/SSRF_COMPLETE.md
+++ b/docs/implementation/SSRF_COMPLETE.md
@@ -35,12 +35,14 @@ This document provides a comprehensive summary of the complete Server-Side Reque

 **Attack Scenario**:
 An authenticated admin user could supply a URL pointing to internal resources (localhost, private networks, cloud metadata endpoints), causing the server to make requests to these targets. This could lead to:
+
 - Information disclosure about internal network topology
 - Access to cloud provider metadata services (AWS: 169.254.169.254)
 - Port scanning of internal services
 - Exploitation of trust relationships

 **Original Code Flow**:
+
 ```
 User Input (req.URL)
    ↓
@@ -108,15 +110,18 @@ The complete remediation implements a four-layer security model:
 #### Key Functions

 ##### `ssrfSafeDialer()` (Lines 15-45)
+
 **Purpose**: Custom HTTP dialer that validates IP addresses at connection time

 **Security Controls**:
+
 - DNS resolution with context timeout (prevents DNS slowloris)
 - Validates **ALL** resolved IPs before connection (prevents IP hopping)
 - Uses first valid IP only (prevents DNS rebinding)
 - Atomic resolution → validation → connection sequence (prevents TOCTOU)

 **Code Snippet**:
+
 ```go
 func ssrfSafeDialer() func(ctx context.Context, network, addr string) (net.Conn, error) {
    return func(ctx context.Context, network, addr string) (net.Conn, error) {
@@ -147,14 +152,17 @@ func ssrfSafeDialer() func(ctx context.Context, network, addr string) (net.Conn,
 ```

 **Why This Works**:
+
 1. DNS resolution happens **inside the dialer**, at the moment of connection
 2. Even if DNS changes between validations, the second resolution catches it
 3. All IPs are validated (prevents round-robin DNS bypass)

 ##### `TestURLConnectivity()` (Lines 55-133)
+
 **Purpose**: Server-side URL connectivity testing with SSRF protection

 **Security Controls**:
+
 - Scheme validation (http/https only) - blocks `file://`, `ftp://`, `gopher://`, etc.
 - Integration with `ssrfSafeDialer()` for runtime protection
 - Redirect protection (max 2 redirects)
@@ -162,6 +170,7 @@ func ssrfSafeDialer() func(ctx context.Context, network, addr string) (net.Conn,
 - Custom User-Agent header

 **Code Snippet**:
+
 ```go
 // Create HTTP client with SSRF-safe dialer
 transport := &http.Transport{
@@ -182,9 +191,11 @@ client := &http.Client{
 ```

 ##### `isPrivateIP()` (Lines 136-182)
+
 **Purpose**: Comprehensive IP address validation

 **Protected Ranges** (13+ CIDR blocks):
+
 - ✅ RFC 1918 Private IPv4: `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`
 - ✅ Loopback: `127.0.0.0/8`, `::1/128`
 - ✅ Link-local (AWS/GCP metadata): `169.254.0.0/16`, `fe80::/10`
@@ -194,6 +205,7 @@ client := &http.Client{
 - ✅ IPv6 Documentation: `2001:db8::/32`

 **Code Snippet**:
+
 ```go
 // Cloud metadata service protection (critical!)
 _, linkLocal, _ := net.ParseCIDR("169.254.0.0/16")
@@ -215,6 +227,7 @@ if linkLocal.Contains(ip) {
 #### TestPublicURL Handler (Lines 269-325)

 **Access Control**:
+
 ```go
 // Requires admin role
 role, exists := c.Get("role")
@@ -227,6 +240,7 @@ if !exists || role != "admin" {
 **Validation Layers**:

 **Step 1: Format Validation**
+
 ```go
 normalized, _, err := utils.ValidateURL(req.URL)
 if err != nil {
@@ -239,6 +253,7 @@ if err != nil {
 ```

 **Step 2: SSRF Pre-Validation (Critical - Breaks Taint Chain)**
+
 ```go
 // This step breaks the CodeQL taint chain by returning a NEW validated value
 validatedURL, err := security.ValidateExternalURL(normalized, security.WithAllowHTTP())
@@ -254,18 +269,21 @@ if err != nil {
 ```

 **Why This Breaks the Taint Chain**:
+
 1. `security.ValidateExternalURL()` performs DNS resolution and IP validation
 2. Returns a **new string value** (not a passthrough)
 3. CodeQL's taint tracking sees the data flow break here
 4. The returned `validatedURL` is treated as untainted

 **Step 3: Connectivity Test**
+
 ```go
 // Use validatedURL (NOT req.URL) for network operation
 reachable, latency, err := utils.TestURLConnectivity(validatedURL)
 ```

 **HTTP Status Code Strategy**:
+
 - `400 Bad Request` → Format validation failures (invalid scheme, paths, malformed JSON)
 - `200 OK` → SSRF blocks and connectivity failures (returns `reachable: false` with error details)
 - `403 Forbidden` → Non-admin users
@@ -273,6 +291,7 @@ reachable, latency, err := utils.TestURLConnectivity(validatedURL)
 **Rationale**: SSRF blocks are connectivity constraints, not request format errors. Returning 200 allows clients to distinguish between "URL malformed" vs "URL blocked by security policy".

 **Documentation**:
+
 ```go
 // TestPublicURL performs a server-side connectivity test with comprehensive SSRF protection.
 // This endpoint implements defense-in-depth security:
@@ -291,16 +310,19 @@ reachable, latency, err := utils.TestURLConnectivity(validatedURL)
 ### 4.1 DNS Rebinding / TOCTOU Attacks

 **Attack Scenario**:
+
 1. **Check Time (T1)**: Handler calls `ValidateExternalURL()` which resolves `attacker.com` → `1.2.3.4` (public IP) ✅
 2. Attacker changes DNS record
 3. **Use Time (T2)**: `TestURLConnectivity()` resolves `attacker.com` again → `127.0.0.1` (private IP) ❌ SSRF!

 **Our Defense**:
+
 - `ssrfSafeDialer()` performs **second DNS resolution** at connection time
 - Even if DNS changes between T1 and T2, Layer 4 catches the attack
 - Atomic sequence: resolve → validate → connect (no window for rebinding)

 **Test Evidence**:
+
 ```
 ✅ TestSettingsHandler_TestPublicURL_SSRFProtection/blocks_localhost (0.00s)
 ✅ TestSettingsHandler_TestPublicURL_SSRFProtection/blocks_127.0.0.1 (0.00s)
@@ -309,15 +331,18 @@ reachable, latency, err := utils.TestURLConnectivity(validatedURL)
 ### 4.2 URL Parser Differential Attacks

 **Attack Scenario**:
+
 ```
 http://evil.com@127.0.0.1/
 ```

 Some parsers interpret this as:
+
 - User: `evil.com`
 - Host: `127.0.0.1` ← SSRF target

 **Our Defense**:
+
 ```go
 // In security/url_validator.go
 if parsed.User != nil {
@@ -326,6 +351,7 @@ if parsed.User != nil {
 ```

 **Test Evidence**:
+
 ```
 ✅ TestSettingsHandler_TestPublicURL_EmbeddedCredentials (0.00s)
 ```
@@ -333,11 +359,13 @@ if parsed.User != nil {
 ### 4.3 Cloud Metadata Endpoint Access

 **Attack Scenario**:
+
 ```
 http://169.254.169.254/latest/meta-data/iam/security-credentials/
 ```

 **Our Defense**:
+
 ```go
 // Both Layer 2 and Layer 4 block link-local ranges
 _, linkLocal, _ := net.ParseCIDR("169.254.0.0/16")
@@ -347,6 +375,7 @@ if linkLocal.Contains(ip) {
 ```

 **Test Evidence**:
+
 ```
 ✅ TestSettingsHandler_TestPublicURL_PrivateIPBlocked/blocks_cloud_metadata (0.00s)
 ✅ TestSettingsHandler_TestPublicURL_SSRFProtection/blocks_cloud_metadata (0.00s)
@@ -355,6 +384,7 @@ if linkLocal.Contains(ip) {
 ### 4.4 Protocol Smuggling

 **Attack Scenario**:
+
 ```
 file:///etc/passwd
 ftp://internal.server/data
@@ -362,6 +392,7 @@ gopher://internal.server:70/
 ```

 **Our Defense**:
+
 ```go
 // Layer 1: Format validation
 if parsed.Scheme != "http" && parsed.Scheme != "https" {
@@ -370,6 +401,7 @@ if parsed.Scheme != "http" && parsed.Scheme != "https" {
 ```

 **Test Evidence**:
+
 ```
 ✅ TestSettingsHandler_TestPublicURL_InvalidScheme/ftp_scheme (0.00s)
 ✅ TestSettingsHandler_TestPublicURL_InvalidScheme/file_scheme (0.00s)
@@ -379,11 +411,13 @@ if parsed.Scheme != "http" && parsed.Scheme != "https" {
 ### 4.5 Redirect Chain Abuse

 **Attack Scenario**:
+
 1. Request: `https://evil.com/redirect`
 2. Redirect 1: `http://evil.com/redirect2`
 3. Redirect 2: `http://127.0.0.1/admin`

 **Our Defense**:
+
 ```go
 client := &http.Client{
    CheckRedirect: func(req *http.Request, via []*http.Request) error {
@@ -445,6 +479,7 @@ client := &http.Client{
 **Backend Overall**: 86.4% (exceeds 85% threshold)

 **SSRF Protection Modules**:
+
 - `internal/api/handlers/settings_handler.go`: 100% (TestPublicURL handler)
 - `internal/utils/url_testing.go`: 88.0% (Runtime protection)
 - `internal/security/url_validator.go`: 100% (ValidateExternalURL)
@@ -493,6 +528,7 @@ Sink: http.NewRequestWithContext() - no taint detected
 ```

 **Why This Works**:
+
 1. `ValidateExternalURL()` performs DNS resolution and IP validation
 2. Returns a **new string value**, not a passthrough
 3. Static analysis sees data transformation: tainted input → validated output
@@ -503,6 +539,7 @@ Sink: http.NewRequestWithContext() - no taint detected
 ### 6.3 Expected CodeQL Result

 After implementation:
+
 - ✅ `go/ssrf` finding should be cleared
 - ✅ No new findings introduced
 - ✅ Future scans should not flag this pattern
@@ -522,6 +559,7 @@ After implementation:
 | Valid public URL | 200 | `{"reachable": true/false, "latency": ...}` | Normal operation |

 **Why 200 for SSRF Blocks?**:
+
 - SSRF validation is a *connectivity constraint*, not a request format error
 - Frontend expects 200 with structured JSON containing `reachable` boolean
 - Allows clients to distinguish: "URL malformed" (400) vs "URL blocked by policy" (200)
@@ -532,6 +570,7 @@ After implementation:
 ### 7.2 Response Format

 **Success (public URL reachable)**:
+
 ```json
 {
  "reachable": true,
@@ -541,6 +580,7 @@ After implementation:
 ```

 **SSRF Block**:
+
 ```json
 {
  "reachable": false,
@@ -550,6 +590,7 @@ After implementation:
 ```

 **Format Error**:
+
 ```json
 {
  "reachable": false,
@@ -576,6 +617,7 @@ After implementation:
 ### 8.2 CWE-918 Mitigation

 **Mitigated Attack Vectors**:
+
 1. ✅ DNS Rebinding: Atomic validation at connection time
 2. ✅ Cloud Metadata Access: 169.254.0.0/16 explicitly blocked
 3. ✅ Private Network Access: RFC 1918 ranges blocked
@@ -590,6 +632,7 @@ After implementation:
 ### 9.1 Latency Analysis

 **Added Overhead**:
+
 - DNS resolution (Layer 2): ~10-50ms (typical)
 - IP validation (Layer 2): <1ms (in-memory CIDR checks)
 - DNS re-resolution (Layer 4): ~10-50ms (typical)
@@ -612,6 +655,7 @@ After implementation:
 ### 10.1 Logging

 **SSRF Blocks are Logged**:
+
 ```go
 log.WithFields(log.Fields{
    "url": rawURL,
@@ -627,6 +671,7 @@ log.WithFields(log.Fields{
 ### 10.2 Monitoring

 **Metrics to Monitor**:
+
 - SSRF block count (aggregated from logs)
 - TestPublicURL endpoint latency (should remain <500ms for public URLs)
 - DNS resolution failures
@@ -652,12 +697,12 @@ log.WithFields(log.Fields{

 ### Standards and Guidelines

- **OWASP SSRF**: https://owasp.org/www-community/attacks/Server_Side_Request_Forgery
- **CWE-918**: https://cwe.mitre.org/data/definitions/918.html
- **RFC 1918 (Private IPv4)**: https://datatracker.ietf.org/doc/html/rfc1918
- **RFC 4193 (IPv6 Unique Local)**: https://datatracker.ietf.org/doc/html/rfc4193
- **DNS Rebinding Attacks**: https://en.wikipedia.org/wiki/DNS_rebinding
- **TOCTOU Vulnerabilities**: https://cwe.mitre.org/data/definitions/367.html
+- **OWASP SSRF**: <https://owasp.org/www-community/attacks/Server_Side_Request_Forgery>
+- **CWE-918**: <https://cwe.mitre.org/data/definitions/918.html>
+- **RFC 1918 (Private IPv4)**: <https://datatracker.ietf.org/doc/html/rfc1918>
+- **RFC 4193 (IPv6 Unique Local)**: <https://datatracker.ietf.org/doc/html/rfc4193>
+- **DNS Rebinding Attacks**: <https://en.wikipedia.org/wiki/DNS_rebinding>
+- **TOCTOU Vulnerabilities**: <https://cwe.mitre.org/data/definitions/367.html>

 ### Implementation Files