fix: improve import session management with enhanced cleanup and status handling
This commit is contained in:
@@ -126,7 +126,7 @@ graph TB
|
||||
| **HTTP Framework** | Gin | Latest | Routing, middleware, HTTP handling |
|
||||
| **Database** | SQLite | 3.x | Embedded database |
|
||||
| **ORM** | GORM | Latest | Database abstraction layer |
|
||||
| **Reverse Proxy** | Caddy Server | 2.11.0-beta.2 | Embedded HTTP/HTTPS proxy |
|
||||
| **Reverse Proxy** | Caddy Server | 2.11.1 | Embedded HTTP/HTTPS proxy |
|
||||
| **WebSocket** | gorilla/websocket | Latest | Real-time log streaming |
|
||||
| **Crypto** | golang.org/x/crypto | Latest | Password hashing, encryption |
|
||||
| **Metrics** | Prometheus Client | Latest | Application metrics |
|
||||
@@ -1259,6 +1259,14 @@ go test ./integration/...
|
||||
9. **Release Notes:** Generate changelog from commits
|
||||
10. **Notify:** Send release notification (Discord, email)
|
||||
|
||||
**Mandatory rollout gates (sign-off block):**
|
||||
|
||||
1. Digest freshness and index digest parity across GHCR and Docker Hub
|
||||
2. Per-arch digest parity across GHCR and Docker Hub
|
||||
3. SBOM and vulnerability scans against immutable refs (`image@sha256:...`)
|
||||
4. Artifact freshness timestamps after push
|
||||
5. Evidence block with required rollout verification fields
|
||||
|
||||
### Supply Chain Security
|
||||
|
||||
**Components:**
|
||||
@@ -1292,10 +1300,10 @@ cosign verify \
|
||||
wikid82/charon:latest
|
||||
|
||||
# Inspect SBOM
|
||||
syft wikid82/charon:latest -o json
|
||||
syft ghcr.io/wikid82/charon@sha256:<index-digest> -o json
|
||||
|
||||
# Scan for vulnerabilities
|
||||
grype wikid82/charon:latest
|
||||
grype ghcr.io/wikid82/charon@sha256:<index-digest>
|
||||
```
|
||||
|
||||
### Rollback Strategy
|
||||
|
||||
74
VERSION.md
74
VERSION.md
@@ -19,36 +19,76 @@ Example: `0.1.0-alpha`, `1.0.0-beta.1`, `2.0.0-rc.2`
|
||||
|
||||
## Creating a Release
|
||||
|
||||
### Automated Release Process
|
||||
### Canonical Release Process (Tag-Derived CI)
|
||||
|
||||
1. **Update version** in `.version` file:
|
||||
1. **Create and push a release tag**:
|
||||
|
||||
```bash
|
||||
echo "1.0.0" > .version
|
||||
git tag -a v1.0.0 -m "Release v1.0.0"
|
||||
git push origin v1.0.0
|
||||
```
|
||||
|
||||
2. **Commit version bump**:
|
||||
2. **GitHub Actions automatically**:
|
||||
- Runs release workflow from the pushed tag (`.github/workflows/release-goreleaser.yml`)
|
||||
- Builds and publishes release artifacts/images through CI (`.github/workflows/docker-build.yml`)
|
||||
- Creates/updates GitHub Release metadata
|
||||
|
||||
3. **Container tags are published**:
|
||||
- `v1.0.0` (exact version)
|
||||
- `1.0` (minor version)
|
||||
- `1` (major version)
|
||||
- `latest` (for non-prerelease on main branch)
|
||||
|
||||
### Legacy/Optional `.version` Path
|
||||
|
||||
The `.version` file is optional and not the canonical release trigger.
|
||||
|
||||
Use it only when you need local/version-file parity checks:
|
||||
|
||||
1. **Set `.version` locally (optional)**:
|
||||
|
||||
```bash
|
||||
git add .version
|
||||
git commit -m "chore: bump version to 1.0.0"
|
||||
echo "1.0.0" > .version
|
||||
```
|
||||
|
||||
3. **Create and push tag**:
|
||||
2. **Validate `.version` matches the latest tag**:
|
||||
|
||||
```bash
|
||||
git tag -a v1.0.0 -m "Release v1.0.0"
|
||||
git push origin v1.0.0
|
||||
bash scripts/check-version-match-tag.sh
|
||||
```
|
||||
|
||||
4. **GitHub Actions automatically**:
|
||||
- Creates GitHub Release with changelog
|
||||
- Builds multi-arch Docker images (amd64, arm64)
|
||||
- Publishes to GitHub Container Registry with tags:
|
||||
- `v1.0.0` (exact version)
|
||||
- `1.0` (minor version)
|
||||
- `1` (major version)
|
||||
- `latest` (for non-prerelease on main branch)
|
||||
### Deterministic Rollout Verification Gates (Mandatory)
|
||||
|
||||
Release sign-off is blocked until all items below pass in the same validation
|
||||
run.
|
||||
|
||||
Enforcement points:
|
||||
|
||||
- Release sign-off checklist/process (mandatory): All gates below remain required for release sign-off.
|
||||
- CI-supported checks (current): `.github/workflows/docker-build.yml` and `.github/workflows/supply-chain-verify.yml` enforce the subset currently implemented in workflows.
|
||||
- Manual validation required until CI parity: Validate any not-yet-implemented workflow gates via VS Code tasks `Security: Full Supply Chain Audit`, `Security: Verify SBOM`, `Security: Generate SLSA Provenance`, and `Security: Sign with Cosign`.
|
||||
- Optional version-file parity check: `Utility: Check Version Match Tag` (script: `scripts/check-version-match-tag.sh`).
|
||||
|
||||
- [ ] **Digest freshness/parity:** Capture pre-push and post-push index digests
|
||||
for the target tag in GHCR and Docker Hub, confirm expected freshness,
|
||||
and confirm cross-registry index digest parity.
|
||||
- [ ] **Per-arch parity:** Confirm per-platform (`linux/amd64`, `linux/arm64`,
|
||||
and any published platform) digest parity between GHCR and Docker Hub.
|
||||
- [ ] **Immutable digest scanning:** Run SBOM and vulnerability scans against
|
||||
immutable refs only, using `image@sha256:<index-digest>`.
|
||||
- [ ] **Artifact freshness:** Confirm scan artifacts are generated after the
|
||||
push timestamp and in the same validation run.
|
||||
- [ ] **Evidence block present:** Include the mandatory evidence block fields
|
||||
listed below.
|
||||
|
||||
#### Mandatory Evidence Block Fields
|
||||
|
||||
- Tag name
|
||||
- Index digest (`sha256:...`)
|
||||
- Per-arch digests (platform -> digest)
|
||||
- Scan tool versions
|
||||
- Push timestamp and scan timestamp(s)
|
||||
- Artifact file names generated in this run
|
||||
|
||||
## Container Image Tags
|
||||
|
||||
|
||||
@@ -1,356 +1,155 @@
|
||||
## 1. Introduction
|
||||
|
||||
### Overview
|
||||
Compatibility rollout for Caddy `2.11.1` is already reflected in the build
|
||||
default (`Dockerfile` currently sets `ARG CADDY_VERSION=2.11.1`).
|
||||
|
||||
`Nightly Build & Package` currently has two active workflow failures that must
|
||||
be fixed together in one minimal-scope PR:
|
||||
This plan is now focused on rollout verification and regression-proofing, not
|
||||
changing the default ARG.
|
||||
|
||||
1. SBOM generation failure in `Generate SBOM` (Syft fetch/version resolution).
|
||||
2. Dispatch failure from nightly workflow with `Missing required input
|
||||
'pr_number' not provided`.
|
||||
### Objective
|
||||
Establish deterministic, evidence-backed gates that prove published images and
|
||||
security artifacts are fresh, digest-bound, and aligned across registries for
|
||||
the Caddy `2.11.1` rollout.
|
||||
|
||||
This plan hard-locks runtime code changes to
|
||||
`.github/workflows/nightly-build.yml` only.
|
||||
## 2. Current State (Verified)
|
||||
|
||||
### Objectives
|
||||
|
||||
1. Restore deterministic nightly SBOM generation.
|
||||
2. Enforce strict default-deny dispatch behavior for non-PR nightly events
|
||||
(`schedule`, `workflow_dispatch`).
|
||||
3. Preserve GitHub Actions best practices: pinned SHAs, least privilege, and
|
||||
deterministic behavior.
|
||||
4. Keep both current failures in a single scope and do not pivot to unrelated fixes.
|
||||
5. Remove `security-pr.yml` from nightly dispatch list unless a hard
|
||||
requirement is proven.
|
||||
|
||||
## 2. Research Findings
|
||||
|
||||
### 2.1 Primary Workflow Scope
|
||||
|
||||
File analyzed: `.github/workflows/nightly-build.yml`
|
||||
|
||||
Relevant areas:
|
||||
|
||||
1. Job `build-and-push-nightly`, step `Generate SBOM` uses
|
||||
`anchore/sbom-action@17ae1740179002c89186b61233e0f892c3118b11`.
|
||||
2. Job `trigger-nightly-validation` dispatches downstream workflows using
|
||||
`actions/github-script` and currently includes `security-pr.yml`.
|
||||
|
||||
### 2.2 Root Cause: Missing `pr_number`
|
||||
|
||||
Directly related called workflow:
|
||||
|
||||
1. `.github/workflows/security-pr.yml`
|
||||
2. Trigger contract includes:
|
||||
- `workflow_dispatch.inputs.pr_number.required: true`
|
||||
|
||||
Impact:
|
||||
|
||||
1. Nightly dispatcher invokes `createWorkflowDispatch` for `security-pr.yml`
|
||||
without `pr_number`.
|
||||
2. For nightly non-PR contexts (scheduled/manual nightly), there is no natural
|
||||
PR number, so dispatch fails by contract.
|
||||
3. PR lookup by nightly head SHA is not a valid safety mechanism for nightly
|
||||
non-PR trigger types and must not be relied on for `schedule` or
|
||||
`workflow_dispatch`.
|
||||
|
||||
### 2.3 Decision: Remove PR-Only Workflow from Nightly Dispatch List
|
||||
|
||||
Assessment result:
|
||||
|
||||
1. No hard requirement was found that requires nightly workflow to dispatch
|
||||
`security-pr.yml`.
|
||||
2. `security-pr.yml` is contractually PR/manual-oriented because it requires
|
||||
`pr_number`.
|
||||
3. Keeping it in nightly fan-out adds avoidable failure risk and encourages
|
||||
invalid context synthesis.
|
||||
|
||||
Decision:
|
||||
|
||||
1. Remove `security-pr.yml` from nightly dispatch list.
|
||||
2. Keep strict default-deny guard logic to prevent accidental future dispatch
|
||||
from non-PR events.
|
||||
|
||||
Risk reduction from removal:
|
||||
|
||||
1. Eliminates `pr_number` contract mismatch in nightly non-PR events.
|
||||
2. Removes a class of false failures from nightly reliability metrics.
|
||||
3. Simplifies dispatcher logic and review surface.
|
||||
|
||||
### 2.4 Root Cause: SBOM/Syft Fetch Failure
|
||||
|
||||
Observed behavior indicates Syft retrieval/version resolution instability during
|
||||
the SBOM step. In current workflow, no explicit `syft-version` is set in
|
||||
`nightly-build.yml`, so resolution is not explicitly pinned at the workflow
|
||||
layer.
|
||||
|
||||
### 2.5 Constraints and Policy Alignment
|
||||
|
||||
1. Keep action SHAs pinned.
|
||||
2. Keep permission scopes unchanged unless required.
|
||||
3. Keep change minimal and limited to nightly workflow path only.
|
||||
1. `Dockerfile` default is already `CADDY_VERSION=2.11.1`.
|
||||
2. `ARCHITECTURE.md` now reports Caddy `2.11.1`.
|
||||
3. Existing scan artifacts can become stale if not explicitly tied to pushed
|
||||
digests.
|
||||
|
||||
## 3. Technical Specification (EARS)
|
||||
|
||||
1. WHEN nightly runs from `schedule` or `workflow_dispatch`, THE SYSTEM SHALL
|
||||
enforce strict default-deny for PR-only dispatches.
|
||||
1. WHEN image builds run without an explicit `CADDY_VERSION` override, THE
|
||||
SYSTEM SHALL continue producing Caddy `2.11.1`.
|
||||
2. WHEN an image tag is pushed, THE SYSTEM SHALL validate index digest parity
|
||||
between GHCR and Docker Hub for that same tag.
|
||||
3. WHEN multi-arch images are published, THE SYSTEM SHALL validate per-arch
|
||||
digest parity across GHCR and Docker Hub for each platform present.
|
||||
4. WHEN vulnerability and SBOM scans execute, THE SYSTEM SHALL scan
|
||||
`image@sha256:<index-digest>` instead of mutable tags.
|
||||
5. WHEN scan artifacts are generated, THE SYSTEM SHALL prove artifacts were
|
||||
produced after the push event in the same validation run.
|
||||
6. IF a verification gate fails, THEN THE SYSTEM SHALL block rollout sign-off
|
||||
until all gates pass.
|
||||
|
||||
2. WHEN nightly runs from `schedule` or `workflow_dispatch`, THE SYSTEM SHALL
|
||||
NOT perform PR-number lookup from nightly head SHA.
|
||||
## 4. Scope and Planned Edits
|
||||
|
||||
3. WHEN evaluating downstream nightly dispatches, THE SYSTEM SHALL exclude
|
||||
`security-pr.yml` from nightly dispatch targets unless a hard requirement
|
||||
is explicitly introduced and documented.
|
||||
### In scope
|
||||
1. `docs/plans/current_spec.md` (this plan refresh).
|
||||
2. `ARCHITECTURE.md` version sync is already complete (`2.11.1`); no pending
|
||||
update is required in this plan.
|
||||
3. Verification workflow/checklist updates needed to enforce deterministic gates.
|
||||
|
||||
4. IF `security-pr.yml` is reintroduced in the future, THEN THE SYSTEM SHALL
|
||||
dispatch it ONLY when a real PR context includes a concrete `pr_number`,
|
||||
and SHALL deny by default in all other contexts.
|
||||
### Out of scope
|
||||
1. No functional Caddy build logic changes unless a verification failure proves
|
||||
they are required.
|
||||
2. No plugin list or patch-scenario refactors.
|
||||
|
||||
5. WHEN `Generate SBOM` runs in nightly, THE SYSTEM SHALL use a deterministic
|
||||
two-stage strategy in the same PR scope:
|
||||
- Primary path: `syft-version: v1.42.1` via `anchore/sbom-action`
|
||||
- In-PR fallback path: explicit Syft CLI installation/generation
|
||||
with pinned version/checksum and hard verification
|
||||
## 5. Deterministic Acceptance Gates
|
||||
|
||||
6. IF primary SBOM generation fails or does not produce a valid file, THEN THE
|
||||
SYSTEM SHALL execute fallback generation and SHALL fail the job when fallback
|
||||
also fails or output validation fails.
|
||||
### Gate 1: Digest Freshness (pre/post push)
|
||||
1. Capture pre-push index digest for target tag on GHCR and Docker Hub.
|
||||
2. Push image.
|
||||
3. Capture post-push index digest on GHCR and Docker Hub.
|
||||
4. Pass criteria:
|
||||
- Post-push index digest changed as expected from pre-push (or matches
|
||||
intended new digest when creating new tag).
|
||||
- GHCR and Docker Hub index digests are identical for the tag.
|
||||
- Per-arch digests are identical across registries for each published
|
||||
platform.
|
||||
|
||||
7. THE SYSTEM SHALL keep GitHub Actions pinned to immutable SHAs and SHALL NOT
|
||||
broaden token permissions for this fix.
|
||||
### Gate 2: Digest-Bound Rescan
|
||||
1. Resolve the post-push index digest.
|
||||
2. Run all security scans against immutable ref:
|
||||
- `ghcr.io/<owner>/<repo>@sha256:<index-digest>`
|
||||
- Optional mirror check against Docker Hub digest ref.
|
||||
3. Pass criteria:
|
||||
- No scan uses mutable tags as the primary target.
|
||||
- Artifact metadata and logs show digest reference.
|
||||
|
||||
## 4. Exact Implementation Edits
|
||||
### Gate 3: Artifact Freshness
|
||||
1. Record push timestamp and digest capture timestamp.
|
||||
2. Generate SBOM and vuln artifacts after push in the same run.
|
||||
3. Pass criteria:
|
||||
- Artifact generation timestamps are greater than push timestamp.
|
||||
- Artifacts are newly created/overwritten in this run.
|
||||
- Evidence ties each artifact to the scanned digest.
|
||||
|
||||
### 4.1 `.github/workflows/nightly-build.yml`
|
||||
### Gate 4: Evidence Block (mandatory)
|
||||
Every validation run must include a structured evidence block with:
|
||||
1. Tag name.
|
||||
2. Index digest.
|
||||
3. Per-arch digests.
|
||||
4. Scan tool versions.
|
||||
5. Push and scan timestamps.
|
||||
6. Artifact file names produced in this run.
|
||||
|
||||
### Edit A: Harden downstream dispatch for non-PR triggers
|
||||
## 6. Implementation Plan
|
||||
|
||||
Location: job `trigger-nightly-validation`, step
|
||||
`Dispatch Missing Nightly Validation Workflows`.
|
||||
### Phase 1: Baseline Capture
|
||||
1. Confirm current `Dockerfile` default remains `2.11.1`.
|
||||
2. Capture pre-push digest state for target tag across both registries.
|
||||
|
||||
Exact change intent:
|
||||
### Phase 2: Docs Sync
|
||||
1. Confirm `ARCHITECTURE.md` remains synced at Caddy `2.11.1`.
|
||||
|
||||
1. Remove `security-pr.yml` from the nightly dispatch list.
|
||||
2. Keep dispatch for `e2e-tests-split.yml`, `codecov-upload.yml`,
|
||||
`supply-chain-verify.yml`, and `codeql.yml` unchanged.
|
||||
3. Add explicit guard comments and logging stating non-PR nightly events are
|
||||
default-deny for PR-only workflows.
|
||||
4. Explicitly prohibit PR number synthesis and prohibit PR lookup from nightly
|
||||
SHA for `schedule` and `workflow_dispatch`.
|
||||
### Phase 3: Push and Verification
|
||||
1. Push validation tag.
|
||||
2. Execute Gate 1 (digest freshness and parity).
|
||||
3. Execute Gate 2 (digest-bound rescan).
|
||||
4. Execute Gate 3 (artifact freshness).
|
||||
5. Produce Gate 4 evidence block.
|
||||
|
||||
Implementation shape (script-level):
|
||||
### Phase 4: Sign-off
|
||||
1. Mark rollout verified only when all gates pass.
|
||||
2. If any gate fails, open follow-up remediation task before merge.
|
||||
|
||||
1. Keep workflow list explicit.
|
||||
2. Keep a local denylist/set for PR-only workflows and ensure they are never
|
||||
dispatched from nightly non-PR events.
|
||||
3. No PR-number inputs are synthesized from nightly SHA or non-PR context.
|
||||
4. No PR lookup calls are executed for nightly non-PR events.
|
||||
## 7. Acceptance Criteria
|
||||
|
||||
### Edit B: Stabilize Syft source in `Generate SBOM`
|
||||
1. Plan and execution no longer assume Dockerfile default is beta.
|
||||
2. Objective is rollout verification/regression-proofing for Caddy `2.11.1`.
|
||||
3. `ARCHITECTURE.md` version metadata is included in required docs sync.
|
||||
4. Digest freshness gate passes:
|
||||
- Pre/post push validation completed.
|
||||
- GHCR and Docker Hub index digest parity confirmed.
|
||||
- Per-arch digest parity confirmed.
|
||||
5. Digest-bound rescan gate passes with `image@sha256` scan targets.
|
||||
6. Artifact freshness gate passes with artifacts produced after push in the same
|
||||
run.
|
||||
7. Evidence block is present and complete with:
|
||||
- Tag
|
||||
- Index digest
|
||||
- Per-arch digests
|
||||
- Scan tool versions
|
||||
- Timestamps
|
||||
- Artifact names
|
||||
|
||||
Location: job `build-and-push-nightly`, step `Generate SBOM`.
|
||||
|
||||
Exact change intent:
|
||||
|
||||
1. Keep existing pinned `anchore/sbom-action` SHA unless evidence shows that SHA
|
||||
itself is the failure source.
|
||||
2. Add explicit `syft-version: v1.42.1` in `with:` block as the primary pin.
|
||||
3. Set the primary SBOM step to `continue-on-error: true` to allow deterministic
|
||||
in-PR fallback execution.
|
||||
4. Add fallback step gated on primary step failure OR missing/invalid output:
|
||||
- Install Syft CLI `v1.42.1` from official release with checksum validation.
|
||||
- Generate `sbom-nightly.json` via CLI.
|
||||
5. Add mandatory verification step (no `continue-on-error`) with explicit
|
||||
pass/fail criteria:
|
||||
- `sbom-nightly.json` exists.
|
||||
- file size is greater than 0 bytes.
|
||||
- JSON parses successfully (`jq empty`).
|
||||
- expected top-level fields exist for selected format.
|
||||
6. If verification fails, job fails. SBOM cannot pass silently without
|
||||
generated artifact.
|
||||
|
||||
### 4.2 Scope Lock
|
||||
|
||||
1. No edits to `.github/workflows/security-pr.yml` in this plan.
|
||||
2. Contract remains unchanged: `workflow_dispatch.inputs.pr_number.required: true`.
|
||||
|
||||
## 5. Reconfirmation: Non-Target Files
|
||||
|
||||
No changes required:
|
||||
|
||||
1. `.gitignore`
|
||||
2. `codecov.yml`
|
||||
3. `.dockerignore`
|
||||
4. `Dockerfile`
|
||||
|
||||
Rationale:
|
||||
|
||||
1. Both failures are workflow orchestration issues, not source-ignore, coverage
|
||||
policy, Docker context, or image build recipe issues.
|
||||
|
||||
## 6. Risks and Mitigations
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|---|---|---|
|
||||
| `security-pr.yml` accidentally dispatched in non-PR mode | Low | Remove from nightly dispatch list and enforce default-deny comments/guards |
|
||||
| Primary Syft acquisition fails (`v1.42.1`) | Medium | Execute deterministic in-PR fallback with pinned checksum and hard output verification |
|
||||
| SBOM step appears green without real artifact | High | Mandatory verification step with explicit file/JSON checks and hard fail |
|
||||
| Action SHA update introduces side effects | Medium | Limit SHA change to `Generate SBOM` step only and validate end-to-end nightly path |
|
||||
| Over-dispatch/under-dispatch in validation job | Low | Preserve existing dispatch logic for all non-PR-dependent workflows |
|
||||
|
||||
## 7. Rollback Plan
|
||||
|
||||
1. Revert runtime behavior changes in
|
||||
`.github/workflows/nightly-build.yml`:
|
||||
- `trigger-nightly-validation` dispatch logic
|
||||
- `Generate SBOM` primary + fallback + verification sequence
|
||||
2. Re-run nightly dispatch manually to verify previous baseline runtime
|
||||
behavior.
|
||||
|
||||
Rollback scope: runtime workflow behavior only in
|
||||
`.github/workflows/nightly-build.yml`. Documentation updates are not part of
|
||||
runtime rollback.
|
||||
|
||||
## 8. Validation Plan
|
||||
|
||||
### 8.1 Static Validation
|
||||
|
||||
```bash
|
||||
cd /projects/Charon
|
||||
pre-commit run actionlint --files .github/workflows/nightly-build.yml
|
||||
```
|
||||
|
||||
### 8.2 Behavioral Validation (Nightly non-PR)
|
||||
|
||||
```bash
|
||||
gh workflow run nightly-build.yml --ref nightly -f reason="nightly dual-fix validation" -f skip_tests=true
|
||||
gh run list --workflow "Nightly Build & Package" --branch nightly --limit 1
|
||||
gh run view <run-id> --json databaseId,headSha,event,status,conclusion,createdAt
|
||||
gh run view <run-id> --log
|
||||
```
|
||||
|
||||
Expected outcomes:
|
||||
|
||||
1. `Generate SBOM` succeeds through primary path or deterministic fallback and
|
||||
`sbom-nightly.json` is uploaded.
|
||||
2. Dispatch step does not attempt `security-pr.yml` from nightly run.
|
||||
3. No `Missing required input 'pr_number' not provided` error.
|
||||
4. Both targeted nightly failures are resolved in the same run scope:
|
||||
`pr_number` dispatch failure and Syft/SBOM failure.
|
||||
|
||||
### 8.3 Explicit Negative Dispatch Verification (Run-Scoped/Time-Scoped)
|
||||
|
||||
Verify `security-pr.yml` was not dispatched by this specific nightly run using
|
||||
time scope and actor scope (not SHA-only):
|
||||
|
||||
```bash
|
||||
RUN_JSON=$(gh run view <nightly-run-id> --json databaseId,createdAt,updatedAt,event,headBranch)
|
||||
START=$(echo "$RUN_JSON" | jq -r '.createdAt')
|
||||
END=$(echo "$RUN_JSON" | jq -r '.updatedAt')
|
||||
|
||||
gh api repos/<owner>/<repo>/actions/workflows/security-pr.yml/runs \
|
||||
--paginate \
|
||||
-f event=workflow_dispatch | \
|
||||
jq --arg start "$START" --arg end "$END" '
|
||||
[ .workflow_runs[]
|
||||
| select(.created_at >= $start and .created_at <= $end)
|
||||
| select(.head_branch == "nightly")
|
||||
| select(.triggering_actor.login == "github-actions[bot]")
|
||||
] | length'
|
||||
```
|
||||
|
||||
Expected result: `0`
|
||||
|
||||
### 8.4 Positive Validation: Manual `security-pr.yml` Dispatch Still Works
|
||||
|
||||
Run a manual dispatch with a valid PR number and verify successful start:
|
||||
|
||||
```bash
|
||||
gh workflow run security-pr.yml --ref <pr-branch> -f pr_number=<valid-pr-number>
|
||||
gh run list --workflow "Security Scan (PR)" --limit 5 \
|
||||
--json databaseId,event,status,conclusion,createdAt,headBranch
|
||||
gh run view <security-pr-run-id> --log
|
||||
```
|
||||
|
||||
Expected results:
|
||||
|
||||
1. Workflow is accepted (no missing-input validation errors).
|
||||
2. Run event is `workflow_dispatch`.
|
||||
3. Run completes according to existing workflow behavior.
|
||||
|
||||
### 8.5 Contract Validation (No Contract Change)
|
||||
|
||||
1. `security-pr.yml` contract remains PR/manual specific and unchanged.
|
||||
2. Nightly non-PR paths do not consume or synthesize `pr_number`.
|
||||
|
||||
## 9. Acceptance Criteria
|
||||
|
||||
1. `Nightly Build & Package` no longer fails in `Generate SBOM` due to Syft
|
||||
fetch/version resolution, with deterministic in-PR fallback.
|
||||
2. Nightly validation dispatch no longer fails with missing required
|
||||
`pr_number`.
|
||||
3. For non-PR nightly triggers (`schedule`/`workflow_dispatch`), PR-only
|
||||
dispatch of `security-pr.yml` is default-deny and not attempted from nightly
|
||||
dispatch targets.
|
||||
4. Workflow remains SHA-pinned and permissions are not broadened.
|
||||
5. Validation evidence includes explicit run-scoped/time-scoped proof that
|
||||
`security-pr.yml` was not dispatched by the tested nightly run.
|
||||
6. No changes made to `.gitignore`, `codecov.yml`, `.dockerignore`, or
|
||||
`Dockerfile`.
|
||||
7. Manual dispatch of `security-pr.yml` with valid `pr_number` is validated to
|
||||
still work.
|
||||
8. SBOM step fails hard when neither primary nor fallback path produces a valid
|
||||
SBOM artifact.
|
||||
|
||||
## 10. PR Slicing Strategy
|
||||
## 8. PR Slicing Strategy
|
||||
|
||||
### Decision
|
||||
|
||||
Single PR.
|
||||
|
||||
### Trigger Reasons
|
||||
1. Scope is narrow and cross-cutting risk is low.
|
||||
2. Verification logic and docs sync are tightly coupled.
|
||||
3. Review size remains small and rollback is straightforward.
|
||||
|
||||
1. Changes are tightly coupled inside one workflow path.
|
||||
2. Shared validation path (nightly run) verifies both fixes together.
|
||||
3. Rollback safety is high with one-file revert.
|
||||
### PR-1
|
||||
1. Scope:
|
||||
- Refresh `docs/plans/current_spec.md` to verification-focused plan.
|
||||
- Sync `ARCHITECTURE.md` Caddy version metadata.
|
||||
- Add/adjust verification checklist content needed for gates.
|
||||
2. Dependencies:
|
||||
- Existing publish/scanning pipeline availability.
|
||||
3. Validation gates:
|
||||
- Gate 1 through Gate 4 all required.
|
||||
|
||||
### Ordered Slices
|
||||
## 9. Rollback and Contingency
|
||||
|
||||
#### PR-1: Nightly Dual-Failure Workflow Fix
|
||||
|
||||
Scope:
|
||||
|
||||
1. `.github/workflows/nightly-build.yml` only.
|
||||
2. SBOM Syft stabilization with explicit tag pin + fallback rule.
|
||||
3. Remove `security-pr.yml` from nightly dispatch list and enforce strict
|
||||
default-deny semantics for non-PR nightly events.
|
||||
|
||||
Files:
|
||||
|
||||
1. `.github/workflows/nightly-build.yml`
|
||||
2. `docs/plans/current_spec.md`
|
||||
|
||||
Dependencies:
|
||||
|
||||
1. `security-pr.yml` keeps required `workflow_dispatch` `pr_number` contract.
|
||||
|
||||
Validation gates:
|
||||
|
||||
1. `actionlint` passes.
|
||||
2. Nightly manual dispatch run passes both targeted failure points.
|
||||
3. SBOM artifact upload succeeds through primary path or fallback path.
|
||||
4. Explicit run-scoped/time-scoped negative check confirms zero
|
||||
bot-triggered `security-pr.yml` dispatches during the nightly run window.
|
||||
5. Positive manual dispatch check with valid `pr_number` succeeds.
|
||||
|
||||
Rollback and contingency:
|
||||
|
||||
1. Revert PR-1.
|
||||
2. If both primary and fallback Syft paths fail, treat as blocking regression
|
||||
and do not merge until generation criteria pass.
|
||||
|
||||
## 11. Complexity Estimate
|
||||
|
||||
1. Implementation complexity: Low.
|
||||
2. Validation complexity: Medium (requires workflow run completion).
|
||||
3. Blast radius: Low (single workflow file, no runtime code changes).
|
||||
1. If verification updates are incorrect or incomplete, revert PR-1.
|
||||
2. If rollout evidence fails, hold release sign-off and keep last known-good
|
||||
digest as active reference.
|
||||
3. Re-run verification with corrected commands/artifacts before reattempting
|
||||
sign-off.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# QA Report: Nightly Workflow Fix Audit
|
||||
double check our caddy version# QA Report: Nightly Workflow Fix Audit
|
||||
|
||||
- Date: 2026-02-27
|
||||
- Scope:
|
||||
|
||||
@@ -105,6 +105,10 @@ async function completeImportFlow(
|
||||
}
|
||||
|
||||
test.describe('Caddy Import Gap Coverage @caddy-import-gaps', () => {
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await resetImportSession(page);
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page }) => {
|
||||
await resetImportSession(page);
|
||||
});
|
||||
|
||||
@@ -239,17 +239,9 @@ export async function resetImportSession(page: Page): Promise<void> {
|
||||
// Best-effort navigation only
|
||||
}
|
||||
|
||||
try {
|
||||
const statusResponse = await page.request.get('/api/v1/import/status');
|
||||
if (statusResponse.ok()) {
|
||||
const statusBody = await statusResponse.json();
|
||||
if (statusBody?.has_pending) {
|
||||
await page.request.post('/api/v1/import/cancel');
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
await clearPendingImportSession(page).catch(() => {
|
||||
// Best-effort cleanup only
|
||||
}
|
||||
});
|
||||
|
||||
try {
|
||||
await page.goto(IMPORT_PAGE_PATH, { waitUntil: 'domcontentloaded' });
|
||||
@@ -258,6 +250,65 @@ export async function resetImportSession(page: Page): Promise<void> {
|
||||
}
|
||||
}
|
||||
|
||||
async function readImportStatus(page: Page): Promise<{ hasPending: boolean; sessionId: string }> {
|
||||
try {
|
||||
const statusResponse = await page.request.get('/api/v1/import/status');
|
||||
if (!statusResponse.ok()) {
|
||||
return { hasPending: false, sessionId: '' };
|
||||
}
|
||||
|
||||
const statusBody = (await statusResponse.json().catch(() => ({}))) as {
|
||||
has_pending?: boolean;
|
||||
session?: { id?: string };
|
||||
};
|
||||
|
||||
return {
|
||||
hasPending: Boolean(statusBody?.has_pending),
|
||||
sessionId: statusBody?.session?.id || '',
|
||||
};
|
||||
} catch {
|
||||
return { hasPending: false, sessionId: '' };
|
||||
}
|
||||
}
|
||||
|
||||
async function issuePendingSessionCancel(page: Page, sessionId: string): Promise<void> {
|
||||
if (sessionId) {
|
||||
await page
|
||||
.request
|
||||
.delete(`/api/v1/import/cancel?session_uuid=${encodeURIComponent(sessionId)}`)
|
||||
.catch(() => null);
|
||||
}
|
||||
|
||||
// Keep legacy endpoints for compatibility across backend variants.
|
||||
await page.request.delete('/api/v1/import/cancel').catch(() => null);
|
||||
await page.request.post('/api/v1/import/cancel').catch(() => null);
|
||||
}
|
||||
|
||||
async function clearPendingImportSession(page: Page): Promise<void> {
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
const status = await readImportStatus(page);
|
||||
if (!status.hasPending) {
|
||||
return;
|
||||
}
|
||||
|
||||
await issuePendingSessionCancel(page, status.sessionId);
|
||||
|
||||
await expect
|
||||
.poll(async () => {
|
||||
const next = await readImportStatus(page);
|
||||
return next.hasPending;
|
||||
}, {
|
||||
timeout: 3000,
|
||||
})
|
||||
.toBeFalsy();
|
||||
}
|
||||
|
||||
const finalStatus = await readImportStatus(page);
|
||||
if (finalStatus.hasPending) {
|
||||
throw new Error(`Unable to clear pending import session after retries (sessionId=${finalStatus.sessionId || 'unknown'})`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function ensureImportFormReady(page: Page): Promise<void> {
|
||||
await assertNoAuthRedirect(page, 'ensureImportFormReady initial check');
|
||||
|
||||
@@ -275,57 +326,24 @@ export async function ensureImportFormReady(page: Page): Promise<void> {
|
||||
}
|
||||
|
||||
const textarea = page.locator('textarea').first();
|
||||
const textareaVisible = await textarea.isVisible().catch(() => false);
|
||||
let textareaVisible = await textarea.isVisible().catch(() => false);
|
||||
if (!textareaVisible) {
|
||||
const pendingSessionVisible = await page.getByText(/pending import session/i).first().isVisible().catch(() => false);
|
||||
if (pendingSessionVisible) {
|
||||
diagnosticLog('[Diag:import-ready] pending import session detected, canceling to restore textarea');
|
||||
|
||||
const browserCancelStatus = await page
|
||||
.evaluate(async () => {
|
||||
const token = localStorage.getItem('charon_auth_token');
|
||||
const commonHeaders = token ? { Authorization: `Bearer ${token}` } : {};
|
||||
|
||||
const statusResponse = await fetch('/api/v1/import/status', {
|
||||
method: 'GET',
|
||||
credentials: 'include',
|
||||
headers: commonHeaders,
|
||||
});
|
||||
let sessionId = '';
|
||||
if (statusResponse.ok) {
|
||||
const statusBody = (await statusResponse.json()) as { session?: { id?: string } };
|
||||
sessionId = statusBody?.session?.id || '';
|
||||
}
|
||||
|
||||
const cancelUrl = sessionId
|
||||
? `/api/v1/import/cancel?session_uuid=${encodeURIComponent(sessionId)}`
|
||||
: '/api/v1/import/cancel';
|
||||
|
||||
const response = await fetch(cancelUrl, {
|
||||
method: 'DELETE',
|
||||
credentials: 'include',
|
||||
headers: commonHeaders,
|
||||
});
|
||||
return response.status;
|
||||
})
|
||||
.catch(() => null);
|
||||
diagnosticLog(`[Diag:import-ready] browser cancel status=${browserCancelStatus ?? 'n/a'}`);
|
||||
|
||||
const cancelButton = page.getByRole('button', { name: /^cancel$/i }).first();
|
||||
const cancelButtonVisible = await cancelButton.isVisible().catch(() => false);
|
||||
|
||||
if (cancelButtonVisible) {
|
||||
await Promise.all([
|
||||
page.waitForResponse((response) => response.url().includes('/api/v1/import/cancel'), { timeout: 10000 }).catch(() => null),
|
||||
cancelButton.click(),
|
||||
]);
|
||||
}
|
||||
|
||||
await clearPendingImportSession(page);
|
||||
await page.goto(IMPORT_PAGE_PATH, { waitUntil: 'domcontentloaded' });
|
||||
await assertNoAuthRedirect(page, 'ensureImportFormReady after pending-session reset');
|
||||
textareaVisible = await textarea.isVisible().catch(() => false);
|
||||
}
|
||||
}
|
||||
|
||||
if (!textareaVisible) {
|
||||
// One deterministic refresh recovers WebKit hydration timing without broad retries.
|
||||
await page.reload({ waitUntil: 'domcontentloaded' });
|
||||
await assertNoAuthRedirect(page, 'ensureImportFormReady after reload recovery');
|
||||
}
|
||||
|
||||
await expect(textarea).toBeVisible();
|
||||
await expect(page.getByRole('button', { name: /parse|review/i }).first()).toBeVisible();
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user