feat: add nightly branch workflow
This commit is contained in:
@@ -11,11 +11,13 @@ Manually verify that the CI workflow fixes work correctly in production, focusin
|
||||
## Background
|
||||
|
||||
**What Was Fixed:**
|
||||
|
||||
1. Removed `branches` filter from `supply-chain-verify.yml` to enable `workflow_run` triggering on all branches
|
||||
2. Added documentation to explain the GitHub Security warning (false positive)
|
||||
3. Updated SECURITY.md with comprehensive security scanning documentation
|
||||
|
||||
**Expected Behavior:**
|
||||
|
||||
- Supply Chain Verification should now trigger via `workflow_run` after Docker Build completes on ANY branch
|
||||
- Previous behavior: Only triggered via `pull_request` fallback (branch filter prevented workflow_run)
|
||||
|
||||
@@ -26,23 +28,27 @@ Manually verify that the CI workflow fixes work correctly in production, focusin
|
||||
**Goal:** Verify `workflow_run` trigger works on feature branches after fix
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Create a small test commit on `feature/beta-release`
|
||||
2. Push the commit
|
||||
3. Monitor GitHub Actions workflow runs
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Docker Build workflow triggers and completes successfully
|
||||
- ✅ Supply Chain Verification triggers **via workflow_run event** (not pull_request)
|
||||
- ✅ Supply Chain completes successfully
|
||||
- ✅ GitHub Actions logs show event type is `workflow_run`
|
||||
|
||||
**How to Verify Event Type:**
|
||||
|
||||
```bash
|
||||
gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusion
|
||||
# Should show: "event": "workflow_run", "conclusion": "success"
|
||||
```
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Supply Chain doesn't trigger at all
|
||||
- ❌ Supply Chain triggers but fails
|
||||
- ❌ Multiple simultaneous runs (race condition)
|
||||
@@ -55,16 +61,19 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
**Goal:** Verify `pull_request` fallback trigger still works correctly
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. With PR #461 open, push another small commit
|
||||
2. Monitor GitHub Actions workflow runs
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Docker Build triggers via `pull_request` event
|
||||
- ✅ Supply Chain may trigger via BOTH `workflow_run` AND `pull_request` (race condition possible)
|
||||
- ✅ If both trigger, both should complete successfully without conflict
|
||||
- ✅ PR should show both workflow checks passing
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Duplicate runs causing conflicts
|
||||
- ❌ Race condition causing failures
|
||||
- ❌ PR checks showing "pending" indefinitely
|
||||
@@ -77,16 +86,19 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
**Goal:** Verify fix doesn't break main branch behavior
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. After PR #461 merges to main, monitor the merge commit
|
||||
2. Check GitHub Actions runs
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Docker Build runs on main
|
||||
- ✅ Supply Chain triggers via `workflow_run`
|
||||
- ✅ Both complete successfully
|
||||
- ✅ Weekly scheduled runs continue to work
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Main branch workflows broken
|
||||
- ❌ Weekly schedule interferes with workflow_run
|
||||
- ❌ Permissions issues on main branch
|
||||
@@ -98,16 +110,19 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
**Goal:** Verify Supply Chain doesn't trigger when Docker Build fails
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Intentionally break Docker Build (e.g., invalid Dockerfile syntax)
|
||||
2. Push to a test branch
|
||||
3. Monitor workflow behavior
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Docker Build fails as expected
|
||||
- ✅ Supply Chain **does NOT trigger** (workflow_run only fires on `completed` and `success`)
|
||||
- ✅ No cascading failures
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Supply Chain triggers on failed builds
|
||||
- ❌ Error handling missing
|
||||
- ❌ Workflow stuck in pending state
|
||||
@@ -119,17 +134,20 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
**Goal:** Verify manual trigger still works
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Go to GitHub Actions → Supply Chain Verification
|
||||
2. Click "Run workflow"
|
||||
3. Select `feature/beta-release` branch
|
||||
4. Click "Run workflow"
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Workflow starts via `workflow_dispatch` event
|
||||
- ✅ Completes successfully
|
||||
- ✅ SBOM and attestations generated
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Manual dispatch broken
|
||||
- ❌ Branch selector doesn't work
|
||||
- ❌ Workflow fails with "branch not found"
|
||||
@@ -141,15 +159,18 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
**Goal:** Verify scheduled trigger still works
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Wait for next Monday 00:00 UTC
|
||||
2. Check GitHub Actions for scheduled run
|
||||
|
||||
**Expected Results:**
|
||||
|
||||
- ✅ Workflow triggers via `schedule` event
|
||||
- ✅ Runs on main branch
|
||||
- ✅ Completes successfully
|
||||
|
||||
**Potential Bugs to Watch For:**
|
||||
|
||||
- ❌ Schedule doesn't fire
|
||||
- ❌ Wrong branch selected
|
||||
- ❌ Interference with other workflows
|
||||
@@ -159,16 +180,19 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
## Edge Cases to Test
|
||||
|
||||
### Edge Case 1: Rapid Pushes (Rate Limiting)
|
||||
|
||||
**Test:** Push 3-5 commits rapidly to feature branch
|
||||
**Expected:** All Docker Builds run, Supply Chain may queue or skip redundant runs
|
||||
**Watch For:** Workflow queue overflow, cancellations, failures
|
||||
|
||||
### Edge Case 2: Long-Running Docker Build
|
||||
|
||||
**Test:** Create a commit that makes Docker Build take >10 minutes
|
||||
**Expected:** Supply Chain waits for completion before triggering
|
||||
**Watch For:** Timeouts, abandoned runs, state corruption
|
||||
|
||||
### Edge Case 3: Branch Deletion During Run
|
||||
|
||||
**Test:** Delete feature branch while workflows are running
|
||||
**Expected:** Workflows complete or cancel gracefully
|
||||
**Watch For:** Orphaned runs, resource leaks, errors
|
||||
@@ -187,21 +211,25 @@ gh run list --workflow="supply-chain-verify.yml" --limit 1 --json event,conclusi
|
||||
## Bug Severity Guidelines
|
||||
|
||||
**CRITICAL** (Block Merge):
|
||||
|
||||
- Supply Chain doesn't run at all
|
||||
- Cascading failures breaking other workflows
|
||||
- Security vulnerabilities introduced
|
||||
|
||||
**HIGH** (Fix Before Release):
|
||||
|
||||
- Race conditions causing frequent failures
|
||||
- Resource leaks or orphaned workflows
|
||||
- Error handling missing
|
||||
|
||||
**MEDIUM** (Fix in Future PR):
|
||||
|
||||
- Duplicate runs (but both succeed)
|
||||
- Inconsistent behavior (works sometimes)
|
||||
- Minor UX issues
|
||||
|
||||
**LOW** (Document as Known Issue):
|
||||
|
||||
- Cosmetic issues in logs
|
||||
- Non-breaking edge cases
|
||||
- Timing inconsistencies
|
||||
|
||||
Reference in New Issue
Block a user