chore: Implement manual test plan for SMTP mock server flakiness fix

- Added a new documentation file outlining the manual test plan to validate the SMTP mock server flakiness fix, ensuring improved mail test reliability without affecting production behavior.
- Updated the current specification document to reflect the focus on stabilizing flaky SMTP STARTTLS+AUTH unit tests, including detailed research findings and requirements for the implementation.
- Created a QA/Security validation report for the SMTP flaky test fix, confirming that changes are test-only, stable under repeated runs, and do not introduce new security risks.
This commit is contained in:
GitHub Actions
2026-02-22 06:29:37 +00:00
parent b14f6f040f
commit 7497cbecd0
4 changed files with 542 additions and 467 deletions

View File

@@ -0,0 +1,48 @@
---
title: Manual Test Plan - SMTP Mock Server Flakiness Fix
status: Open
priority: High
assignee: QA
labels: testing, backend, reliability
---
# Test Objective
Confirm the SMTP mock server flakiness fix improves mail test reliability without changing production mail behavior.
# Scope
- In scope: test reliability for SMTP mock server flows used by backend mail tests.
- Out of scope: production SMTP sending behavior and user-facing mail features.
# Prerequisites
- Charon repository is up to date.
- Backend test environment is available.
- Ability to run backend tests repeatedly.
# Manual Scenarios
## 1) Target flaky test repeated run
- [ ] Run `TestMailService_TestConnection_StartTLSSuccessWithAuth` repeatedly (at least 20 times).
- [ ] Record pass/fail count and any intermittent errors.
## 2) Mail service targeted subset run
- [ ] Run mail service connection and send test subset once.
- [ ] Confirm no new intermittent failures appear in related tests.
## 3) Race-focused verification
- [ ] Run targeted mail service tests with race detection enabled.
- [ ] Confirm no race warnings or hangs occur.
## 4) Cleanup/shutdown stability check
- [ ] Repeat targeted runs and watch for stuck test processes or timeout behavior.
- [ ] Confirm test execution exits cleanly each run.
# Expected Results
- Repeated target test runs complete with zero flaky failures.
- Related mail service test subset remains stable.
- No race detector findings for targeted scenarios.
- No hangs during test cleanup/shutdown.
# Regression Checks (No Production Impact)
- [ ] Confirm only test reliability behavior changed; no production mail behavior changes are required.
- [ ] Confirm no production endpoints, settings, or user-facing mail flows are affected.
- [ ] Confirm standard backend test workflow still completes successfully after this fix.