feat(eval): add executor audit closure checks

2026-07-08 10:21:39 +08:00
parent 6015bcbf6f
commit a08672b31e
20 changed files with 1142 additions and 188 deletions
@@ -0,0 +1,23 @@
+## 1. Eval Contract
+
+- [x] 1.1 Extend `DiagnosisEvalCase` with optional V2 audit expectations.
+- [x] 1.2 Extend `DiagnosisEvalResult` and report output with observed audit signals.
+- [x] 1.3 Add deterministic evaluator checks for Gatekeeper, claim checks, Composer output, and raw JSON leakage.
+
+## 2. Fixtures and Baseline
+
+- [x] 2.1 Add fixed eval cases for Gatekeeper failure, unsupported claim filtering, and Composer fallback leakage prevention.
+- [x] 2.2 Add matching trace fixtures for each new case.
+- [x] 2.3 Regenerate baseline JSON and Markdown reports.
+
+## 3. Documentation
+
+- [x] 3.1 Update eval README with stage 5 scope and verification commands.
+- [x] 3.2 Rewrite eval schema documentation so V2 audit fields and case expectations are readable.
+
+## 4. Verification and Archive
+
+- [x] 4.1 Add or update unit tests for the new V2 audit checks.
+- [x] 4.2 Run focused evaluator and agent-chain tests.
+- [x] 4.3 Validate and archive the OpenSpec change.
+- [x] 4.4 Commit the completed phase 5 changes.