24 lines
1013 B
Markdown
24 lines
1013 B
Markdown
## 1. Eval Contract
|
|
|
|
- [x] 1.1 Extend `DiagnosisEvalCase` with optional V2 audit expectations.
|
|
- [x] 1.2 Extend `DiagnosisEvalResult` and report output with observed audit signals.
|
|
- [x] 1.3 Add deterministic evaluator checks for Gatekeeper, claim checks, Composer output, and raw JSON leakage.
|
|
|
|
## 2. Fixtures and Baseline
|
|
|
|
- [x] 2.1 Add fixed eval cases for Gatekeeper failure, unsupported claim filtering, and Composer fallback leakage prevention.
|
|
- [x] 2.2 Add matching trace fixtures for each new case.
|
|
- [x] 2.3 Regenerate baseline JSON and Markdown reports.
|
|
|
|
## 3. Documentation
|
|
|
|
- [x] 3.1 Update eval README with stage 5 scope and verification commands.
|
|
- [x] 3.2 Rewrite eval schema documentation so V2 audit fields and case expectations are readable.
|
|
|
|
## 4. Verification and Archive
|
|
|
|
- [x] 4.1 Add or update unit tests for the new V2 audit checks.
|
|
- [x] 4.2 Run focused evaluator and agent-chain tests.
|
|
- [x] 4.3 Validate and archive the OpenSpec change.
|
|
- [x] 4.4 Commit the completed phase 5 changes.
|