Expand diagnosis eval fixtures

This commit is contained in:
aruo
2026-07-05 00:27:57 +08:00
parent ca5c61fabf
commit 4c7c53b024
21 changed files with 622 additions and 11 deletions
+12
View File
@@ -7,6 +7,7 @@ This folder contains the first fixed-case evaluation set for the MVP diagnosis A
- Case definitions: `cases/diagnosis-cases.json`
- Offline trace fixtures: `fixtures/*.json`
- Field definitions: `schema.md`
- Baseline reports: `reports/baseline-report.json` and `reports/baseline-report.md`
- Evaluator implementation: `DiagnosisTraceEvaluator`
- Report writer: `DiagnosisEvalReportWriter`
@@ -22,6 +23,17 @@ Run the focused evaluator test:
mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test
```
The committed baseline report represents the current fixed fixture set:
```text
5 fixed cases
5 passing fixture evaluations
2 PASS verdicts
3 LOW_CONFID verdicts
```
When fixtures or evaluator rules change, regenerate the report from the same case file and fixture directory, then update both JSON and Markdown outputs together.
## Interview Story
The harness gives the MVP a repeatable baseline: