Add diagnosis eval baseline diff

2026-07-05 00:59:53 +08:00
parent 4c7c53b024
commit 69deb15330
22 changed files with 1069 additions and 0 deletions
@@ -8,6 +8,7 @@ This folder contains the first fixed-case evaluation set for the MVP diagnosis A
 - Offline trace fixtures: `fixtures/*.json`
 - Field definitions: `schema.md`
 - Baseline reports: `reports/baseline-report.json` and `reports/baseline-report.md`
+- Baseline diff sample: `reports/baseline-diff-sample.json` and `reports/baseline-diff-sample.md`
 - Evaluator implementation: `DiagnosisTraceEvaluator`
 - Report writer: `DiagnosisEvalReportWriter`

@@ -45,3 +46,16 @@ fixed diagnosis case
 -> JSON / Markdown report
 -> regression signal for prompts, tools, retrieval, and verifier behavior
 ```
+
+## Baseline Diff
+
+Baseline diff compares a current report against `reports/baseline-report.json`.
+
+```text
+baseline report
+current report
+-> deterministic diff
+-> regressions, improvements, and changed signals
+```
+
+Use it to answer: did a prompt, tool, retrieval, or verifier change make the Agent worse than the fixed baseline?