Expand diagnosis eval fixtures

2026-07-05 00:27:57 +08:00
parent ca5c61fabf
commit 4c7c53b024
21 changed files with 622 additions and 11 deletions
@@ -0,0 +1,31 @@
+# Brief: expand-diagnosis-eval-fixtures
+
+## Background
+
+The diagnosis eval harness is implemented and archived, but the fixed baseline is incomplete because three of the five diagnosis cases still reference missing fixtures.
+
+## Goals
+
+1. Add representative trace fixtures for all remaining fixed diagnosis cases.
+2. Save a reproducible baseline report in JSON and Markdown.
+3. Document how to regenerate and interpret the baseline.
+4. Keep evaluation offline and deterministic.
+
+## Scope
+
+- Redis timeout fixture
+- Slow response fixture
+- JVM memory risk fixture
+- Baseline reports under `mvp/eval/reports`
+- Focused tests for full fixture coverage and report generation
+
+## Non-Goals
+
+- No new diagnosis cases
+- No production Agent runtime changes
+- No LLM-as-judge
+- No live infrastructure requirement
+
+## Related OpenSpec
+
+`openspec/changes/expand-diagnosis-eval-fixtures/`