32 lines
878 B
Markdown
32 lines
878 B
Markdown
# Brief: expand-diagnosis-eval-fixtures
|
|
|
|
## Background
|
|
|
|
The diagnosis eval harness is implemented and archived, but the fixed baseline is incomplete because three of the five diagnosis cases still reference missing fixtures.
|
|
|
|
## Goals
|
|
|
|
1. Add representative trace fixtures for all remaining fixed diagnosis cases.
|
|
2. Save a reproducible baseline report in JSON and Markdown.
|
|
3. Document how to regenerate and interpret the baseline.
|
|
4. Keep evaluation offline and deterministic.
|
|
|
|
## Scope
|
|
|
|
- Redis timeout fixture
|
|
- Slow response fixture
|
|
- JVM memory risk fixture
|
|
- Baseline reports under `mvp/eval/reports`
|
|
- Focused tests for full fixture coverage and report generation
|
|
|
|
## Non-Goals
|
|
|
|
- No new diagnosis cases
|
|
- No production Agent runtime changes
|
|
- No LLM-as-judge
|
|
- No live infrastructure requirement
|
|
|
|
## Related OpenSpec
|
|
|
|
`openspec/changes/expand-diagnosis-eval-fixtures/`
|