29 lines
1.0 KiB
Markdown
29 lines
1.0 KiB
Markdown
# Expand Diagnosis Eval Fixtures Decisions
|
|
|
|
## Clarify
|
|
|
|
- Entry summary: complete the fixed diagnosis eval baseline after the harness is in place.
|
|
- Slug: `expand-diagnosis-eval-fixtures`
|
|
- Devflow scale: standard-light
|
|
|
|
## Context
|
|
|
|
- `diagnosis-eval-harness` created the evaluator, case file, fixture mode, and report writer.
|
|
- The first baseline still has missing fixtures by design.
|
|
- This follow-up turns that partial baseline into a full fixed-case baseline.
|
|
|
|
## Key Decisions
|
|
|
|
- Decision: Keep this change data-focused.
|
|
- Reason: the evaluator rules already landed; this change should not blur fixture expansion with harness behavior changes.
|
|
|
|
- Decision: Save baseline reports in the repository.
|
|
- Reason: interview review and future diffs are easier when the expected baseline is visible.
|
|
|
|
- Decision: Use deterministic fixture traces instead of live trace generation.
|
|
- Reason: this baseline should run without infrastructure or external model calls.
|
|
|
|
## Open Questions
|
|
|
|
- Whether a future change should add a CLI or Maven goal for report regeneration.
|