29 lines
926 B
Markdown
29 lines
926 B
Markdown
# Diagnosis Eval Baseline Diff Decisions
|
|
|
|
## Clarify
|
|
|
|
- Entry summary: add report diffing on top of the completed diagnosis eval baseline.
|
|
- Slug: `diagnosis-eval-baseline-diff`
|
|
- Devflow scale: standard-light
|
|
|
|
## Context
|
|
|
|
- `diagnosis-eval-harness` created deterministic fixture evaluation.
|
|
- `expand-diagnosis-eval-fixtures` created a complete saved baseline.
|
|
- This change compares new reports against that baseline.
|
|
|
|
## Key Decisions
|
|
|
|
- Decision: Diff report DTOs instead of raw traces.
|
|
- Reason: the report is the stable contract for regression review.
|
|
|
|
- Decision: Use deterministic code rules instead of LLM-as-judge.
|
|
- Reason: baseline regression checks should be repeatable and explainable.
|
|
|
|
- Decision: Output both JSON and Markdown.
|
|
- Reason: JSON supports automation; Markdown is useful in reviews and interviews.
|
|
|
|
## Open Questions
|
|
|
|
- Whether a future change should expose this through a CLI or Maven goal.
|