1.0 KiB
1.0 KiB
Acceptance: diagnosis-eval-harness
Classification
standard-light
Task Status
| Task | Status | Notes |
|---|---|---|
| Issue and OpenSpec setup | Done | ISS-006 and initial OpenSpec artifacts were created. |
| Implementation | Done | Added fixed cases, fixture-mode trace evaluation, aggregate metrics, and JSON / Markdown report writer. |
| Verification | Done | Targeted evaluator tests, compile verification, and OpenSpec validation passed. |
Current State
- First implementation uses fixture-mode evaluation.
- Live trace API polling remains a follow-up option.
Verification
Script Verification
- Command:
mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test - Result: passed
- Notes: Covers fixed case loading, fixture evaluation, missing fixture reporting, reject degraded-output validation, and report writing.
Static Verification
- Command:
mvn -q -DskipTests compile - Result: passed
OpenSpec Verification
- Command:
openspec validate diagnosis-eval-harness --strict - Result: passed