37 lines
1.0 KiB
Markdown
37 lines
1.0 KiB
Markdown
# Acceptance: diagnosis-eval-harness
|
|
|
|
## Classification
|
|
|
|
standard-light
|
|
|
|
## Task Status
|
|
|
|
| Task | Status | Notes |
|
|
| --- | --- | --- |
|
|
| Issue and OpenSpec setup | Done | `ISS-006` and initial OpenSpec artifacts were created. |
|
|
| Implementation | Done | Added fixed cases, fixture-mode trace evaluation, aggregate metrics, and JSON / Markdown report writer. |
|
|
| Verification | Done | Targeted evaluator tests, compile verification, and OpenSpec validation passed. |
|
|
|
|
## Current State
|
|
|
|
- First implementation uses fixture-mode evaluation.
|
|
- Live trace API polling remains a follow-up option.
|
|
|
|
## Verification
|
|
|
|
### Script Verification
|
|
|
|
- Command: `mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test`
|
|
- Result: passed
|
|
- Notes: Covers fixed case loading, fixture evaluation, missing fixture reporting, reject degraded-output validation, and report writing.
|
|
|
|
### Static Verification
|
|
|
|
- Command: `mvn -q -DskipTests compile`
|
|
- Result: passed
|
|
|
|
### OpenSpec Verification
|
|
|
|
- Command: `openspec validate diagnosis-eval-harness --strict`
|
|
- Result: passed
|