36 lines
887 B
Markdown
36 lines
887 B
Markdown
# Diagnosis Eval Harness
|
|
|
|
This folder contains the first fixed-case evaluation set for the MVP diagnosis Agent.
|
|
|
|
## Scope
|
|
|
|
- Case definitions: `cases/diagnosis-cases.json`
|
|
- Offline trace fixtures: `fixtures/*.json`
|
|
- Field definitions: `schema.md`
|
|
- Evaluator implementation: `DiagnosisTraceEvaluator`
|
|
- Report writer: `DiagnosisEvalReportWriter`
|
|
|
|
## Current Mode
|
|
|
|
The first version evaluates saved trace fixtures. It does not start the application and does not require MySQL, Redis, Milvus, or a real LLM.
|
|
|
|
## Verification
|
|
|
|
Run the focused evaluator test:
|
|
|
|
```powershell
|
|
mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test
|
|
```
|
|
|
|
## Interview Story
|
|
|
|
The harness gives the MVP a repeatable baseline:
|
|
|
|
```text
|
|
fixed diagnosis case
|
|
-> saved or runtime trace
|
|
-> rule-based trace validation
|
|
-> JSON / Markdown report
|
|
-> regression signal for prompts, tools, retrieval, and verifier behavior
|
|
```
|