Diagnosis Eval Harness

This folder contains the first fixed-case evaluation set for the MVP diagnosis Agent.

Scope

Case definitions: cases/diagnosis-cases.json
Offline trace fixtures: fixtures/*.json
Field definitions: schema.md
Evaluator implementation: DiagnosisTraceEvaluator
Report writer: DiagnosisEvalReportWriter

Current Mode

The first version evaluates saved trace fixtures. It does not start the application and does not require MySQL, Redis, Milvus, or a real LLM.

Verification

Run the focused evaluator test:

mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test

Interview Story

The harness gives the MVP a repeatable baseline:

fixed diagnosis case
-> saved or runtime trace
-> rule-based trace validation
-> JSON / Markdown report
-> regression signal for prompts, tools, retrieval, and verifier behavior