## ADDED Requirements ### Requirement: Evaluation harness SHALL compare reports against a baseline The system SHALL compare a current diagnosis evaluation report against a saved baseline report using deterministic rules. #### Scenario: Aggregate regression detection - **WHEN** the current report has a lower pass rate than the baseline report - **THEN** the diff SHALL record a regression with the old value, new value, and delta #### Scenario: Cost signal detection - **WHEN** average tool-call count or average duration changes between reports - **THEN** the diff SHALL record the baseline value, current value, and delta #### Scenario: Verdict distribution comparison - **WHEN** verdict counts differ between reports - **THEN** the diff SHALL record the verdict distribution changes ### Requirement: Evaluation harness SHALL compare case-level report results The system SHALL compare case results by case id and report actionable per-case changes. #### Scenario: Case pass/fail regression - **WHEN** a case changes from passing in the baseline to failing in the current report - **THEN** the diff SHALL record a regression for that case #### Scenario: Evidence coverage regression - **WHEN** a required evidence tool changes from covered to uncovered for a case - **THEN** the diff SHALL record a regression naming the case and tool #### Scenario: Missing case detection - **WHEN** a baseline case is absent from the current report - **THEN** the diff SHALL record a regression for the missing case #### Scenario: New case detection - **WHEN** a current report contains a case absent from the baseline - **THEN** the diff SHALL record the case as a non-regression change ### Requirement: Evaluation harness SHALL report baseline diff results The system SHALL expose baseline diff output in structured JSON and reviewable Markdown formats. #### Scenario: JSON diff output - **WHEN** a baseline diff is written as JSON - **THEN** it SHALL include aggregate summary fields and detailed diff items #### Scenario: Markdown diff output - **WHEN** a baseline diff is written as Markdown - **THEN** it SHALL include a readable summary and a table of diff items