SuperBizAgent-java/openspec/changes/archive/2026-07-04-rag-retrieval-baseline/tasks.md

1. Golden Cases

• 1.1 Create retrieval evaluation directory structure.
• 1.2 Add fixed golden retrieval cases covering Chat and AIOps retrieval scenarios.
• 1.3 Add matching offline retrieval fixtures for every golden case.

2. Evaluator

• 2.1 Implement an offline retrieval evaluator script.
• 2.2 Support hit-level classification and first expected document rank.
• 2.3 Support JSON and Markdown report output.

3. Baseline Report

• 3.1 Generate the baseline JSON report from the fixed cases and fixtures.
• 3.2 Generate the baseline Markdown report from the fixed cases and fixtures.

4. Documentation

• 4.1 Document the retrieval baseline purpose, file layout, and regeneration command.
• 4.2 Link the retrieval baseline from the RAG refactor issue or related MVP documentation.

5. Verification

• 5.1 Run the evaluator successfully against the fixed baseline cases.
• 5.2 Run OpenSpec status/validation for the change and confirm tasks are complete.