1.3 KiB
1.3 KiB
Why
The RAG refactor needs a repeatable baseline before changing L0, metadata filtering, post-processing, or Spring AI retriever integration. Without fixed retrieval cases and measurable output, later changes can look cleaner architecturally while silently degrading recall or evidence quality.
What Changes
- Add a retrieval evaluation baseline for RAG queries, separate from full diagnosis evaluation.
- Define golden retrieval cases covering Chat-style knowledge lookup and AIOps-style alert diagnosis retrieval.
- Add a lightweight offline evaluator that compares retrieved candidates against expected documents, breadcrumbs, and evidence keywords.
- Preserve baseline JSON and Markdown reports so future changes can compare retrieval behavior.
- No production retrieval behavior changes in this change.
Capabilities
New Capabilities
rag-retrieval-evaluation: Defines fixed retrieval golden cases, deterministic retrieval evaluation, and baseline report preservation.
Modified Capabilities
- None.
Impact
- Adds retrieval evaluation fixtures, documentation, and scripts.
- May read existing retrieval/tool trace output or saved fixtures, but does not require live LLM calls.
- Does not change the
lookup_knowledgeruntime behavior, Milvus schema, document upload API, or Agent flow.