28 lines
1.3 KiB
Markdown
28 lines
1.3 KiB
Markdown
## Why
|
|
|
|
The RAG refactor needs a repeatable baseline before changing L0, metadata filtering, post-processing, or Spring AI retriever integration. Without fixed retrieval cases and measurable output, later changes can look cleaner architecturally while silently degrading recall or evidence quality.
|
|
|
|
## What Changes
|
|
|
|
- Add a retrieval evaluation baseline for RAG queries, separate from full diagnosis evaluation.
|
|
- Define golden retrieval cases covering Chat-style knowledge lookup and AIOps-style alert diagnosis retrieval.
|
|
- Add a lightweight offline evaluator that compares retrieved candidates against expected documents, breadcrumbs, and evidence keywords.
|
|
- Preserve baseline JSON and Markdown reports so future changes can compare retrieval behavior.
|
|
- No production retrieval behavior changes in this change.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
|
|
- `rag-retrieval-evaluation`: Defines fixed retrieval golden cases, deterministic retrieval evaluation, and baseline report preservation.
|
|
|
|
### Modified Capabilities
|
|
|
|
- None.
|
|
|
|
## Impact
|
|
|
|
- Adds retrieval evaluation fixtures, documentation, and scripts.
|
|
- May read existing retrieval/tool trace output or saved fixtures, but does not require live LLM calls.
|
|
- Does not change the `lookup_knowledge` runtime behavior, Milvus schema, document upload API, or Agent flow.
|