415 lines
11 KiB
Markdown
415 lines
11 KiB
Markdown
# RAG 新架构
|
||
|
||
**更新日期**：2026-07-05
|
||
**状态**：当前主架构 + 后续演进边界
|
||
**关联计划**：`mvp/issues/rag-refactor-plan.md`
|
||
|
||
## 1. 架构目标
|
||
|
||
RAG 重构的目标不是把所有能力交给框架，也不是继续维护一套完全自研检索框架，而是形成：
|
||
|
||
```text
|
||
成熟框架能力 + 业务可观测编排
|
||
```
|
||
|
||
具体原则：
|
||
|
||
- 通用向量检索能力交给 Spring AI `VectorStore`。
|
||
- 项目保留 Agent Tool 入口、AIOps 业务 query 映射、证据打包、trace 记录。
|
||
- `lookup_knowledge` 继续是显式工具，不替换成隐式 Advisor。
|
||
- Spring AI 读取路径作为主路径，Milvus SDK 作为 fallback。
|
||
- 所有检索行为必须可评测、可回放、可解释。
|
||
|
||
## 2. 当前主链路
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Agent["Agent Executor"] --> Tool["lookup_knowledge(query)"]
|
||
|
||
Tool --> L0["KnowledgeIndexService.analyzeQuery"]
|
||
L0 --> Hint["L0 hint: domain / entities / matchedKeywords"]
|
||
Hint --> Filter["category filter candidate"]
|
||
|
||
Tool --> Search["VectorSearchService.searchSimilarDocuments"]
|
||
Filter --> Search
|
||
|
||
Search --> Mode{"retrieval.vector-store.mode"}
|
||
Mode -->|auto| SpringTry["try Spring AI VectorStore"]
|
||
SpringTry -->|success| Results["SearchResult list"]
|
||
SpringTry -->|failure| SdkFallback["Milvus SDK fallback"]
|
||
Mode -->|spring-ai| SpringOnly["Spring AI VectorStore only"]
|
||
Mode -->|sdk| SdkOnly["Milvus SDK only"]
|
||
|
||
SpringOnly --> Results
|
||
SdkFallback --> Results
|
||
SdkOnly --> Results
|
||
|
||
Results --> Normalize["relevance normalization"]
|
||
Normalize --> Dedup["session dedup: RetrievedDocTracker"]
|
||
Dedup --> Output["LookupResult"]
|
||
Output --> Record["tool_invocation record"]
|
||
Output --> Agent
|
||
```
|
||
|
||
```text
|
||
Agent Executor
|
||
-> lookup_knowledge(query)
|
||
-> KnowledgeIndexService.analyzeQuery
|
||
-> L0 domain/entity hint
|
||
-> matchedKeywords
|
||
-> category filter candidate
|
||
-> VectorSearchService.searchSimilarDocuments
|
||
-> mode=auto
|
||
-> Spring AI VectorStore
|
||
-> fallback: Milvus SDK
|
||
-> mode=spring-ai
|
||
-> Spring AI VectorStore only
|
||
-> mode=sdk
|
||
-> Milvus SDK only
|
||
-> result normalization
|
||
-> relevanceLevel
|
||
-> completenessHint
|
||
-> score/rawScore/scoreLabel
|
||
-> session dedup
|
||
-> RetrievedDocTracker
|
||
-> tool_invocation record
|
||
```
|
||
|
||
运行配置：
|
||
|
||
```properties
|
||
retrieval.vector-store.mode=auto
|
||
retrieval.normalization.max-l2-distance=2.0
|
||
retrieval.normalization.highly-relevant-threshold=0.75
|
||
retrieval.normalization.reference-threshold=0.5
|
||
```
|
||
|
||
## 3. 稳定边界
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
subgraph AgentBoundary["Agent boundary"]
|
||
Executor["Executor Agent"]
|
||
Tool["LookupKnowledgeTool"]
|
||
end
|
||
|
||
subgraph RetrievalBoundary["Retrieval boundary"]
|
||
Search["VectorSearchService"]
|
||
Spring["Spring AI VectorStore"]
|
||
SDK["Milvus SDK"]
|
||
end
|
||
|
||
subgraph ObservabilityBoundary["Observability boundary"]
|
||
Invocation["tool_invocation"]
|
||
Eval["RAG baseline / trace inspection"]
|
||
end
|
||
|
||
Executor --> Tool
|
||
Tool --> Search
|
||
Search --> Spring
|
||
Search --> SDK
|
||
Tool --> Invocation
|
||
Invocation --> Eval
|
||
```
|
||
|
||
### 3.1 Agent 边界
|
||
|
||
Agent 只知道自己可以调用 `lookup_knowledge`，不直接关心底层是 Spring AI VectorStore 还是 Milvus SDK。
|
||
|
||
```text
|
||
Executor -> LookupKnowledgeTool -> VectorSearchService
|
||
```
|
||
|
||
这个边界让 RAG 底层迁移不影响 Agent prompt、工具声明和 trace 数据结构。
|
||
|
||
### 3.2 检索边界
|
||
|
||
`VectorSearchService` 是当前检索门面：
|
||
|
||
- `auto`：优先 Spring AI VectorStore，失败后 fallback 到 SDK。
|
||
- `spring-ai`：只走 Spring AI VectorStore。
|
||
- `sdk`：只走原 Milvus SDK。
|
||
|
||
这样可以在不改 Agent 工具的情况下切换检索实现，并支持线上验证和回退。
|
||
|
||
### 3.3 可观测边界
|
||
|
||
无论底层检索路径如何变化，都必须写入 `tool_invocation`：
|
||
|
||
```text
|
||
sessionId
|
||
toolName
|
||
inputParams
|
||
outputPreview
|
||
retrievalLayer
|
||
l0MatchCount
|
||
l1MatchCount
|
||
retrievalDetails
|
||
relevanceLevel
|
||
dedupReason
|
||
duration
|
||
success
|
||
```
|
||
|
||
## 4. L0 的新职责
|
||
|
||
旧版 L0 容易承担过重职责，例如唯一匹配后直接跳过 L1。当前架构中 L0 被降级为 hint 层。
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Input["query / AIOps payload"] --> L0["L0 hint analysis"]
|
||
L0 --> Domain["domain detector"]
|
||
L0 --> Entity["entity extractor"]
|
||
L0 --> Keyword["matched keyword explanation"]
|
||
L0 --> Filter["metadata/category filter candidate"]
|
||
|
||
Domain --> Retrieval["L1 semantic retrieval"]
|
||
Entity --> Retrieval
|
||
Keyword --> Trace["hit reason in tool_invocation"]
|
||
Filter --> Retrieval
|
||
|
||
Retrieval --> Normalize["relevance normalization"]
|
||
Normalize --> Evidence["evidence returned to Agent"]
|
||
```
|
||
|
||
L0 负责：
|
||
|
||
- domain detector
|
||
- entity extractor
|
||
- matched keyword explanation
|
||
- metadata/category filter candidate
|
||
- trace 中的 hit reason
|
||
|
||
L0 不再默认负责：
|
||
|
||
```text
|
||
L0 unique hit -> 直接作为最终检索结果
|
||
```
|
||
|
||
当前职责是：
|
||
|
||
```text
|
||
query / AIOps payload
|
||
-> L0 matched keywords / domains / entities
|
||
-> category filter candidate
|
||
-> L1 semantic retrieval
|
||
-> relevance normalization
|
||
```
|
||
|
||
这样既保留精确关键词和领域 hint 的价值，也避免 L0 误召回直接污染最终证据。
|
||
|
||
## 5. L1 向量检索
|
||
|
||
L1 语义检索通过 `VectorSearchService` 调度。
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Search["VectorSearchService"] --> Request["SearchRequest: query / topK / threshold / filter"]
|
||
Request --> VectorStore["Spring AI VectorStore"]
|
||
VectorStore --> Docs["Document results"]
|
||
Docs --> Map["map to SearchResult"]
|
||
Map --> Score["score compatibility mapping"]
|
||
|
||
Search --> SDK["Milvus SDK fallback"]
|
||
SDK --> SdkRows["id / content / metadata / L2 distance"]
|
||
SdkRows --> Map
|
||
|
||
Score --> Output["id / content / metadata / score / rawScore / scoreLabel"]
|
||
```
|
||
|
||
### Spring AI VectorStore 路径
|
||
|
||
```text
|
||
SearchRequest
|
||
-> query
|
||
-> topK
|
||
-> similarityThresholdAll
|
||
-> optional filterExpression: category == '...'
|
||
-> VectorStore.similaritySearch
|
||
```
|
||
|
||
返回结果会映射为项目兼容结构：
|
||
|
||
```text
|
||
id
|
||
content
|
||
metadata
|
||
score
|
||
rawScore
|
||
scoreLabel
|
||
```
|
||
|
||
### Milvus SDK fallback
|
||
|
||
SDK 路径仍保留：
|
||
|
||
- 用于 `auto` 模式兜底。
|
||
- 用于与旧链路对比。
|
||
- 用于 VectorStore 配置或 collection schema 异常时保证 MVP 可运行。
|
||
|
||
## 6. 分数语义
|
||
|
||
旧 SDK 使用 L2 distance，Spring AI 返回 similarity。两者不能混用为同一个含义。
|
||
|
||
当前统一输出：
|
||
|
||
| 字段 | 含义 |
|
||
|---|---|
|
||
| `score` | 兼容旧逻辑的距离型分数，越小越近 |
|
||
| `rawScore` | 底层实现的原始分数 |
|
||
| `scoreLabel` | `l2_distance` 或 `similarity` |
|
||
|
||
SDK 路径：
|
||
|
||
```text
|
||
score = L2 distance
|
||
rawScore = L2 distance
|
||
scoreLabel = l2_distance
|
||
```
|
||
|
||
VectorStore 路径：
|
||
|
||
```text
|
||
rawScore = Spring AI similarity
|
||
scoreLabel = similarity
|
||
score = metadata.distance if available else compatible distance
|
||
```
|
||
|
||
## 7. 文档切片和 embedding 输入
|
||
|
||
当前保留 Markdown-aware chunking：
|
||
|
||
- 识别 Markdown 标题层级。
|
||
- 生成 `title`。
|
||
- 生成 `breadcrumb`。
|
||
- 保留 `chunkIndex`。
|
||
- 使用 token 估算和软/硬上限控制 chunk 大小。
|
||
- 尽量不打断列表和代码块。
|
||
|
||
embedding 输入中已经加强：
|
||
|
||
```text
|
||
title + breadcrumb + content
|
||
```
|
||
|
||
这样可以降低单个 chunk 脱离章节上下文后的召回损失。
|
||
|
||
## 8. AIOps query 增强
|
||
|
||
AIOps payload 中的业务字段不能完全交给通用检索框架隐式理解。
|
||
|
||
payload 模式会把以下字段拼成推荐知识库 query：
|
||
|
||
- `alertName`
|
||
- `service`
|
||
- `severity`
|
||
- `description`
|
||
- `timeRange`
|
||
- `userRequest`
|
||
|
||
Prompt 会明确要求 Agent 在需要知识库证据时，优先使用推荐 query 或保留 alertName/service 的更窄 query。
|
||
|
||
```text
|
||
AIOps payload
|
||
-> buildKnowledgeRetrievalQuery
|
||
-> Recommended lookup_knowledge query
|
||
-> lookup_knowledge
|
||
-> tool_invocation
|
||
```
|
||
|
||
## 9. Evidence 与去重
|
||
|
||
当前 evidence 输出仍以 `LookupResult` 和工具返回文本为主，已经具备：
|
||
|
||
- L0/L1 命中数量。
|
||
- 检索层记录。
|
||
- relevance level。
|
||
- completeness hint。
|
||
- session 级文档去重。
|
||
- domain 行动记忆。
|
||
- `tool_invocation` 明细记录。
|
||
|
||
后续更完整的 evidence block 目标：
|
||
|
||
```text
|
||
source
|
||
docId
|
||
chunkIndex
|
||
title
|
||
breadcrumb
|
||
score
|
||
rawScore
|
||
scoreLabel
|
||
hitReason
|
||
content
|
||
expandedFrom
|
||
```
|
||
|
||
这部分应作为下一阶段增强，而不是当前已完全完成能力。
|
||
|
||
## 10. 评测与验收
|
||
|
||
RAG 架构变更必须先过评测，再认为可合入主链路。
|
||
|
||
当前评测资产：
|
||
|
||
- `eval/rag-retrieval/cases/golden-cases.json`
|
||
- `eval/rag-retrieval/fixtures/`
|
||
- `eval/rag-retrieval/reports/baseline.json`
|
||
- `eval/rag-retrieval/reports/baseline.md`
|
||
- `scripts/eval_rag_retrieval.py`
|
||
- `scripts/eval_rag_live_acceptance.py`
|
||
|
||
评测层次：
|
||
|
||
| 层次 | 作用 |
|
||
|---|---|
|
||
| Offline baseline | 不依赖 MySQL、Redis、Milvus、LLM，用固定 fixtures 检查召回行为 |
|
||
| Live acceptance | 应用运行并重建索引后，调用 `/api/search/similar` 验证真实检索 |
|
||
| Trace inspection | 通过 `tool_invocation` 检查 Agent 是否真的使用了证据 |
|
||
|
||
## 11. 当前已完成
|
||
|
||
- `lookup_knowledge` 保持显式 Agent Tool。
|
||
- L0 降级为 domain/entity hint。
|
||
- L1 默认执行语义检索。
|
||
- `VectorSearchService` 支持 `auto`、`spring-ai`、`sdk` 三种模式。
|
||
- Spring AI VectorStore 成为读取主路径。
|
||
- Milvus SDK fallback 保留。
|
||
- 分数语义拆成 `score`、`rawScore`、`scoreLabel`。
|
||
- Markdown chunk 保留 `title` 和 `breadcrumb`。
|
||
- embedding 输入包含 `title`、`breadcrumb` 和 `content`。
|
||
- AIOps payload 生成推荐知识库 query。
|
||
- `tool_invocation` 记录 relevance level 和 dedup reason。
|
||
- RAG offline baseline 和 live acceptance 脚本已补齐。
|
||
|
||
## 12. 后续演进
|
||
|
||
近期优先：
|
||
|
||
1. 完整 evidence block 结构化输出。
|
||
2. 命中 chunk 的相邻 chunk / 同章节上下文扩展。
|
||
3. metadata taxonomy 清理，例如 `database` 与 `infrastructure` 的分类边界。
|
||
4. Query Transformer / MultiQuery 的可回退接入。
|
||
5. VectorStore 写入路径评估。
|
||
|
||
暂不优先：
|
||
|
||
- 把 `lookup_knowledge` 替换成隐式 Advisor。
|
||
- 完整自研 RRF 框架。
|
||
- 立即引入 Elasticsearch / OpenSearch。
|
||
- 立即引入 cross-encoder 或 LLM rerank。
|
||
|
||
## 13. 关键代码索引
|
||
|
||
| 能力 | 代码 |
|
||
|---|---|
|
||
| Agent 工具入口 | `src/main/java/com/superbiz/agent/tool/LookupKnowledgeTool.java` |
|
||
| L0 hint | `src/main/java/com/superbiz/agent/service/KnowledgeIndexService.java` |
|
||
| 向量检索门面 | `src/main/java/com/superbiz/agent/service/VectorSearchService.java` |
|
||
| 文档切片 | `src/main/java/com/superbiz/agent/service/DocumentChunkService.java` |
|
||
| 文档管理 | `src/main/java/com/superbiz/agent/service/DocumentManagementService.java` |
|
||
| 向量写入 | `src/main/java/com/superbiz/agent/service/VectorIndexService.java` |
|
||
| AIOps query 增强 | `src/main/java/com/superbiz/agent/service/AiOpsService.java` |
|
||
| 工具调用记录 | `src/main/java/com/superbiz/agent/service/ToolInvocationRecorder.java` |
|