zhuyongxin
de5a5b09d9
Archive pre-refactor interview notes and add current deep-dives on architecture evolution, issue-derived stories, and evidence gates.
199 lines
4.3 KiB
Markdown
199 lines
4.3 KiB
Markdown
# RAG VectorStore Live 验收说明
|
||
|
||
## 1. 目的
|
||
|
||
本文记录 RAG 检索重构的 live 验收结论。
|
||
|
||
这次重构的目标不只是接入 Spring AI 抽象，而是证明线上读路径能够：
|
||
|
||
- 优先使用 Spring AI `VectorStore` 做 Milvus 检索。
|
||
- 保留原 Milvus SDK 作为 fallback。
|
||
- 保持 `lookup_knowledge` 工具契约稳定。
|
||
- 保持基于 L2 distance 的相关性归一化兼容。
|
||
|
||
## 2. 当前检索形态
|
||
|
||
```text
|
||
lookup_knowledge / /api/search/similar
|
||
-> VectorSearchService.searchSimilarDocuments(...)
|
||
-> retrieval.vector-store.mode
|
||
-> auto
|
||
-> Spring AI VectorStore
|
||
-> VectorStore 失败时 fallback 到 Milvus SDK
|
||
-> spring-ai
|
||
-> 只走 Spring AI VectorStore
|
||
-> sdk
|
||
-> 只走 Milvus SDK
|
||
```
|
||
|
||
## 3. 已验证配置
|
||
|
||
live Milvus/Zilliz 数据库中存在 collection：
|
||
|
||
```text
|
||
biz
|
||
```
|
||
|
||
Spring AI VectorStore 配置与 SDK 使用的 collection 对齐：
|
||
|
||
```yaml
|
||
spring:
|
||
ai:
|
||
vectorstore:
|
||
type: milvus
|
||
milvus:
|
||
initialize-schema: false
|
||
database-name: ${milvus.database}
|
||
collection-name: biz
|
||
embedding-dimension: ${milvus.vector-dim}
|
||
metric-type: L2
|
||
id-field-name: id
|
||
content-field-name: content
|
||
metadata-field-name: metadata
|
||
embedding-field-name: vector
|
||
```
|
||
|
||
为什么重要：早期配置使用 `business_knowledge`，而真实 collection 是 `biz`。这个错配证明了 fallback 生效，但也说明修正前 VectorStore 不是成功主路径。
|
||
|
||
## 4. 验收命令
|
||
|
||
健康检查：
|
||
|
||
```powershell
|
||
Invoke-RestMethod `
|
||
-Uri "http://127.0.0.1:9900/milvus/health" `
|
||
-Method Get
|
||
```
|
||
|
||
期望：
|
||
|
||
```json
|
||
{
|
||
"collections": ["biz"],
|
||
"message": "ok"
|
||
}
|
||
```
|
||
|
||
直接检索：
|
||
|
||
```powershell
|
||
Invoke-RestMethod `
|
||
-Uri "http://127.0.0.1:9900/api/search/similar?query=ERR_TIMEOUT&topK=3" `
|
||
-Method Get
|
||
```
|
||
|
||
期望结果形态：
|
||
|
||
```json
|
||
{
|
||
"code": 200,
|
||
"message": "success",
|
||
"data": [
|
||
{
|
||
"content": "### ERR_TIMEOUT ...",
|
||
"score": 0.5662,
|
||
"rawScore": 0.4337,
|
||
"scoreLabel": "similarity",
|
||
"metadata": {
|
||
"distance": 0.5662,
|
||
"title": "ERR_TIMEOUT",
|
||
"category": "api"
|
||
}
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
## 5. 日志证明了什么
|
||
|
||
collection 修正前：
|
||
|
||
```text
|
||
Starting Spring AI VectorStore search
|
||
Spring AI VectorStore retrieval failed, falling back to Milvus SDK
|
||
Starting Milvus SDK search
|
||
```
|
||
|
||
collection 修正后：
|
||
|
||
```text
|
||
Starting Spring AI VectorStore search: query=ERR_TIMEOUT
|
||
Spring AI VectorStore search complete, candidates=3
|
||
```
|
||
|
||
这证明：
|
||
|
||
- `auto` 模式确实先尝试 VectorStore。
|
||
- VectorStore 失败时 fallback 可用。
|
||
- 配置对齐后，主路径是 Spring AI VectorStore，而不是 SDK fallback。
|
||
|
||
## 6. 分数语义
|
||
|
||
项目保留三个分数字段：
|
||
|
||
```text
|
||
rawScore -> 当前检索实现的原始分数
|
||
scoreLabel -> rawScore 的语义
|
||
score -> lookup relevance normalization 使用的兼容分数
|
||
```
|
||
|
||
SDK：
|
||
|
||
```text
|
||
rawScore = L2 distance
|
||
scoreLabel = l2_distance
|
||
score = L2 distance
|
||
```
|
||
|
||
Spring AI VectorStore：
|
||
|
||
```text
|
||
rawScore = Spring AI similarity score
|
||
scoreLabel = similarity
|
||
score = Milvus distance metadata when available
|
||
```
|
||
|
||
使用 `metadata.distance` 的原因：`LookupKnowledgeTool` 已经基于 L2 distance 做相关性归一化。Spring AI Milvus 主分数是 similarity，但 metadata 中仍有 Milvus distance。用 distance 保持旧逻辑稳定，同时通过 `rawScore` 暴露新语义。
|
||
|
||
## 7. 回归检查
|
||
|
||
目标测试：
|
||
|
||
```powershell
|
||
mvn -q "-Dtest=VectorSearchServiceTest,LookupKnowledgeToolTest" test
|
||
```
|
||
|
||
相关 spec：
|
||
|
||
```powershell
|
||
openspec.cmd validate rag-knowledge-retrieval --specs
|
||
openspec.cmd validate rag-retrieval-evaluation --specs
|
||
```
|
||
|
||
diff 检查：
|
||
|
||
```powershell
|
||
git diff --check
|
||
```
|
||
|
||
验收结论：
|
||
|
||
```text
|
||
目标测试通过。
|
||
相关 spec 通过。
|
||
diff-check 无错误。
|
||
```
|
||
|
||
## 8. 验收结论
|
||
|
||
VectorStore 读路径可以接受：
|
||
|
||
- Spring AI VectorStore 已集成，并在 `auto` 模式中优先使用。
|
||
- SDK fallback 保留且已被实际验证。
|
||
- live collection 配置与现有 Milvus collection 对齐。
|
||
- `lookup_knowledge` 对外契约保持稳定。
|
||
- 旧的 L2 relevance normalization 仍兼容。
|
||
|
||
写入和索引路径仍使用 Milvus SDK。这是有意的分阶段迁移，不是验收失败项。
|
||
|