docs: add rag vectorstore interview notes

2026-07-05 11:13:50 +08:00
parent f2bae0382c
commit 9376448804
2 changed files with 366 additions and 0 deletions
@@ -0,0 +1,199 @@
+# RAG VectorStore Live Acceptance
+
+## Purpose
+
+This note records the live acceptance result for the RAG retrieval refactor.
+
+The goal of this refactor was not only to add a Spring AI abstraction, but to prove that the production retrieval path can:
+
+- Prefer Spring AI `VectorStore` for Milvus retrieval.
+- Preserve the existing Milvus SDK path as fallback.
+- Keep the `lookup_knowledge` tool contract stable.
+- Keep L2-distance based relevance normalization compatible.
+
+## Current Retrieval Shape
+
+```text
+lookup_knowledge / /api/search/similar
+  -> VectorSearchService.searchSimilarDocuments(...)
+      -> retrieval.vector-store.mode
+          -> auto
+              -> Spring AI VectorStore
+              -> fallback to Milvus SDK if VectorStore fails
+          -> spring-ai
+              -> Spring AI VectorStore only
+          -> sdk
+              -> Milvus SDK only
+```
+
+## Configuration Verified
+
+The live Milvus/Zilliz database contains the collection:
+
+```text
+biz
+```
+
+The Spring AI VectorStore configuration was aligned with the existing SDK collection:
+
+```yaml
+spring:
+  ai:
+    vectorstore:
+      type: milvus
+      milvus:
+        initialize-schema: false
+        database-name: ${milvus.database}
+        collection-name: biz
+        embedding-dimension: ${milvus.vector-dim}
+        metric-type: L2
+        id-field-name: id
+        content-field-name: content
+        metadata-field-name: metadata
+        embedding-field-name: vector
+```
+
+Why this matters: the earlier config used `business_knowledge`, but the SDK path and real collection use `biz`. That mismatch proved the fallback worked, but it also meant VectorStore was not the successful main path until the config was corrected.
+
+## Commands Used
+
+Health check:
+
+```powershell
+Invoke-RestMethod `
+  -Uri "http://127.0.0.1:9900/milvus/health" `
+  -Method Get
+```
+
+Observed result:
+
+```json
+{
+  "collections": ["biz"],
+  "message": "ok"
+}
+```
+
+Direct retrieval check:
+
+```powershell
+Invoke-RestMethod `
+  -Uri "http://127.0.0.1:9900/api/search/similar?query=ERR_TIMEOUT&topK=3" `
+  -Method Get
+```
+
+Observed result shape:
+
+```json
+{
+  "code": 200,
+  "message": "success",
+  "data": [
+    {
+      "id": "f7dff7c8-5665-3145-9f75-ef741528b914",
+      "content": "### ERR_TIMEOUT ...",
+      "score": 0.5662,
+      "rawScore": 0.4337,
+      "scoreLabel": "similarity",
+      "metadata": {
+        "distance": 0.5662,
+        "title": "ERR_TIMEOUT",
+        "category": "api"
+      }
+    }
+  ]
+}
+```
+
+## What The Logs Proved
+
+Before collection alignment:
+
+```text
+Starting Spring AI VectorStore search
+SearchRequest collectionName:business_knowledge failed
+Spring AI VectorStore retrieval failed, falling back to Milvus SDK
+Starting Milvus SDK search
+```
+
+After collection alignment:
+
+```text
+Starting Spring AI VectorStore search: query=ERR_TIMEOUT
+Spring AI VectorStore search complete, candidates=3
+```
+
+This proves:
+
+- `auto` mode really attempts VectorStore first.
+- The fallback is functional when VectorStore fails.
+- After config alignment, the main path is Spring AI VectorStore rather than SDK fallback.
+
+## Score Semantics
+
+The project keeps three score fields intentionally:
+
+```text
+rawScore    -> the raw score from the active retrieval implementation
+scoreLabel  -> the semantic meaning of rawScore
+score       -> compatibility score used by existing lookup relevance normalization
+```
+
+For SDK retrieval:
+
+```text
+rawScore = L2 distance
+scoreLabel = l2_distance
+score = L2 distance
+```
+
+For Spring AI VectorStore retrieval:
+
+```text
+rawScore = Spring AI similarity score
+scoreLabel = similarity
+score = Milvus distance metadata when available
+```
+
+Why use `metadata.distance` for `score`: `LookupKnowledgeTool` already normalizes relevance from L2 distance. Spring AI Milvus returns similarity as the document score, but also includes the Milvus distance in metadata. Using distance preserves the old relevance behavior while still exposing the new VectorStore score semantics through `rawScore` and `scoreLabel`.
+
+## Regression Checks
+
+Targeted tests:
+
+```powershell
+mvn -q "-Dtest=VectorSearchServiceTest,LookupKnowledgeToolTest" test
+```
+
+Spec validation:
+
+```powershell
+openspec.cmd validate rag-knowledge-retrieval --specs
+openspec.cmd validate rag-retrieval-evaluation --specs
+```
+
+Whitespace check:
+
+```powershell
+git diff --check
+```
+
+Observed result:
+
+```text
+All targeted tests passed.
+All related specs passed.
+No diff-check errors.
+```
+
+## Acceptance Conclusion
+
+The VectorStore refactor is accepted for the read path:
+
+- Spring AI VectorStore is integrated and selected in `auto` mode.
+- The SDK path remains available and was proven by fallback behavior.
+- The live collection configuration is aligned with the existing Milvus collection.
+- The `lookup_knowledge` public contract remains stable.
+- Existing L2-based relevance normalization remains compatible.
+
+The write/indexing path still uses the Milvus SDK. That is an intentional staged migration decision, not a failed acceptance item.