SuperBizAgent-java/interview/rag-vectorstore-live-acceptance.md

# RAG VectorStore Live Acceptance

## Purpose

This note records the live acceptance result for the RAG retrieval refactor.

The goal of this refactor was not only to add a Spring AI abstraction, but to prove that the production retrieval path can:

- Prefer Spring AI `VectorStore` for Milvus retrieval.
- Preserve the existing Milvus SDK path as fallback.
- Keep the `lookup_knowledge` tool contract stable.
- Keep L2-distance based relevance normalization compatible.

## Current Retrieval Shape

```text
lookup_knowledge / /api/search/similar
  -> VectorSearchService.searchSimilarDocuments(...)
      -> retrieval.vector-store.mode
          -> auto
              -> Spring AI VectorStore
              -> fallback to Milvus SDK if VectorStore fails
          -> spring-ai
              -> Spring AI VectorStore only
          -> sdk
              -> Milvus SDK only
```

## Configuration Verified

The live Milvus/Zilliz database contains the collection:

```text
biz
```

The Spring AI VectorStore configuration was aligned with the existing SDK collection:

```yaml
spring:
  ai:
    vectorstore:
      type: milvus
      milvus:
        initialize-schema: false
        database-name: ${milvus.database}
        collection-name: biz
        embedding-dimension: ${milvus.vector-dim}
        metric-type: L2
        id-field-name: id
        content-field-name: content
        metadata-field-name: metadata
        embedding-field-name: vector
```

Why this matters: the earlier config used `business_knowledge`, but the SDK path and real collection use `biz`. That mismatch proved the fallback worked, but it also meant VectorStore was not the successful main path until the config was corrected.

## Commands Used

Health check:

```powershell
Invoke-RestMethod `
  -Uri "http://127.0.0.1:9900/milvus/health" `
  -Method Get
```

Observed result:

```json
{
  "collections": ["biz"],
  "message": "ok"
}
```

Direct retrieval check:

```powershell
Invoke-RestMethod `
  -Uri "http://127.0.0.1:9900/api/search/similar?query=ERR_TIMEOUT&topK=3" `
  -Method Get
```

Observed result shape:

```json
{
  "code": 200,
  "message": "success",
  "data": [
    {
      "id": "f7dff7c8-5665-3145-9f75-ef741528b914",
      "content": "### ERR_TIMEOUT ...",
      "score": 0.5662,
      "rawScore": 0.4337,
      "scoreLabel": "similarity",
      "metadata": {
        "distance": 0.5662,
        "title": "ERR_TIMEOUT",
        "category": "api"
      }
    }
  ]
}
```

## What The Logs Proved

Before collection alignment:

```text
Starting Spring AI VectorStore search
SearchRequest collectionName:business_knowledge failed
Spring AI VectorStore retrieval failed, falling back to Milvus SDK
Starting Milvus SDK search
```

After collection alignment:

```text
Starting Spring AI VectorStore search: query=ERR_TIMEOUT
Spring AI VectorStore search complete, candidates=3
```

This proves:

- `auto` mode really attempts VectorStore first.
- The fallback is functional when VectorStore fails.
- After config alignment, the main path is Spring AI VectorStore rather than SDK fallback.

## Score Semantics

The project keeps three score fields intentionally:

```text
rawScore    -> the raw score from the active retrieval implementation
scoreLabel  -> the semantic meaning of rawScore
score       -> compatibility score used by existing lookup relevance normalization
```

For SDK retrieval:

```text
rawScore = L2 distance
scoreLabel = l2_distance
score = L2 distance
```

For Spring AI VectorStore retrieval:

```text
rawScore = Spring AI similarity score
scoreLabel = similarity
score = Milvus distance metadata when available
```

Why use `metadata.distance` for `score`: `LookupKnowledgeTool` already normalizes relevance from L2 distance. Spring AI Milvus returns similarity as the document score, but also includes the Milvus distance in metadata. Using distance preserves the old relevance behavior while still exposing the new VectorStore score semantics through `rawScore` and `scoreLabel`.

## Regression Checks

Targeted tests:

```powershell
mvn -q "-Dtest=VectorSearchServiceTest,LookupKnowledgeToolTest" test
```

Spec validation:

```powershell
openspec.cmd validate rag-knowledge-retrieval --specs
openspec.cmd validate rag-retrieval-evaluation --specs
```

Whitespace check:

```powershell
git diff --check
```

Observed result:

```text
All targeted tests passed.
All related specs passed.
No diff-check errors.
```

## Acceptance Conclusion

The VectorStore refactor is accepted for the read path:

- Spring AI VectorStore is integrated and selected in `auto` mode.
- The SDK path remains available and was proven by fallback behavior.
- The live collection configuration is aligned with the existing Milvus collection.
- The `lookup_knowledge` public contract remains stable.
- Existing L2-based relevance normalization remains compatible.

The write/indexing path still uses the Milvus SDK. That is an intentional staged migration decision, not a failed acceptance item.