# Design: rag-bm25-hybrid-drop-sdk

## Decision: single backend

```text
KnowledgeSearchPort
  -> MilvusHybridKnowledgeStore (MilvusClientV2 only)
Write
  -> VectorIndexService -> MilvusHybridKnowledgeStore
```

No `retrieval.vector-store.mode=sdk|spring|auto` for knowledge lookup.

## Schema (`milvus.collection`, default `biz_hybrid`)

| field | type | notes |
|---|---|---|
| id | VarChar PK | chunk id |
| content | VarChar | original chunk body returned to agent |
| search_text | VarChar + analyzer | BM25 input (title/path/content) |
| sparse_vector | SparseFloatVector | BM25 function output |
| vector | FloatVector | dense embedding |
| metadata | JSON | docId, chunkIndex, category, kb_scope, ... |

Function: `BM25(search_text -> sparse_vector)`

Indexes:

- `vector`: IVF_FLAT / L2 (or COSINE if configured later)
- `sparse_vector`: SPARSE_INVERTED_INDEX / BM25

## Hybrid query

```text
AnnSearchReq(vector, FloatVec(queryEmbedding), topK, filter?)
AnnSearchReq(sparse_vector, EmbeddedText(query), topK, filter?)
HybridSearchReq + RRFRanker(k)
```

Dense-compatible score for thresholds: if entity lacks dense distance, map fused score conservatively or re-use dense-only probe. Preferred: keep dense distance when available from a parallel dense search metadata; for hybrid hits use inverse-rank placeholder only in metadata and set score from optional dense sub-hit map.

Implementation approach for threshold stability:

1. Run hybridSearch for ordering/identity
2. Build map evidenceKey -> dense L2 from a concurrent dense search (same filter/topK)
3. Attach dense score onto fused hits when present; else maxL2 (low similarity) so weak BM25-only hits don't fake PRECISE

## Migration

- New collection name avoids mutating legacy `biz`
- Operators re-run knowledge init / document reindex
- Document in acceptance

## Drop SDK search

- Delete SDK search methods usage from knowledge path
- `MilvusServiceClient` bean may remain temporarily only if other non-search utilities need it; prefer migrate write/delete fully to V2 and stop creating V1 client if unused