Files

T

zhuyongxin f035538531 feat(rag): dense+BM25 hybrid on MilvusClientV2, drop SDK path

Replace legacy MilvusServiceClient knowledge search/write with a single
MilvusClientV2 hybrid store (BM25 function + dense ANN + RRFRanker).
Use collection biz_hybrid and require knowledge reindex.

2026-07-27 18:49:20 +08:00

1.4 KiB

Raw Blame History

Change: Real BM25 hybrid search and drop legacy SDK path

Why

Delivery 2 shipped application-layer multi-path + sparse-lite lexical ranking. Product requirement is true dense + BM25 hybrid retrieval. Legacy MilvusServiceClient search path will be abandoned in this change; keep a single knowledge vector backend based on Milvus Java SDK v2 (MilvusClientV2) with BM25 function + hybridSearch + RRF.

What Changes

New hybrid collection schema: dense float vector + analyzed text + sparse BM25 output
Write path inserts text for BM25 auto-sparsification and dense embedding
Search path: single backend
- dense mode: dense ANN only
- hybrid mode: dense ANN + BM25 sparse ANN fused by RRFRanker
Remove/disable legacy SDK search / searchSimilarDocumentsWithSdk knowledge path and mode routing (sdk|spring|auto)
KnowledgeSearchPort adapter calls the single backend
Config: collection name, search mode, rrf-k, path weights optional
Tests for store mapping/fusion wiring; no dependency on live Zilliz in unit tests

Non-goals

Automatic full production reindex job UI
Cross-encoder model rerank
Keeping dual Spring-AI + SDK knowledge search modes

Impact

Breaking for existing biz collection: requires new hybrid collection + reindex
Agent tool contract unchanged
Interface: L2 internal retrieval backend

Depends on

Delivery 1 chunk identity
Delivery 2 SearchPort (will replace sparse-lite hybrid implementation)

1.4 KiB Raw Blame History