zhuyongxin
376ad0c241
Add configurable hybrid mode on KnowledgeSearchPort that fuses dense unfiltered, dense filtered, and lexical ranks via RRF while preserving dense-compatible threshold scores. Archives Delivery 2 OpenSpec change.
1.8 KiB
1.8 KiB
Change: RAG hybrid search with RRF
Why
Delivery 1 fixed chunk identity/dedup. Delivery 2 enables multi-path retrieval fused by RRF so category filters no longer whole-replace good results and lexical signals can complement dense ranks.
True Milvus sparse/BM25 schema migration remains staged: this change ships a working hybrid mode on KnowledgeSearchPort using multi-path dense (+ lexical rank path) and RRF, without thickening the legacy SDK API surface.
What Changes
retrieval.search.mode=dense|hybrid(default dense)- Hybrid search on
KnowledgeSearchPort:- dense unfiltered path
- dense filtered path when category present
- lexical rank path over the candidate union (sparse-lite until BM25 schema lands)
- RRF / optional weighted RRF fusion by evidenceKey
- Preserve dense L2 scores for quality thresholds; RRF only orders
- Lookup tool uses search mode; filter low-quality retry remains as safety net
- Config for rrf-k and path weights
- Tests for RRF fusion ordering and hybrid adapter behavior
Non-goals
- Full Milvus BM25/sparse collection rebuild (documented follow-up)
- Deleting legacy SDK mode entirely
- Cross-encoder model rerank
- Changing Agent tool schema name/input
Capabilities
New Capabilities
rag-hybrid-search: hybrid mode, multi-path recall, RRF fusion via search port
Modified Capabilities
rag-chunk-evidence-identity: hybrid hits must keep chunk identityrag-knowledge-retrieval: filtered/unfiltered cooperation via fusion rather than only serial replace
Impact
- Code: retrieval package, lookup wiring/config, tests
- Runtime default remains dense unless hybrid enabled
- Interface: L2 internal (search port behavior)
Depends on
- Archived Delivery 1:
rag-chunk-evidence-identity-dedup