feat(rag): hybrid multi-path search with RRF fusion

Add configurable hybrid mode on KnowledgeSearchPort that fuses dense unfiltered, dense filtered, and lexical ranks via RRF while preserving dense-compatible threshold scores. Archives Delivery 2 OpenSpec change.
2026-07-27 18:33:31 +08:00
parent ac1f831903
commit 376ad0c241
19 changed files with 661 additions and 8 deletions
@@ -0,0 +1,48 @@
+# Change: RAG hybrid search with RRF
+
+## Why
+
+Delivery 1 fixed chunk identity/dedup. Delivery 2 enables multi-path retrieval fused by RRF so category filters no longer whole-replace good results and lexical signals can complement dense ranks.
+
+True Milvus sparse/BM25 schema migration remains staged: this change ships a **working hybrid mode on KnowledgeSearchPort** using multi-path dense (+ lexical rank path) and RRF, without thickening the legacy SDK API surface.
+
+## What Changes
+
+- `retrieval.search.mode=dense|hybrid` (default dense)
+- Hybrid search on `KnowledgeSearchPort`:
+  - dense unfiltered path
+  - dense filtered path when category present
+  - lexical rank path over the candidate union (sparse-lite until BM25 schema lands)
+  - RRF / optional weighted RRF fusion by evidenceKey
+- Preserve dense L2 scores for quality thresholds; RRF only orders
+- Lookup tool uses search mode; filter low-quality retry remains as safety net
+- Config for rrf-k and path weights
+- Tests for RRF fusion ordering and hybrid adapter behavior
+
+## Non-goals
+
+- Full Milvus BM25/sparse collection rebuild (documented follow-up)
+- Deleting legacy SDK mode entirely
+- Cross-encoder model rerank
+- Changing Agent tool schema name/input
+
+## Capabilities
+
+### New Capabilities
+
+- `rag-hybrid-search`: hybrid mode, multi-path recall, RRF fusion via search port
+
+### Modified Capabilities
+
+- `rag-chunk-evidence-identity`: hybrid hits must keep chunk identity
+- `rag-knowledge-retrieval`: filtered/unfiltered cooperation via fusion rather than only serial replace
+
+## Impact
+
+- Code: retrieval package, lookup wiring/config, tests
+- Runtime default remains dense unless hybrid enabled
+- Interface: L2 internal (search port behavior)
+
+## Depends on
+
+- Archived Delivery 1: `rag-chunk-evidence-identity-dedup`