feat(rag): hybrid multi-path search with RRF fusion

Add configurable hybrid mode on KnowledgeSearchPort that fuses dense unfiltered, dense filtered, and lexical ranks via RRF while preserving dense-compatible threshold scores. Archives Delivery 2 OpenSpec change.
2026-07-27 18:33:31 +08:00
parent ac1f831903
commit 376ad0c241
19 changed files with 661 additions and 8 deletions
@@ -11,7 +11,8 @@

 | 日期 | slug | 说明 | 领域 | 关键词 | 关联 OpenSpec | 状态 |
 |---|---|---|---|---|---|---|
-| 2026-07-27 | rag-chunk-evidence-identity-dedup | chunk 级证据身份、去重、retrieve-k/return-n 与 SearchPort 地基，为 hybrid 铺路。 | RAG/证据身份/去重 | evidenceKey, maxChunksPerDocument, retrieve-k, return-n, KnowledgeSearchPort, document_id chunk-scoped | openspec/changes/rag-chunk-evidence-identity-dedup | accepted-unarchived |
+| 2026-07-27 | rag-chunk-evidence-identity-dedup | chunk 级证据身份、去重、retrieve-k/return-n 与 SearchPort 地基，为 hybrid 铺路。 | RAG/证据身份/去重 | evidenceKey, maxChunksPerDocument, retrieve-k, return-n, KnowledgeSearchPort, document_id chunk-scoped | openspec/changes/archive/2026-07-27-rag-chunk-evidence-identity-dedup | archived |
+| 2026-07-27 | rag-hybrid-search-rrf | Delivery 2：可配置 hybrid 检索与 RRF 多路融合（不绑旧 SDK）。 | RAG/hybrid/RRF | hybrid mode, RRF, KnowledgeSearchPort, filtered+unfiltered fusion, sparse-lite lexical | openspec/changes/archive/2026-07-27-rag-hybrid-search-rrf | archived |
 | 2026-07-21 | single-react-tool-invocation-store | 建立统一 ToolBoundary 与 Redis canonical invocation store，集中生命周期、证据状态、TTL、容量和 Run 所有权。 | Harness/Tool boundary/Canonical store | ISS-014, ToolBoundary, canonical invocation, PROJECTING, READY, ERROR, TTL, RESULT_TOO_LARGE | openspec/changes/archive/2026-07-21-single-react-tool-invocation-store | archived |
 | 2026-07-21 | single-react-harness-run-context | 建立显式 RunContext、Harness Core、预算、取消、类型化重试和 Tool Store 基础。 | Harness/Run lifecycle/Budget | ISS-014, RunContext, deadline, cancellation, budget, retry, ToolCallKey | openspec/changes/archive/2026-07-21-single-react-harness-run-context | archived |
 | 2026-07-21 | single-react-aci-tool-contracts | 冻结 RAG、日志和 MySQL evidence Tool 的 Agent-facing ACI Schema、状态、框架调用引用和描述边界。 | Harness/Agent Tool contract | ISS-014, ACI, tool_call_id, evidence_status, RAG, query_logs, query_mysql, MOCK | openspec/changes/archive/2026-07-21-single-react-aci-tool-contracts | archived |
@@ -0,0 +1,23 @@
+# Acceptance: rag-hybrid-search-rrf
+
+## Result
+
+Hybrid mode implemented on KnowledgeSearchPort:
+
+- dense unfiltered + dense filtered + lexical rank over union
+- RRF fusion by evidenceKey
+- dense-compatible score preserved for thresholds
+- default mode remains dense
+
+## Verification
+
+```text
+mvn -q "-Dtest=RrfFusionTest,VectorKnowledgeSearchAdapterHybridTest,LookupKnowledgeToolTest" test
+```
+
+Pass.
+
+## Residual
+
+- True Milvus BM25/sparse schema + reindex still follow-up
+- Lexical path only ranks dense-recalled candidates (does not expand pure-term misses outside dense topK)
@@ -0,0 +1,3 @@
+# Brief: rag-hybrid-search-rrf
+
+Delivery 2 after chunk identity. Enable hybrid multi-path + RRF on KnowledgeSearchPort without legacy SDK hybrid API. True BM25 schema rebuild is staged follow-up; this change ships sparse-lite lexical ranking over dense candidate union + filtered/unfiltered dense fusion.
@@ -0,0 +1,19 @@
+# Decisions: rag-hybrid-search-rrf
+
+## Capability
+
+sm-flow + OpenSpec fallback; apply pre-authorized.
+
+## Depends
+
+Delivery 1 archived.
+
+## Grill (compressed, pre-authorized)
+
+- Q: Full BM25 schema now? A: No — sparse-lite + RRF first; schema rebuild follow-up.
+- Q: Default mode? A: dense default; hybrid opt-in.
+- Q: Threshold score? A: keep dense-compatible L2 mapping.
+
+## Design
+
+Hybrid paths: dense unfiltered + dense filtered + lexical rank over union; RRF fuse by evidenceKey.
@@ -0,0 +1,5 @@
+# Evidence
+
+- Delivery 1 identity/port foundation required
+- RRF utility and hybrid adapter unit tests green
+- Lexical sparse-lite intentionally intermediate until BM25 schema