Add configurable hybrid mode on KnowledgeSearchPort that fuses dense unfiltered, dense filtered, and lexical ranks via RRF while preserving dense-compatible threshold scores. Archives Delivery 2 OpenSpec change.
1.9 KiB
1.9 KiB
rag-hybrid-search Specification
Purpose
TBD - created by archiving change rag-hybrid-search-rrf. Update Purpose after archive.
Requirements
Requirement: Knowledge search SHALL support configurable dense and hybrid modes
The knowledge search port SHALL support retrieval.search.mode values dense and hybrid. Default SHALL be dense for backward-compatible behavior.
Scenario: Dense mode single path
- WHEN mode is dense
- THEN the port SHALL perform a single dense vector search with the provided category filter
Scenario: Hybrid mode multi-path fusion
- WHEN mode is hybrid
- THEN the port SHALL gather multiple retrieval paths and fuse them by RRF before returning hits
Requirement: Hybrid fusion SHALL use RRF over evidence identities
Hybrid fusion SHALL score candidates by reciprocal rank fusion on evidenceKey identities and SHALL NOT require comparable raw scores across paths.
Scenario: Multi-path candidate rises with RRF
- WHEN a chunk ranks highly on more than one hybrid path
- THEN its fused rank SHALL improve relative to a chunk that ranks highly on only one path
Requirement: Hybrid SHALL preserve dense quality scores for thresholds
Fused ordering SHALL NOT replace the dense compatibility score used by post-process quality thresholds.
Scenario: Threshold score remains dense-compatible
- WHEN hybrid returns a hit that originated from dense search
- THEN hit.score SHALL remain a dense-compatible distance/similarity mapping usable by existing normalizeL2 thresholds
Requirement: Hybrid SHALL keep chunk identity fields
Hybrid hits SHALL populate docId, chunkIndex, and evidenceKey consistently with Delivery 1 identity rules.
Scenario: Fused hit keeps evidenceKey
- WHEN hybrid merges the same chunk from two paths
- THEN the returned hit SHALL use one evidenceKey and retain chunk identity metadata