feat(rag): chunk evidence identity, dedup, and search port

Preserve same-document multi-chunk evidence with evidenceKey identity,
per-document caps, retrieve-k/return-n split, and a dense KnowledgeSearchPort.
Archives Delivery 1 OpenSpec change as the foundation for hybrid retrieval.
This commit is contained in:
zhuyongxin
2026-07-27 18:26:15 +08:00
parent 99d4f6f216
commit ac1f831903
34 changed files with 2880 additions and 298 deletions
@@ -0,0 +1,43 @@
# Tasks: rag-chunk-evidence-identity-dedup
## 1. Identity model
- [x] 1.1 Add `docId`, `chunkIndex`, `evidenceKey` to `RetrievedEvidenceCandidate` and `EvidenceBlock`
- [x] 1.2 Implement shared identity helper (`docId#chunk-N` / `vector:{id}` / `rank:{n}`)
- [x] 1.3 Extract identity in `KnowledgeDocumentRetriever` (or search-port mapper) from metadata
## 2. Search port boundary
- [x] 2.1 Add `KnowledgeSearchPort`, `KnowledgeSearchRequest`, `KnowledgeSearchHit`
- [x] 2.2 Implement dense adapter delegating to `VectorSearchService`
- [x] 2.3 Wire retriever to port; keep tool orchestration free of SDK details
## 3. Post-process dedup and caps
- [x] 3.1 Change dedup key to `evidenceKey`
- [x] 3.2 Add `rag.max-chunks-per-document` (default 2)
- [x] 3.3 Apply `rag.return-n` truncation after ranking/dedup/cap
- [x] 3.4 Keep score-threshold / relevance behavior unchanged except ordering inputs
## 4. Lookup config
- [x] 4.1 Add `rag.retrieve-k` and `rag.return-n` with legacy `rag.top-k` fallback
- [x] 4.2 Use retrieve-k for search port calls in `LookupKnowledgeTool`
## 5. Projector alignment
- [x] 5.1 Prefer evidenceKey / chunk-scoped id as projected `document_id`
- [x] 5.2 Stop dropping second evidence solely because `source` matches
- [x] 5.3 Keep existing budget/truncation behavior
## 6. Tests
- [x] 6.1 Update `LookupKnowledgeToolTest` source-dedup case to chunk-preserving behavior
- [x] 6.2 Add post-processor tests: multi-chunk keep, true-dup merge, maxChunksPerDocument
- [x] 6.3 Update/add `RagResultProjectorTest` for same-source multi-chunk projection
- [x] 6.4 Add search-port adapter smoke test if practical
## 7. Verification
- [x] 7.1 Run targeted unit tests for lookup / post-processor / projector
- [x] 7.2 Mark tasks complete and note any residual risks for Delivery 2