## 1. Pipeline Models - [x] 1.1 Add query transformation model for original query, rewritten query, domain hints, matched keywords, entities, category filter, and trace-only L0 titles. - [x] 1.2 Add retrieved evidence candidate model that normalizes vector result metadata, score semantics, source, title, breadcrumb, content, retrieval attempt, and original rank. - [x] 1.3 Add context pack, retrieval trace, and rerank trace DTOs for the new `LookupResult` contract. - [x] 1.4 Update `LookupResult` to expose evidence blocks, context pack, retrieval trace, rerank trace, relevance level, completeness hint, retrieved domains, and message. ## 2. Query And Retrieval Pipeline - [x] 2.1 Implement `KnowledgeQueryTransformer` by reusing `KnowledgeIndexService.analyzeQuery` and mapping L0 output to query hints and optional category filter. - [x] 2.2 Implement `KnowledgeDocumentRetriever` as a wrapper around `VectorSearchService` for filtered and unfiltered vector retrieval attempts. - [x] 2.3 Implement low-quality detection using empty candidates, empty final evidence, or top normalized similarity below `retrieval.normalization.reference-threshold`. - [x] 2.4 Implement unfiltered raw-query retry when filtered retrieval is low quality and record fallback reason in retrieval trace. ## 3. Post-Retrieval Processing - [x] 3.1 Move relevance normalization out of `LookupKnowledgeTool` into `KnowledgeEvidencePostProcessor`. - [x] 3.2 Move evidence block creation and source-level deduplication out of `LookupKnowledgeTool` into the post-processor. - [x] 3.3 Implement rule-based lightweight rerank using vector similarity, domain match, entity match, keyword match, and metadata/source-type signals. - [x] 3.4 Ensure L0 hints influence filter/rerank/trace only and are not returned as standalone fact evidence when L1 has no usable evidence. ## 4. Context Packing And Result Assembly - [x] 4.1 Implement `KnowledgeContextPacker` with a configurable MVP character budget. - [x] 4.2 Pack evidence blocks while preserving source, title, breadcrumb, and hit reasons before truncating content. - [x] 4.3 Implement `LookupResultAssembler` to build evidence-first results for usable evidence, no-evidence, and session dedup cases. - [x] 4.4 Remove or migrate all `primary` and `supplement` result usage from production code. ## 5. Tool Boundary And Trace Recording - [x] 5.1 Refactor `LookupKnowledgeTool` into a thin orchestrator that invokes the pipeline and handles tool boundary concerns. - [x] 5.2 Update `ToolInvocationRecorder.LookupKnowledgeRecord` to summarize context pack, retrieval trace, rerank trace, fallback reason, and evidence blocks without relying on `result.getPrimary()`. - [x] 5.3 Preserve stable `tool_invocation` table fields and store new retrieval details in JSON. - [x] 5.4 Update `@Tool` description and relevant executor prompt text to describe evidence blocks, context pack, and no-realtime-data boundaries. ## 6. Tests And Evaluation - [x] 6.1 Update `LookupKnowledgeToolTest` for evidence-first result contract and removal of `primary` / `supplement`. - [x] 6.2 Add tests for filtered L1 success without retry. - [x] 6.3 Add tests for filtered low-quality retrieval triggering raw unfiltered L1 retry. - [x] 6.4 Add tests proving L0 hint data does not become standalone fact evidence when L1 fails. - [x] 6.5 Add tests for rerank ordering, rerank trace, context pack budget behavior, and source metadata preservation. - [x] 6.6 Update `ToolInvocationRecorderTest` for new retrieval detail summaries and output preview source. - [x] 6.7 Run targeted Java tests for lookup knowledge and recorder changes. - [x] 6.8 Run OpenSpec validation for `modular-rag-pipeline`. ## 7. Documentation Cleanup - [x] 7.1 Update RAG architecture docs to reflect modular pipeline, unfiltered vector retry, and evidence-first contract. - [x] 7.2 Update retrieval observability docs to remove L0 primary fallback and `primary` / `supplement` compatibility language. - [x] 7.3 Review git diff to confirm only expected RAG, prompt, test, and spec files changed.