SuperBizAgent-java/openspec/specs/rag-knowledge-retrieval/spec.md

rag-knowledge-retrieval Specification

Purpose

Define the runtime contract for the explicit lookup_knowledge Agent tool, including how L0 keyword/frontmatter hints cooperate with L1 semantic retrieval while preserving metadata filters, fallback evidence, and traceable retrieval details.

Requirements

Requirement: Knowledge retrieval SHALL keep L0 as a hint provider

The lookup_knowledge retrieval flow SHALL retain L0 keyword/frontmatter matching but use it only as query understanding, filtering, rerank, and explainability hint data rather than as a final evidence retrieval decision.

Scenario: L0 produces traceable hint data

• WHEN L0 matches one or more indexed knowledge entries
• THEN the retrieval flow SHALL expose matched titles, matched keywords, domains or categories, and entity terms as structured hint data

Scenario: L0 does not bypass semantic retrieval

• WHEN L0 returns exactly one match
• THEN the retrieval flow SHALL still attempt semantic L1 retrieval unless L1 is explicitly disabled by configuration

Scenario: L0 hints do not become normal evidence

• WHEN L1 retrieval returns no usable evidence
• THEN L0 matched documents SHALL NOT be returned as fact evidence blocks
• AND L0 hint data MAY still be recorded in retrieval trace details

Requirement: Knowledge retrieval SHALL use L0 domain as optional L1 filter

The retrieval flow SHALL use L0 domain/category information as an optional metadata filter for L1 retrieval when the domain is unambiguous.

Scenario: Single domain filter

• WHEN L0 hint data contains exactly one nonblank domain or category
• THEN the L1 retrieval request SHALL include that category as a metadata filter

Scenario: Ambiguous domain fallback

• WHEN L0 hint data contains zero domains or multiple domains
• THEN the L1 retrieval request SHALL run without an L0-derived category filter

Requirement: Knowledge retrieval SHALL persist L0 hints

The system SHALL persist L0 hint details in tool_invocation.retrieval_details for lookup_knowledge calls.

Scenario: Retrieval details include L0 hints

• WHEN a lookup_knowledge call records a tool invocation
• THEN retrieval_details SHALL include L0 matched keywords, domains, entities, and titles when available

Scenario: Retrieval layer reflects cooperating retrieval

• WHEN both L0 hint data and L1 candidates participate in a lookup
• THEN the recorded retrieval layer SHALL be L0+L1

Requirement: Knowledge retrieval SHALL return structured evidence blocks

The lookup_knowledge retrieval flow SHALL expose retrieved evidence as structured evidence blocks.

Scenario: Evidence block contains source metadata

• WHEN a lookup_knowledge call returns evidence
• THEN each evidence block SHALL include source, title when available, breadcrumb when available, retrieval layer, content, and hit reasons

Scenario: Evidence blocks are the primary evidence contract

• WHEN evidence blocks are returned
• THEN Agent-facing knowledge content SHALL be derived from evidence blocks and context pack
• AND the result SHALL NOT rely on legacy primary or supplement fields for L0/L1 meaning

Requirement: Knowledge retrieval SHALL deduplicate evidence blocks

The retrieval flow SHALL remove duplicate evidence blocks before returning them to the Agent. Duplicates are defined by chunk-level evidenceKey identity, not by document source alone.

Scenario: Chunk-level deduplication keeps distinct chunks

• WHEN L1 produces multiple candidates with the same source but different chunk identities
• THEN the retrieval flow SHALL keep separate evidence blocks for those chunk identities subject to per-document caps
• AND SHALL NOT collapse them solely because source is equal

Scenario: Same evidenceKey collapses

• WHEN two candidates share the same evidenceKey
• THEN the retrieval flow SHALL keep a single evidence block for that identity
• AND the evidence block SHALL preserve hit reasons from both paths when available

Scenario: Postprocess count tracking

• WHEN evidence post-processing completes
• THEN the tool invocation details SHALL record candidate count and final evidence block count

Requirement: Knowledge retrieval SHALL persist evidence block summaries

The system SHALL persist compact evidence block summaries in tool_invocation.retrieval_details.

Scenario: Evidence summaries are persisted

• WHEN a lookup_knowledge call records a tool invocation
• THEN retrieval_details SHALL include evidence block summaries containing source, title, retrieval layer, score when available, and hit reasons

Scenario: Full content is not duplicated into retrieval details

• WHEN evidence block summaries are persisted
• THEN full evidence content SHALL be omitted or truncated so the trace record remains compact

Requirement: Knowledge retrieval SHALL support a disabled-by-default Spring AI sidecar

The retrieval system SHALL allow a Spring AI VectorStore retrieval path to be wired as a sidecar without changing the default lookup_knowledge runtime path.

Scenario: Sidecar disabled by default

• WHEN the application starts without explicit sidecar enablement
• THEN lookup_knowledge SHALL continue using the existing retrieval path
• AND Chat and AIOps runtime behavior SHALL not depend on the sidecar

Scenario: Sidecar failure does not break main retrieval

• WHEN the Spring AI sidecar is enabled but cannot initialize or query successfully
• THEN the existing retrieval path SHALL remain usable
• AND the failure SHALL be reported as sidecar status rather than as a main retrieval failure

Requirement: Knowledge retrieval SHALL normalize sidecar results for comparison

The sidecar retrieval path SHALL expose results in a comparable structure aligned with the current retrieval result shape.

Scenario: Comparable result metadata

• WHEN sidecar retrieval returns candidates
• THEN each comparable result SHALL include source or doc id, title when available, breadcrumb when available, category when available, rank, content preview, and the sidecar score label/value

Scenario: Score semantics are explicit

• WHEN current retrieval and sidecar retrieval scores are compared
• THEN the report SHALL label score semantics by path instead of assuming direct numeric equivalence

Requirement: Knowledge retrieval SHALL prefer Spring AI VectorStore when configured

The retrieval service SHALL support Spring AI VectorStore as the preferred vector retrieval abstraction without changing the lookup_knowledge tool contract.

Scenario: VectorStore mode uses Spring AI

• WHEN retrieval vector store mode is configured as spring-ai
• THEN semantic retrieval SHALL query through Spring AI VectorStore
• AND the returned candidates SHALL be normalized into the existing vector search result shape

Scenario: Auto mode prefers VectorStore

• WHEN retrieval vector store mode is configured as auto
• AND a Spring AI VectorStore bean is available
• THEN semantic retrieval SHALL attempt Spring AI VectorStore before the SDK path

Requirement: Knowledge retrieval SHALL preserve SDK fallback

The retrieval service SHALL keep the existing Milvus SDK retrieval implementation available.

Scenario: SDK mode bypasses VectorStore

• WHEN retrieval vector store mode is configured as sdk
• THEN semantic retrieval SHALL use the existing Milvus SDK path

Scenario: Auto fallback uses SDK

• WHEN retrieval vector store mode is auto
• AND Spring AI VectorStore is unavailable or fails
• THEN semantic retrieval SHALL fall back to the existing Milvus SDK path

Requirement: Knowledge retrieval SHALL keep score semantics explicit

The retrieval service SHALL preserve score semantics when results come from different retrieval implementations.

Scenario: SDK score remains L2 distance

• WHEN a candidate is returned by the SDK path
• THEN its score semantics SHALL remain compatible with existing L2 distance normalization

Scenario: VectorStore score is mapped without changing tool contract

• WHEN a candidate is returned by Spring AI VectorStore
• THEN it SHALL be mapped into the existing result shape
• AND trace or comparison code SHALL be able to distinguish it as a VectorStore similarity score when needed

Requirement: Knowledge retrieval SHALL reuse the existing Milvus collection

Spring AI Milvus integration SHALL be configured to use the existing collection schema unless explicitly changed.

Scenario: Existing field mapping

• WHEN Spring AI Milvus VectorStore is configured
• THEN it SHALL use the existing id, content, vector, and metadata field names
• AND it SHALL use the configured embedding dimension and metric type compatible with existing vectors

Requirement: Knowledge retrieval SHALL use a modular RAG pipeline

The lookup_knowledge tool SHALL route each request through explicit query transformation, vector retrieval, post-retrieval processing, context packing, result assembly, and trace recording components.

Scenario: Pipeline components execute in order

• WHEN lookup_knowledge receives a query
• THEN the system SHALL transform the query before retrieval
• AND it SHALL retrieve vector candidates before post-processing
• AND it SHALL build evidence blocks before context packing
• AND it SHALL record trace details after result assembly

Scenario: Tool boundary remains explicit

• WHEN the modular pipeline is used
• THEN the Agent SHALL still call the explicit lookup_knowledge tool with the same query argument
• AND the implementation SHALL NOT require an implicit Advisor to inject knowledge into every chat response

Requirement: Knowledge retrieval SHALL retry without L0 filter when filtered L1 is low quality

The retrieval flow SHALL treat L0-derived category filtering as an optimization, not as a hard dependency for final recall.

Scenario: Filtered retrieval succeeds

• WHEN L0 provides an unambiguous category filter
• AND filtered L1 retrieval returns usable evidence at or above the configured reference threshold
• THEN the tool SHALL use the filtered L1 candidates without running an unfiltered retry

Scenario: Filtered retrieval returns no evidence

• WHEN L0 provides a category filter
• AND filtered L1 retrieval returns no candidates or no final evidence blocks
• THEN the tool SHALL retry L1 retrieval with the raw query and no L0-derived category filter
• AND the retrieval trace SHALL record fallback reason filtered_vector_no_evidence

Scenario: Filtered retrieval is below reference quality

• WHEN L0 provides a category filter
• AND filtered L1 retrieval returns candidates whose top normalized similarity is below the configured reference threshold
• THEN the tool SHALL retry L1 retrieval with the raw query and no L0-derived category filter
• AND the retrieval trace SHALL record fallback reason filtered_vector_low_quality

Scenario: Both retrieval attempts fail

• WHEN filtered L1 retrieval and unfiltered L1 retry both produce no usable evidence
• THEN the tool SHALL return found=false
• AND the tool SHALL set evidence status to no_evidence
• AND the tool SHALL NOT return L0 documents as fact evidence

Requirement: Knowledge retrieval SHALL return an evidence-first result contract

The lookup_knowledge result SHALL expose structured evidence and packed context as the preferred contract.

Scenario: Evidence result contains context and traces

• WHEN lookup_knowledge returns usable evidence
• THEN the result SHALL include evidenceBlocks
• AND it SHALL include contextPack
• AND it SHALL include retrievalTrace
• AND it SHALL include rerankTrace
• AND it SHALL include relevanceLevel and completenessHint

Scenario: No-evidence result keeps traceability

• WHEN lookup_knowledge returns no usable evidence
• THEN the result SHALL include found=false
• AND it SHALL include a message explaining that no knowledge evidence was found
• AND it SHALL include retrieval trace details for attempted retrieval paths

Requirement: Knowledge retrieval SHALL pack evidence context for Agent consumption

The post-retrieval flow SHALL convert final evidence blocks into a compact context package for the Agent.

Scenario: Context pack preserves source metadata

• WHEN evidence blocks are packed
• THEN the packed context SHALL preserve source, title when available, breadcrumb when available, and hit reasons for included evidence

Scenario: Context pack respects budget

• WHEN final evidence content exceeds the configured context budget
• THEN the packer SHALL truncate content rather than source metadata
• AND it SHALL record included and omitted sources in the context pack summary

Requirement: Knowledge retrieval SHALL rerank evidence with traceable rule signals

The post-retrieval flow SHALL rerank vector candidates using deterministic rule-based signals and expose the explanation.

Scenario: Rerank trace records score contributions

• WHEN candidates are reranked
• THEN the rerank trace SHALL record final rank, source, base retrieval score when available, and major boost reasons for top evidence blocks

Scenario: Query hints influence rerank without becoming evidence

• WHEN L0 query hints match candidate metadata or content
• THEN the reranker MAY boost the candidate
• AND the evidence block SHALL record the hint as a hit reason
• AND the system SHALL NOT treat the L0 hint itself as fact evidence

Requirement: Lookup knowledge SHALL persist modular RAG trace details

The lookup_knowledge tool SHALL persist modular RAG pipeline details in tool_invocation.retrieval_details for every new lookup invocation.

Scenario: Evidence lookup persists modular detail keys

• WHEN lookup_knowledge returns usable evidence
• THEN retrieval_details SHALL include query_transform
• AND it SHALL include retrieval_trace
• AND it SHALL include context_pack_summary
• AND it SHALL include rerank_trace
• AND it SHALL include evidence_blocks

Scenario: Fallback lookup persists fallback reason

• WHEN lookup_knowledge performs an unfiltered retry after filtered retrieval fails or is low quality
• THEN retrieval_details.retrieval_trace SHALL include the selected attempt
• AND retrieval_details.fallback_reason SHALL preserve the fallback reason

Scenario: No-evidence lookup still preserves trace

• WHEN lookup_knowledge returns no usable evidence
• THEN retrieval_details SHALL still include retrieval trace information for attempted retrieval paths
• AND it SHALL not include full evidence content as duplicated trace data