154 lines
8.7 KiB
Markdown
154 lines
8.7 KiB
Markdown
# rag-knowledge-retrieval Specification
|
|
|
|
## Purpose
|
|
Define the runtime contract for the explicit `lookup_knowledge` Agent tool, including how L0 keyword/frontmatter hints cooperate with L1 semantic retrieval while preserving metadata filters, fallback evidence, and traceable retrieval details.
|
|
## Requirements
|
|
### Requirement: Knowledge retrieval SHALL keep L0 as a hint provider
|
|
The `lookup_knowledge` retrieval flow SHALL retain L0 keyword/frontmatter matching but use it as domain, entity, and explainability hint data rather than as the sole final retrieval decision.
|
|
|
|
#### Scenario: L0 produces traceable hint data
|
|
- **WHEN** L0 matches one or more indexed knowledge entries
|
|
- **THEN** the retrieval flow SHALL expose matched titles, matched keywords, domains or categories, and entity terms as structured hint data
|
|
|
|
#### Scenario: L0 does not bypass semantic retrieval by default
|
|
- **WHEN** L0 returns exactly one match
|
|
- **THEN** the retrieval flow SHALL still attempt semantic L1 retrieval unless L1 is unavailable or explicitly disabled by configuration
|
|
|
|
### Requirement: Knowledge retrieval SHALL use L0 domain as optional L1 filter
|
|
The retrieval flow SHALL use L0 domain/category information as an optional metadata filter for L1 retrieval when the domain is unambiguous.
|
|
|
|
#### Scenario: Single domain filter
|
|
- **WHEN** L0 hint data contains exactly one nonblank domain or category
|
|
- **THEN** the L1 retrieval request SHALL include that category as a metadata filter
|
|
|
|
#### Scenario: Ambiguous domain fallback
|
|
- **WHEN** L0 hint data contains zero domains or multiple domains
|
|
- **THEN** the L1 retrieval request SHALL run without an L0-derived category filter
|
|
|
|
### Requirement: Knowledge retrieval SHALL preserve fallback evidence
|
|
The retrieval flow SHALL still return useful L0 evidence when L1 produces no usable result.
|
|
|
|
#### Scenario: L1 has no results
|
|
- **WHEN** L0 has at least one match and L1 returns no candidates
|
|
- **THEN** the tool SHALL return an L0-based primary result
|
|
- **AND** the relevance assessment SHALL not claim semantic support from L1
|
|
|
|
#### Scenario: L1 fails
|
|
- **WHEN** L0 has at least one match and L1 retrieval throws or fails
|
|
- **THEN** the tool SHALL return an L0-based primary result
|
|
- **AND** the tool invocation record SHALL preserve the L0 hint details
|
|
|
|
### Requirement: Knowledge retrieval SHALL persist L0 hints
|
|
The system SHALL persist L0 hint details in `tool_invocation.retrieval_details` for `lookup_knowledge` calls.
|
|
|
|
#### Scenario: Retrieval details include L0 hints
|
|
- **WHEN** a `lookup_knowledge` call records a tool invocation
|
|
- **THEN** `retrieval_details` SHALL include L0 matched keywords, domains, entities, and titles when available
|
|
|
|
#### Scenario: Retrieval layer reflects cooperating retrieval
|
|
- **WHEN** both L0 hint data and L1 candidates participate in a lookup
|
|
- **THEN** the recorded retrieval layer SHALL be `L0+L1`
|
|
|
|
### Requirement: Knowledge retrieval SHALL return structured evidence blocks
|
|
The `lookup_knowledge` retrieval flow SHALL expose retrieved evidence as structured evidence blocks in addition to the existing compatibility fields.
|
|
|
|
#### Scenario: Evidence block contains source metadata
|
|
- **WHEN** a `lookup_knowledge` call returns evidence
|
|
- **THEN** each evidence block SHALL include source, title when available, breadcrumb when available, retrieval layer, content, and hit reasons
|
|
|
|
#### Scenario: Compatibility fields remain available
|
|
- **WHEN** evidence blocks are returned
|
|
- **THEN** the existing `primary` and `supplement` result fields SHALL remain available when their source evidence exists
|
|
|
|
### Requirement: Knowledge retrieval SHALL deduplicate evidence blocks
|
|
The retrieval flow SHALL remove duplicate evidence blocks before returning them to the Agent.
|
|
|
|
#### Scenario: Duplicate source deduplication
|
|
- **WHEN** L0 and L1 produce evidence with the same source identity
|
|
- **THEN** the retrieval flow SHALL keep a single evidence block for that source
|
|
- **AND** the evidence block SHALL preserve hit reasons from both retrieval paths when available
|
|
|
|
#### Scenario: Postprocess count tracking
|
|
- **WHEN** evidence post-processing completes
|
|
- **THEN** the tool invocation details SHALL record candidate count and final evidence block count
|
|
|
|
### Requirement: Knowledge retrieval SHALL persist evidence block summaries
|
|
The system SHALL persist compact evidence block summaries in `tool_invocation.retrieval_details`.
|
|
|
|
#### Scenario: Evidence summaries are persisted
|
|
- **WHEN** a `lookup_knowledge` call records a tool invocation
|
|
- **THEN** `retrieval_details` SHALL include evidence block summaries containing source, title, retrieval layer, score when available, and hit reasons
|
|
|
|
#### Scenario: Full content is not duplicated into retrieval details
|
|
- **WHEN** evidence block summaries are persisted
|
|
- **THEN** full evidence content SHALL be omitted or truncated so the trace record remains compact
|
|
|
|
### Requirement: Knowledge retrieval SHALL support a disabled-by-default Spring AI sidecar
|
|
The retrieval system SHALL allow a Spring AI VectorStore retrieval path to be wired as a sidecar without changing the default `lookup_knowledge` runtime path.
|
|
|
|
#### Scenario: Sidecar disabled by default
|
|
- **WHEN** the application starts without explicit sidecar enablement
|
|
- **THEN** `lookup_knowledge` SHALL continue using the existing retrieval path
|
|
- **AND** Chat and AIOps runtime behavior SHALL not depend on the sidecar
|
|
|
|
#### Scenario: Sidecar failure does not break main retrieval
|
|
- **WHEN** the Spring AI sidecar is enabled but cannot initialize or query successfully
|
|
- **THEN** the existing retrieval path SHALL remain usable
|
|
- **AND** the failure SHALL be reported as sidecar status rather than as a main retrieval failure
|
|
|
|
### Requirement: Knowledge retrieval SHALL normalize sidecar results for comparison
|
|
The sidecar retrieval path SHALL expose results in a comparable structure aligned with the current retrieval result shape.
|
|
|
|
#### Scenario: Comparable result metadata
|
|
- **WHEN** sidecar retrieval returns candidates
|
|
- **THEN** each comparable result SHALL include source or doc id, title when available, breadcrumb when available, category when available, rank, content preview, and the sidecar score label/value
|
|
|
|
#### Scenario: Score semantics are explicit
|
|
- **WHEN** current retrieval and sidecar retrieval scores are compared
|
|
- **THEN** the report SHALL label score semantics by path instead of assuming direct numeric equivalence
|
|
|
|
### Requirement: Knowledge retrieval SHALL prefer Spring AI VectorStore when configured
|
|
The retrieval service SHALL support Spring AI VectorStore as the preferred vector retrieval abstraction without changing the `lookup_knowledge` tool contract.
|
|
|
|
#### Scenario: VectorStore mode uses Spring AI
|
|
- **WHEN** retrieval vector store mode is configured as `spring-ai`
|
|
- **THEN** semantic retrieval SHALL query through Spring AI `VectorStore`
|
|
- **AND** the returned candidates SHALL be normalized into the existing vector search result shape
|
|
|
|
#### Scenario: Auto mode prefers VectorStore
|
|
- **WHEN** retrieval vector store mode is configured as `auto`
|
|
- **AND** a Spring AI `VectorStore` bean is available
|
|
- **THEN** semantic retrieval SHALL attempt Spring AI `VectorStore` before the SDK path
|
|
|
|
### Requirement: Knowledge retrieval SHALL preserve SDK fallback
|
|
The retrieval service SHALL keep the existing Milvus SDK retrieval implementation available.
|
|
|
|
#### Scenario: SDK mode bypasses VectorStore
|
|
- **WHEN** retrieval vector store mode is configured as `sdk`
|
|
- **THEN** semantic retrieval SHALL use the existing Milvus SDK path
|
|
|
|
#### Scenario: Auto fallback uses SDK
|
|
- **WHEN** retrieval vector store mode is `auto`
|
|
- **AND** Spring AI `VectorStore` is unavailable or fails
|
|
- **THEN** semantic retrieval SHALL fall back to the existing Milvus SDK path
|
|
|
|
### Requirement: Knowledge retrieval SHALL keep score semantics explicit
|
|
The retrieval service SHALL preserve score semantics when results come from different retrieval implementations.
|
|
|
|
#### Scenario: SDK score remains L2 distance
|
|
- **WHEN** a candidate is returned by the SDK path
|
|
- **THEN** its score semantics SHALL remain compatible with existing L2 distance normalization
|
|
|
|
#### Scenario: VectorStore score is mapped without changing tool contract
|
|
- **WHEN** a candidate is returned by Spring AI `VectorStore`
|
|
- **THEN** it SHALL be mapped into the existing result shape
|
|
- **AND** trace or comparison code SHALL be able to distinguish it as a VectorStore similarity score when needed
|
|
|
|
### Requirement: Knowledge retrieval SHALL reuse the existing Milvus collection
|
|
Spring AI Milvus integration SHALL be configured to use the existing collection schema unless explicitly changed.
|
|
|
|
#### Scenario: Existing field mapping
|
|
- **WHEN** Spring AI Milvus VectorStore is configured
|
|
- **THEN** it SHALL use the existing id, content, vector, and metadata field names
|
|
- **AND** it SHALL use the configured embedding dimension and metric type compatible with existing vectors
|