feat: add spring ai retrieval sidecar
This commit is contained in:
aruo
@@ -0,0 +1,2 @@
|
||||
schema: spec-driven
|
||||
created: 2026-07-04
|
||||
@@ -0,0 +1,71 @@
|
||||
## Context
|
||||
|
||||
The project already uses Spring AI/Spring AI Alibaba for model and agent capabilities, but RAG vector retrieval still uses the Milvus Java SDK directly. The current main path is now observable and covered by golden retrieval cases, so the next migration step should compare framework retrieval behavior without changing Chat or AIOps runtime behavior.
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- Introduce a Spring AI VectorStore sidecar behind configuration.
|
||||
- Keep `lookup_knowledge` and `VectorSearchService` as the default production path.
|
||||
- Normalize sidecar results into the same comparable shape as current `VectorSearchService.SearchResult`.
|
||||
- Add an offline or developer-triggered comparison report that runs golden cases through both retrieval paths.
|
||||
- Capture schema and scoring differences before deciding whether to replace the current implementation.
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- Do not replace `VectorSearchService` in this change.
|
||||
- Do not change document upload, chunking, or Milvus collection schema.
|
||||
- Do not introduce query transformer, multi-query, RRF, or rerank behavior.
|
||||
- Do not make Spring AI Advisor the RAG entry point.
|
||||
|
||||
## Decisions
|
||||
|
||||
### Decision: Sidecar over replacement
|
||||
|
||||
Add a separate sidecar service/adapter instead of changing the existing retrieval service.
|
||||
|
||||
Rationale: the current path is already used by Chat and AIOps, and framework behavior may differ in score semantics, metadata filtering, or expected schema. A sidecar lets us compare before cutting over.
|
||||
|
||||
Alternative considered: replace `VectorSearchService` immediately. Rejected because it would conflate dependency integration with retrieval behavior migration.
|
||||
|
||||
### Decision: Preserve explicit tool boundary
|
||||
|
||||
The sidecar will be called by evaluation or diagnostic code, not by implicit Chat Advisor behavior.
|
||||
|
||||
Rationale: the interview value of the project is Agent engineering observability: explicit tool calls, evidence blocks, and `tool_invocation` traces.
|
||||
|
||||
Alternative considered: use Spring AI Advisor directly. Rejected for now because it hides the decision point where the Agent chooses retrieval.
|
||||
|
||||
### Decision: Compare normalized results
|
||||
|
||||
Both retrieval paths should be mapped into a small comparable result shape containing source/doc id, title, breadcrumb, category, score/distance, rank, and content preview.
|
||||
|
||||
Rationale: direct score equality is unlikely because the current path uses Milvus L2 distance while Spring AI abstractions may expose similarity scores or provider-specific values. The first useful comparison is source/rank/metadata coverage.
|
||||
|
||||
### Decision: Keep dependency risk isolated
|
||||
|
||||
If the current dependency set does not expose a compatible Milvus VectorStore, the first implementation should add a narrow optional dependency/config class and keep it disabled by default.
|
||||
|
||||
Rationale: Spring AI version compatibility is a migration risk. The project should still build and run with the current main path if sidecar configuration is absent.
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- Spring AI Milvus schema may not match the existing collection -> keep sidecar disabled by default and report incompatibility rather than failing the app.
|
||||
- Score semantics may differ from current L2 distance -> compare rank/source metadata first and label score fields by retrieval path.
|
||||
- Adding framework dependencies may affect startup auto-configuration -> guard sidecar beans behind properties or conditions.
|
||||
- Sidecar evaluation may require live Milvus unlike the baseline fixture evaluator -> make live comparison opt-in and keep offline baseline unchanged.
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. Add sidecar configuration and adapter behind `rag.sidecar.spring-ai.enabled=false`.
|
||||
2. Add comparison command/script/service that runs golden cases through current retrieval plus sidecar when enabled.
|
||||
3. Store comparison reports separately from the offline baseline reports.
|
||||
4. Use report differences to decide whether a later change should replace `VectorSearchService` internals.
|
||||
5. Rollback is disabling the sidecar property or reverting the sidecar dependency/config only; the main path remains unchanged.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Which exact Spring AI Milvus VectorStore artifact is compatible with the existing Spring AI/Spring AI Alibaba BOM versions?
|
||||
- Can the current Milvus collection be queried by Spring AI VectorStore without schema migration, or do we need a second collection for sidecar experiments?
|
||||
- Should sidecar comparison run from Java tests, a script, or a developer-only endpoint/runner?
|
||||
@@ -0,0 +1,27 @@
|
||||
## Why
|
||||
|
||||
The current RAG retrieval path talks to Milvus through the raw Java SDK, so framework-level retrieval behavior cannot be compared safely. Before replacing the main path, we need a Spring AI VectorStore sidecar that can run the same golden cases and expose differences without affecting `lookup_knowledge`.
|
||||
|
||||
## What Changes
|
||||
|
||||
- Add a disabled-by-default Spring AI VectorStore sidecar retrieval path.
|
||||
- Keep the current `VectorSearchService` as the production path for Chat and AIOps.
|
||||
- Add an adapter/reporting surface that can run golden retrieval cases against both current and sidecar paths.
|
||||
- Record comparable fields: result id/source, title, breadcrumb, score/distance, category, and metadata.
|
||||
- Document incompatibilities between the current Milvus schema and Spring AI VectorStore behavior.
|
||||
- No breaking changes.
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
- None.
|
||||
|
||||
### Modified Capabilities
|
||||
- `rag-knowledge-retrieval`: Add requirements for sidecar Spring AI retrieval comparison while preserving the explicit `lookup_knowledge` tool boundary.
|
||||
- `rag-retrieval-evaluation`: Add requirements for comparing baseline retrieval with the sidecar retriever on the golden case set.
|
||||
|
||||
## Impact
|
||||
|
||||
- Affects retrieval service wiring, configuration, and evaluation scripts.
|
||||
- May add Spring AI VectorStore dependency/configuration if the current dependency set does not already expose it.
|
||||
- Does not change document upload, chunking, Milvus collection schema, Agent prompts, AIOps diagnosis flow, or the default `lookup_knowledge` runtime path.
|
||||
+25
@@ -0,0 +1,25 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Knowledge retrieval SHALL support a disabled-by-default Spring AI sidecar
|
||||
The retrieval system SHALL allow a Spring AI VectorStore retrieval path to be wired as a sidecar without changing the default `lookup_knowledge` runtime path.
|
||||
|
||||
#### Scenario: Sidecar disabled by default
|
||||
- **WHEN** the application starts without explicit sidecar enablement
|
||||
- **THEN** `lookup_knowledge` SHALL continue using the existing retrieval path
|
||||
- **AND** Chat and AIOps runtime behavior SHALL not depend on the sidecar
|
||||
|
||||
#### Scenario: Sidecar failure does not break main retrieval
|
||||
- **WHEN** the Spring AI sidecar is enabled but cannot initialize or query successfully
|
||||
- **THEN** the existing retrieval path SHALL remain usable
|
||||
- **AND** the failure SHALL be reported as sidecar status rather than as a main retrieval failure
|
||||
|
||||
### Requirement: Knowledge retrieval SHALL normalize sidecar results for comparison
|
||||
The sidecar retrieval path SHALL expose results in a comparable structure aligned with the current retrieval result shape.
|
||||
|
||||
#### Scenario: Comparable result metadata
|
||||
- **WHEN** sidecar retrieval returns candidates
|
||||
- **THEN** each comparable result SHALL include source or doc id, title when available, breadcrumb when available, category when available, rank, content preview, and the sidecar score label/value
|
||||
|
||||
#### Scenario: Score semantics are explicit
|
||||
- **WHEN** current retrieval and sidecar retrieval scores are compared
|
||||
- **THEN** the report SHALL label score semantics by path instead of assuming direct numeric equivalence
|
||||
+21
@@ -0,0 +1,21 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Retrieval evaluation SHALL compare current and sidecar retrieval paths
|
||||
The retrieval evaluation system SHALL provide an opt-in comparison between the existing retrieval path and the Spring AI sidecar retrieval path.
|
||||
|
||||
#### Scenario: Sidecar comparison report
|
||||
- **WHEN** sidecar comparison is run for the golden case set
|
||||
- **THEN** the report SHALL include per-case current-path top candidates and sidecar top candidates
|
||||
- **AND** it SHALL highlight source, breadcrumb, category, rank, and score-label differences
|
||||
|
||||
#### Scenario: Offline baseline remains unchanged
|
||||
- **WHEN** the fixture-based offline baseline evaluator is run
|
||||
- **THEN** it SHALL not require live Milvus, Spring Boot, or Spring AI sidecar configuration
|
||||
|
||||
### Requirement: Retrieval evaluation SHALL make sidecar readiness visible
|
||||
The sidecar comparison report SHALL show whether the Spring AI sidecar was runnable for the current environment.
|
||||
|
||||
#### Scenario: Sidecar unavailable
|
||||
- **WHEN** sidecar comparison is requested but the sidecar is disabled or unavailable
|
||||
- **THEN** the report SHALL mark sidecar status as unavailable
|
||||
- **AND** it SHALL keep current-path baseline results available for review
|
||||
@@ -0,0 +1,24 @@
|
||||
## 1. Dependency And Configuration
|
||||
|
||||
- [x] 1.1 Inspect available Spring AI VectorStore/Milvus classes for the current dependency set.
|
||||
- [x] 1.2 Add the narrow dependency or optional configuration needed for the sidecar path.
|
||||
- [x] 1.3 Add disabled-by-default sidecar properties under RAG configuration.
|
||||
|
||||
## 2. Sidecar Retrieval Adapter
|
||||
|
||||
- [x] 2.1 Define a comparable retrieval result DTO for current and sidecar paths.
|
||||
- [x] 2.2 Implement a Spring AI sidecar retrieval service that reports readiness and failures without breaking the main path.
|
||||
- [x] 2.3 Normalize sidecar metadata into source/doc id, title, breadcrumb, category, rank, content preview, and score label/value.
|
||||
|
||||
## 3. Comparison Evaluation
|
||||
|
||||
- [x] 3.1 Add a comparison service or script that runs golden cases against current retrieval and the sidecar path when enabled.
|
||||
- [x] 3.2 Write sidecar comparison JSON/Markdown reports separate from the offline baseline reports.
|
||||
- [x] 3.3 Preserve the existing offline evaluator behavior without requiring live Spring AI/Milvus services.
|
||||
|
||||
## 4. Tests And Validation
|
||||
|
||||
- [x] 4.1 Add tests for disabled sidecar fallback/readiness behavior.
|
||||
- [x] 4.2 Add tests for comparable result normalization and report generation.
|
||||
- [x] 4.3 Run targeted tests, the offline RAG retrieval baseline evaluator, and OpenSpec validation.
|
||||
- [x] 4.4 Review git diff to confirm the default `lookup_knowledge` runtime path is unchanged.
|
||||
@@ -82,3 +82,27 @@ The system SHALL persist compact evidence block summaries in `tool_invocation.re
|
||||
#### Scenario: Full content is not duplicated into retrieval details
|
||||
- **WHEN** evidence block summaries are persisted
|
||||
- **THEN** full evidence content SHALL be omitted or truncated so the trace record remains compact
|
||||
|
||||
### Requirement: Knowledge retrieval SHALL support a disabled-by-default Spring AI sidecar
|
||||
The retrieval system SHALL allow a Spring AI VectorStore retrieval path to be wired as a sidecar without changing the default `lookup_knowledge` runtime path.
|
||||
|
||||
#### Scenario: Sidecar disabled by default
|
||||
- **WHEN** the application starts without explicit sidecar enablement
|
||||
- **THEN** `lookup_knowledge` SHALL continue using the existing retrieval path
|
||||
- **AND** Chat and AIOps runtime behavior SHALL not depend on the sidecar
|
||||
|
||||
#### Scenario: Sidecar failure does not break main retrieval
|
||||
- **WHEN** the Spring AI sidecar is enabled but cannot initialize or query successfully
|
||||
- **THEN** the existing retrieval path SHALL remain usable
|
||||
- **AND** the failure SHALL be reported as sidecar status rather than as a main retrieval failure
|
||||
|
||||
### Requirement: Knowledge retrieval SHALL normalize sidecar results for comparison
|
||||
The sidecar retrieval path SHALL expose results in a comparable structure aligned with the current retrieval result shape.
|
||||
|
||||
#### Scenario: Comparable result metadata
|
||||
- **WHEN** sidecar retrieval returns candidates
|
||||
- **THEN** each comparable result SHALL include source or doc id, title when available, breadcrumb when available, category when available, rank, content preview, and the sidecar score label/value
|
||||
|
||||
#### Scenario: Score semantics are explicit
|
||||
- **WHEN** current retrieval and sidecar retrieval scores are compared
|
||||
- **THEN** the report SHALL label score semantics by path instead of assuming direct numeric equivalence
|
||||
|
||||
@@ -62,3 +62,23 @@ The system SHALL preserve generated baseline reports in JSON and Markdown format
|
||||
#### Scenario: Baseline regeneration is documented
|
||||
- **WHEN** a developer changes golden cases, fixtures, or evaluator logic
|
||||
- **THEN** the repository SHALL explain how to regenerate the retrieval baseline reports
|
||||
|
||||
### Requirement: Retrieval evaluation SHALL compare current and sidecar retrieval paths
|
||||
The retrieval evaluation system SHALL provide an opt-in comparison between the existing retrieval path and the Spring AI sidecar retrieval path.
|
||||
|
||||
#### Scenario: Sidecar comparison report
|
||||
- **WHEN** sidecar comparison is run for the golden case set
|
||||
- **THEN** the report SHALL include per-case current-path top candidates and sidecar top candidates
|
||||
- **AND** it SHALL highlight source, breadcrumb, category, rank, and score-label differences
|
||||
|
||||
#### Scenario: Offline baseline remains unchanged
|
||||
- **WHEN** the fixture-based offline baseline evaluator is run
|
||||
- **THEN** it SHALL not require live Milvus, Spring Boot, or Spring AI sidecar configuration
|
||||
|
||||
### Requirement: Retrieval evaluation SHALL make sidecar readiness visible
|
||||
The sidecar comparison report SHALL show whether the Spring AI sidecar was runnable for the current environment.
|
||||
|
||||
#### Scenario: Sidecar unavailable
|
||||
- **WHEN** sidecar comparison is requested but the sidecar is disabled or unavailable
|
||||
- **THEN** the report SHALL mark sidecar status as unavailable
|
||||
- **AND** it SHALL keep current-path baseline results available for review
|
||||
|
||||
Reference in New Issue
Block a user