SuperBizAgent-java/openspec/changes/archive/2026-07-04-rag-spring-ai-vectorstore-sidecar/design.md

Context

The project already uses Spring AI/Spring AI Alibaba for model and agent capabilities, but RAG vector retrieval still uses the Milvus Java SDK directly. The current main path is now observable and covered by golden retrieval cases, so the next migration step should compare framework retrieval behavior without changing Chat or AIOps runtime behavior.

Goals / Non-Goals

Goals:

• Introduce a Spring AI VectorStore sidecar behind configuration.
• Keep lookup_knowledge and VectorSearchService as the default production path.
• Normalize sidecar results into the same comparable shape as current VectorSearchService.SearchResult.
• Add an offline or developer-triggered comparison report that runs golden cases through both retrieval paths.
• Capture schema and scoring differences before deciding whether to replace the current implementation.

Non-Goals:

• Do not replace VectorSearchService in this change.
• Do not change document upload, chunking, or Milvus collection schema.
• Do not introduce query transformer, multi-query, RRF, or rerank behavior.
• Do not make Spring AI Advisor the RAG entry point.

Decisions

Decision: Sidecar over replacement

Add a separate sidecar service/adapter instead of changing the existing retrieval service.

Rationale: the current path is already used by Chat and AIOps, and framework behavior may differ in score semantics, metadata filtering, or expected schema. A sidecar lets us compare before cutting over.

Alternative considered: replace VectorSearchService immediately. Rejected because it would conflate dependency integration with retrieval behavior migration.

Decision: Preserve explicit tool boundary

The sidecar will be called by evaluation or diagnostic code, not by implicit Chat Advisor behavior.

Rationale: the interview value of the project is Agent engineering observability: explicit tool calls, evidence blocks, and tool_invocation traces.

Alternative considered: use Spring AI Advisor directly. Rejected for now because it hides the decision point where the Agent chooses retrieval.

Decision: Compare normalized results

Both retrieval paths should be mapped into a small comparable result shape containing source/doc id, title, breadcrumb, category, score/distance, rank, and content preview.

Rationale: direct score equality is unlikely because the current path uses Milvus L2 distance while Spring AI abstractions may expose similarity scores or provider-specific values. The first useful comparison is source/rank/metadata coverage.

Decision: Keep dependency risk isolated

If the current dependency set does not expose a compatible Milvus VectorStore, the first implementation should add a narrow optional dependency/config class and keep it disabled by default.

Rationale: Spring AI version compatibility is a migration risk. The project should still build and run with the current main path if sidecar configuration is absent.

Risks / Trade-offs

• Spring AI Milvus schema may not match the existing collection -> keep sidecar disabled by default and report incompatibility rather than failing the app.
• Score semantics may differ from current L2 distance -> compare rank/source metadata first and label score fields by retrieval path.
• Adding framework dependencies may affect startup auto-configuration -> guard sidecar beans behind properties or conditions.
• Sidecar evaluation may require live Milvus unlike the baseline fixture evaluator -> make live comparison opt-in and keep offline baseline unchanged.

Migration Plan

1. Add sidecar configuration and adapter behind rag.sidecar.spring-ai.enabled=false.
2. Add comparison command/script/service that runs golden cases through current retrieval plus sidecar when enabled.
3. Store comparison reports separately from the offline baseline reports.
4. Use report differences to decide whether a later change should replace VectorSearchService internals.
5. Rollback is disabling the sidecar property or reverting the sidecar dependency/config only; the main path remains unchanged.

Open Questions

• Which exact Spring AI Milvus VectorStore artifact is compatible with the existing Spring AI/Spring AI Alibaba BOM versions?
• Can the current Milvus collection be queried by Spring AI VectorStore without schema migration, or do we need a second collection for sidecar experiments?
• Should sidecar comparison run from Java tests, a script, or a developer-only endpoint/runner?