docs: archive rag and aiops query changes

This commit is contained in:
aruo
2026-07-05 13:12:47 +08:00
parent 72a3dbf8c5
commit 2658742119
12 changed files with 37 additions and 0 deletions
@@ -42,3 +42,20 @@ The system SHALL continue using existing `agent_step` and `tool_invocation` pers
#### Scenario: AIOps uses evidence tools
- **WHEN** the AIOps flow calls available evidence tools
- **THEN** existing hooks and recorders persist agent steps and tool invocations under the resolved AIOps session id
### Requirement: AIOps payload prompts SHALL include a recommended knowledge query
When an AIOps request includes alert payload fields, the system SHALL include a deterministic recommended knowledge retrieval query in the prompt sent to the Agent flow.
#### Scenario: Payload-targeted prompt includes knowledge query
- **WHEN** an AIOps request contains alert name, service, severity, or description
- **THEN** the generated task prompt SHALL include a recommended `lookup_knowledge` query derived from the supplied payload fields
#### Scenario: Query skips blank fields
- **WHEN** some AIOps payload fields are blank
- **THEN** the recommended knowledge query SHALL omit those blank fields
- **AND** it SHALL preserve the non-blank alert-specific terms
#### Scenario: Auto-discovery prompt does not invent payload query
- **WHEN** an AIOps request does not include alert payload fields
- **THEN** the generated task prompt SHALL remain in auto-discovery mode
- **AND** it SHALL not include a payload-derived recommended knowledge query
@@ -93,3 +93,23 @@ The offline RAG retrieval baseline SHALL remain runnable after the main retrieva
#### Scenario: Baseline is checked during migration
- **WHEN** the VectorStore integration change is implemented
- **THEN** the existing offline baseline evaluator SHALL be run and its generated report noise SHALL not be committed unless the baseline intentionally changes
### Requirement: Retrieval evaluation SHALL provide live post-reindex acceptance
The retrieval evaluation system SHALL provide an opt-in live acceptance flow for validating retrieval behavior after embedding input changes require a knowledge-base reindex.
#### Scenario: Live acceptance requires a running service
- **WHEN** live retrieval acceptance is run
- **THEN** it SHALL call the configured Spring Boot retrieval endpoint
- **AND** it SHALL not be required by the offline fixture baseline
#### Scenario: Live acceptance records retrieval evidence
- **WHEN** a live retrieval case is executed
- **THEN** the report SHALL include the query, requested topK, result count, top candidate titles or sources, score labels, and raw response fields needed for review
#### Scenario: Reindex prerequisite is documented
- **WHEN** a developer prepares to validate breadcrumb-aware embedding behavior
- **THEN** the repository SHALL explain that existing vectors must be reindexed before live validation can reflect the new embedding text
#### Scenario: Live report is reviewable
- **WHEN** the live acceptance script completes
- **THEN** it SHALL write JSON and Markdown outputs that can be inspected or attached to interview evidence