83 lines
4.5 KiB
Markdown
83 lines
4.5 KiB
Markdown
# aiops-traceable-diagnosis-entry Specification
|
|
|
|
## Purpose
|
|
TBD - created by archiving change aiops-traceable-diagnosis-entry. Update Purpose after archive.
|
|
## Requirements
|
|
### Requirement: AIOps endpoint accepts optional alert input
|
|
The system SHALL allow `POST /api/ai_ops` to accept an optional JSON request body describing the alert diagnosis request.
|
|
|
|
#### Scenario: Caller supplies alert input
|
|
- **WHEN** a caller posts to `/api/ai_ops` with alert fields such as session id, alert name, service, severity, description, and time range
|
|
- **THEN** the AIOps analysis uses those fields to build the diagnosis task prompt and persisted session query
|
|
|
|
#### Scenario: Caller omits alert input
|
|
- **WHEN** a caller posts to `/api/ai_ops` without a body
|
|
- **THEN** the system still starts the default AIOps alert-analysis flow
|
|
|
|
### Requirement: AIOps session id is traceable
|
|
The system SHALL resolve a stable AIOps session id from the request when provided, otherwise generate one, and SHALL expose that session id to the SSE caller.
|
|
|
|
#### Scenario: Request includes session id
|
|
- **WHEN** a caller posts to `/api/ai_ops` with `sessionId`
|
|
- **THEN** the created `diagnosis_session.session_id` equals that value
|
|
- **AND** the SSE stream includes the same session id
|
|
|
|
#### Scenario: Request omits session id
|
|
- **WHEN** a caller posts to `/api/ai_ops` without `sessionId`
|
|
- **THEN** the system generates a session id
|
|
- **AND** the SSE stream includes the generated session id
|
|
|
|
### Requirement: AIOps report is persisted for trace replay
|
|
The system SHALL persist the final AIOps report into the existing `diagnosis_session.answer` field when a report is available.
|
|
|
|
#### Scenario: AIOps report is generated
|
|
- **WHEN** the AIOps planner/executor flow returns a final report
|
|
- **THEN** the corresponding diagnosis session is marked successful
|
|
- **AND** `diagnosis_session.answer` stores the final report
|
|
- **AND** `GET /api/diagnosis/{sessionId}/trace` can include that answer
|
|
|
|
### Requirement: AIOps trace uses existing evidence tables
|
|
The system SHALL continue using existing `agent_step` and `tool_invocation` persistence for AIOps trace evidence.
|
|
|
|
#### Scenario: AIOps uses evidence tools
|
|
- **WHEN** the AIOps flow calls available evidence tools
|
|
- **THEN** existing hooks and recorders persist agent steps and tool invocations under the resolved AIOps session id
|
|
|
|
### Requirement: AIOps payload prompts SHALL include a recommended knowledge query
|
|
When an AIOps request includes alert payload fields, the system SHALL include a deterministic recommended knowledge retrieval query in the prompt sent to the Agent flow.
|
|
|
|
#### Scenario: Payload-targeted prompt includes knowledge query
|
|
- **WHEN** an AIOps request contains alert name, service, severity, or description
|
|
- **THEN** the generated task prompt SHALL include a recommended `lookup_knowledge` query derived from the supplied payload fields
|
|
|
|
#### Scenario: Query skips blank fields
|
|
- **WHEN** some AIOps payload fields are blank
|
|
- **THEN** the recommended knowledge query SHALL omit those blank fields
|
|
- **AND** it SHALL preserve the non-blank alert-specific terms
|
|
|
|
#### Scenario: Auto-discovery prompt does not invent payload query
|
|
- **WHEN** an AIOps request does not include alert payload fields
|
|
- **THEN** the generated task prompt SHALL remain in auto-discovery mode
|
|
- **AND** it SHALL not include a payload-derived recommended knowledge query
|
|
|
|
### Requirement: AIOps sessions SHALL persist lightweight rule evaluation
|
|
When an AIOps final report is persisted, the system SHALL evaluate it with deterministic AIOps-specific quality rules and store the result in session self-evaluation.
|
|
|
|
#### Scenario: Payload-focused report is evaluated
|
|
- **WHEN** an AIOps session has alert payload fields and a final report is persisted
|
|
- **THEN** the system SHALL evaluate whether the report mentions the supplied alert and service
|
|
- **AND** it SHALL store the result under `self_evaluation.aiops_rule_evaluation`
|
|
|
|
#### Scenario: Evidence coverage is evaluated
|
|
- **WHEN** an AIOps final report is evaluated
|
|
- **THEN** the system SHALL check whether evidence tool invocations such as `lookup_knowledge`, `query_metrics`, or `query_logs` were persisted for the session
|
|
|
|
#### Scenario: Evaluation is traceable
|
|
- **WHEN** the diagnosis trace API returns an AIOps session
|
|
- **THEN** the session self-evaluation payload SHALL include `aiops_rule_evaluation` when it has been generated
|
|
|
|
#### Scenario: Evaluation uses stable verdicts
|
|
- **WHEN** AIOps rule evaluation completes
|
|
- **THEN** it SHALL produce a verdict from `PASS`, `WARN`, or `FAIL`
|
|
- **AND** it SHALL include check details and a human-readable rationale
|