docs: archive aiops lightweight verifier change
This commit is contained in:
@@ -59,3 +59,24 @@ When an AIOps request includes alert payload fields, the system SHALL include a
|
||||
- **WHEN** an AIOps request does not include alert payload fields
|
||||
- **THEN** the generated task prompt SHALL remain in auto-discovery mode
|
||||
- **AND** it SHALL not include a payload-derived recommended knowledge query
|
||||
|
||||
### Requirement: AIOps sessions SHALL persist lightweight rule evaluation
|
||||
When an AIOps final report is persisted, the system SHALL evaluate it with deterministic AIOps-specific quality rules and store the result in session self-evaluation.
|
||||
|
||||
#### Scenario: Payload-focused report is evaluated
|
||||
- **WHEN** an AIOps session has alert payload fields and a final report is persisted
|
||||
- **THEN** the system SHALL evaluate whether the report mentions the supplied alert and service
|
||||
- **AND** it SHALL store the result under `self_evaluation.aiops_rule_evaluation`
|
||||
|
||||
#### Scenario: Evidence coverage is evaluated
|
||||
- **WHEN** an AIOps final report is evaluated
|
||||
- **THEN** the system SHALL check whether evidence tool invocations such as `lookup_knowledge`, `query_metrics`, or `query_logs` were persisted for the session
|
||||
|
||||
#### Scenario: Evaluation is traceable
|
||||
- **WHEN** the diagnosis trace API returns an AIOps session
|
||||
- **THEN** the session self-evaluation payload SHALL include `aiops_rule_evaluation` when it has been generated
|
||||
|
||||
#### Scenario: Evaluation uses stable verdicts
|
||||
- **WHEN** AIOps rule evaluation completes
|
||||
- **THEN** it SHALL produce a verdict from `PASS`, `WARN`, or `FAIL`
|
||||
- **AND** it SHALL include check details and a human-readable rationale
|
||||
|
||||
Reference in New Issue
Block a user