feat: add aiops lightweight verifier

2026-07-05 13:44:30 +08:00
parent 2658742119
commit ed267d753d
15 changed files with 417 additions and 5 deletions
@@ -0,0 +1,26 @@
+## Why
+
+AIOps now has traceable payload scope control and improved RAG retrieval, but it still lacks a quality gate comparable to Chat's verifier. A lightweight rule-based verifier can check the most important AIOps risks without introducing another LLM agent.
+
+## What Changes
+
+- Add a rule-based AIOps evaluation service that checks final report quality after the AIOps flow completes.
+- Persist the evaluation under `diagnosis_session.self_evaluation.aiops_rule_evaluation`.
+- Evaluate payload focus, evidence-tool coverage, and basic report completeness.
+- Expose the evaluation through the existing trace API self-evaluation payload.
+
+## Capabilities
+
+### New Capabilities
+
+None.
+
+### Modified Capabilities
+
+- `aiops-traceable-diagnosis-entry`: AIOps sessions include a lightweight rule evaluation for trace replay.
+
+## Impact
+
+- Affects AIOps session finalization and trace self-evaluation.
+- Does not change AIOps API input, Agent flow topology, tool signatures, or database schema.
+- Does not add an LLM verifier agent.