feat: add traceable scoped AIOps diagnosis

2026-07-04 22:57:28 +08:00
parent 246c99b954
commit 23ee05c7c3
32 changed files with 1179 additions and 25 deletions
@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-07-04
@@ -0,0 +1,54 @@
+## Context
+
+The AIOps endpoint has two natural modes:
+
+- **Payload mode**: caller supplies `alertName`, `service`, or other alert fields. The caller is asking for targeted diagnosis of that alert.
+- **Auto-discovery mode**: caller omits alert fields. The system should discover active alerts first, then analyze them.
+
+The current task prompt does not distinguish these modes, so the agent may query all active alerts and produce a broad report even when a specific alert payload was supplied.
+
+## Goals / Non-Goals
+
+**Goals:**
+
+- Make AIOps payload mode single-alert focused.
+- Keep no-payload mode compatible with the original "query active alerts then diagnose" behavior.
+- Keep the change prompt-only and low risk.
+- Add tests for prompt scope rules.
+
+**Non-Goals:**
+
+- Do not add a Verifier Agent.
+- Do not force tool calls in Java code.
+- Do not change `/api/ai_ops` request/response contracts.
+- Do not modify mock alert data.
+
+## Decisions
+
+| Decision | Choice | Alternative Considered | Rationale |
+|---|---|---|---|
+| Scope detection | Treat non-empty alert fields as payload mode | Add explicit `mode` field | Existing payload already carries enough intent; no API change needed. |
+| Payload mode behavior | Final report focuses only on supplied alert | Filter tool results in Java | Prompt-level rule is the smallest change and preserves agent flexibility. |
+| Auto mode behavior | Require active-alert discovery first | Always analyze only one alert | Original AIOps value is automated alert discovery when no payload exists. |
+| Other active alerts in payload mode | Mention only as related risk | Ignore entirely | Some context can be useful, but not enough to expand the report. |
+
+## Prompt Rules
+
+Payload mode MUST instruct the agent:
+
+- Treat supplied payload as the primary and only report target.
+- Use `queryPrometheusAlerts` only to verify the supplied alert state or identify related risk.
+- Do not create root-cause sections for unrelated active alerts.
+- Report unrelated alerts only in a brief "关联风险" note if they appear relevant.
+
+Auto-discovery mode MUST instruct the agent:
+
+- First call `queryPrometheusAlerts`.
+- Select P0/P1 or longest-running firing alerts.
+- Analyze one or more active alerts based on severity and evidence.
+
+## Risks / Trade-offs
+
+- [Risk] Prompt-only control may not be perfectly followed by the LLM. -> Mitigation: tests lock prompt wording; runtime can be reviewed through trace.
+- [Risk] Payload mode may miss broader incidents. -> Mitigation: related active alerts may be mentioned as risk, but not expanded into full sections.
+- [Risk] Future stronger enforcement may be needed. -> Mitigation: a later change can filter tool summaries or add AIOps Verifier.
@@ -0,0 +1,27 @@
+## Why
+
+Runtime verification showed that AIOps now correctly accepts an alert payload and persists a trace, but the generated report still expands to every active mock Prometheus alert. That weakens the product boundary between `/api/chat` and `/api/ai_ops`: an alert payload should mean targeted alert diagnosis, while an empty payload should mean automatic active-alert discovery.
+
+## What Changes
+
+- Tighten the AIOps task prompt so payload mode focuses the final report on the supplied alert.
+- Preserve full active-alert discovery when no payload is supplied.
+- Allow Prometheus active-alert lookup in payload mode only as supporting evidence, not as permission to expand the report to unrelated alerts.
+- Update tests and demo acceptance wording to lock the new behavior.
+
+## Capabilities
+
+### New Capabilities
+
+- `aiops-alert-scope-control`: Defines AIOps diagnosis scope rules for payload mode versus auto-discovery mode.
+
+### Modified Capabilities
+
+- `aiops-traceable-diagnosis-entry`: Keeps the same API and trace behavior but clarifies how AIOps should scope its diagnosis.
+
+## Impact
+
+- Affected code: `AiOpsService.buildTaskPrompt(...)`, focused tests, demo documentation, devflow records.
+- Affected API: no endpoint or request/response shape change.
+- Affected persistence: no schema change.
+- Non-goals: no Verifier integration, no tool implementation change, no prompt rewrite for Chat.
@@ -0,0 +1,25 @@
+## ADDED Requirements
+
+### Requirement: AIOps payload mode focuses on supplied alert
+When an AIOps request includes alert payload fields, the system SHALL instruct the agent to focus the final alert analysis report on the supplied alert.
+
+#### Scenario: Request includes alertName and service
+- **WHEN** a caller posts to `/api/ai_ops` with `alertName` and `service`
+- **THEN** the AIOps task prompt identifies payload mode
+- **AND** the prompt instructs the agent not to create full root-cause sections for unrelated active alerts
+
+### Requirement: AIOps auto-discovery mode queries active alerts first
+When an AIOps request omits alert payload fields, the system SHALL instruct the agent to first discover active Prometheus alerts.
+
+#### Scenario: Request body is omitted
+- **WHEN** a caller posts to `/api/ai_ops` without alert fields
+- **THEN** the AIOps task prompt identifies auto-discovery mode
+- **AND** the prompt instructs the agent to call `queryPrometheusAlerts` first
+
+### Requirement: Payload mode may use active alerts as supporting context
+Payload mode SHALL allow active-alert lookup as supporting evidence, but SHALL keep unrelated alerts out of the main report sections.
+
+#### Scenario: Prometheus returns multiple active alerts
+- **WHEN** payload mode is active and `queryPrometheusAlerts` returns unrelated active alerts
+- **THEN** the prompt permits mentioning those alerts only as related risk or context
+- **AND** the final report target remains the supplied alert
@@ -0,0 +1,21 @@
+## 1. Flow Records
+
+- [x] 1.1 Add devflow brief, decisions with lightweight Grill, evidence, and acceptance records.
+- [x] 1.2 Record local impact analysis and GitNexus skip context.
+
+## 2. Prompt Scope Control
+
+- [x] 2.1 Add payload detection helper in `AiOpsService`.
+- [x] 2.2 Update `buildTaskPrompt(...)` with payload-mode and auto-discovery-mode rules.
+
+## 3. Tests And Docs
+
+- [x] 3.1 Add tests for payload-mode prompt rules.
+- [x] 3.2 Add tests for no-payload auto-discovery prompt rules.
+- [x] 3.3 Update AIOps demo acceptance wording for single-alert payload mode.
+
+## 4. Verification
+
+- [x] 4.1 Run targeted tests.
+- [x] 4.2 Run compile verification.
+- [x] 4.3 Run OpenSpec validation.