docs(openspec): archive session run isolation
This commit is contained in:
zhuyongxin
@@ -0,0 +1,157 @@
|
||||
# Phase 5 Evidence: AIOps Run Isolation
|
||||
|
||||
## Scope
|
||||
|
||||
Phase 5 implements AIOps run isolation:
|
||||
|
||||
- `/api/ai_ops` allocates a `runId` before execution.
|
||||
- The SSE stream keeps event name `message` and emits a JSON `SseMessage` with `type=metadata`, `sessionId`, and `runId` before content.
|
||||
- `AiOpsService` creates `diagnosis_run` rows with `agent_flow=AI_OPS`.
|
||||
- AIOps Agent hooks and tool recording receive `sessionId + runId` through `RunnableConfig.metadata` and `SessionContextHolder`.
|
||||
- AIOps status, final report, metrics, and `self_evaluation.aiops_rule_evaluation` write to the current `diagnosis_run`.
|
||||
- Historical `DiagnosisSession` final-report persistence remains available only through the legacy overload.
|
||||
|
||||
## Focused Tests
|
||||
|
||||
Focused tests:
|
||||
|
||||
```text
|
||||
mvn -q "-Dtest=AiOpsServiceTest,ChatControllerTest,AgentLoggingHookTest,ToolInvocationRecorderTest,AiOpsRuleEvaluationServiceTest" test
|
||||
```
|
||||
|
||||
Result: passed.
|
||||
|
||||
Coverage:
|
||||
|
||||
- AIOps final report updates `diagnosis_run.answer` and `diagnosis_run.self_evaluation`.
|
||||
- Rule evaluation reads `tool_invocation` rows by `run_id`.
|
||||
- AIOps run metrics count `agent_step` and `tool_invocation` rows by `run_id`.
|
||||
- The same AIOps `sessionId` can create distinct `runId` values.
|
||||
- SSE metadata uses `type=metadata` and carries `sessionId/runId`.
|
||||
- Existing hook and tool-recorder tests cover `runId` propagation into `agent_step` and `tool_invocation`.
|
||||
|
||||
## E2E Runtime
|
||||
|
||||
Maven startup:
|
||||
|
||||
```text
|
||||
mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo
|
||||
```
|
||||
|
||||
Captured artifacts:
|
||||
|
||||
- `target/e2e/phase5-mvn-20260710-204732.out.log`
|
||||
- `target/e2e/phase5-mvn-20260710-204732.err.log`
|
||||
- `target/e2e/phase5-aiops-request.json`
|
||||
- `target/e2e/phase5-aiops-sse-response.txt`
|
||||
- `target/e2e/phase5-mvn-20260710-205204.out.log`
|
||||
- `target/e2e/phase5-mvn-20260710-205204.err.log`
|
||||
- `target/e2e/phase5-aiops-request-2053.json`
|
||||
- `target/e2e/phase5-aiops-sse-response-2053.txt`
|
||||
- `target/e2e/phase5-aiops-trace-2053.json`
|
||||
|
||||
The Maven process was stopped after evidence collection.
|
||||
|
||||
First E2E attempt exposed an existing AIOps runtime integration bug:
|
||||
|
||||
```text
|
||||
AI Ops 流程失败: mainAgent (ReactAgent) must be provided for supervisor agent
|
||||
```
|
||||
|
||||
Diagnosis result:
|
||||
|
||||
- feedback loop: fixed `/api/ai_ops` request with `mvp-demo` profile;
|
||||
- root cause: current `SupervisorAgent` dependency validates that `mainAgent(ReactAgent)` is set;
|
||||
- fix: `AiOpsService.buildSupervisorAgent(...)` now sets Planner as `mainAgent` and Executor as sub-agent;
|
||||
- regression coverage: `AiOpsServiceTest.buildSupervisorAgentSetsPlannerAsMainAgent`.
|
||||
|
||||
Successful E2E:
|
||||
|
||||
```text
|
||||
sessionId = e2e-phase5-aiops-codex-20260710-2053
|
||||
runId = run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
|
||||
```
|
||||
|
||||
SSE response:
|
||||
|
||||
```text
|
||||
event:message
|
||||
data:{"type":"metadata","data":null,"sessionId":"e2e-phase5-aiops-codex-20260710-2053","runId":"run-84b8c02b-4d1e-4b24-ac88-222c88c8db26"}
|
||||
|
||||
...
|
||||
|
||||
event:message
|
||||
data:{"type":"done","data":null,"sessionId":null,"runId":null}
|
||||
```
|
||||
|
||||
Exact trace check:
|
||||
|
||||
```text
|
||||
GET /api/diagnosis/e2e-phase5-aiops-codex-20260710-2053/trace?runId=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```text
|
||||
code=200
|
||||
runId=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
|
||||
agentFlow=AI_OPS
|
||||
summary.hasAiOpsRuleEvaluation=true
|
||||
summary.persistedStepCount=1
|
||||
summary.persistedToolCallCount=0
|
||||
```
|
||||
|
||||
The E2E model did not call evidence tools. This is captured as rule-evaluation WARN rather than a run-isolation failure.
|
||||
|
||||
## Database Inspection
|
||||
|
||||
Queried through `scripts/query_mysql.py`.
|
||||
|
||||
`diagnosis_run`:
|
||||
|
||||
```text
|
||||
run_id=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
|
||||
session_id=e2e-phase5-aiops-codex-20260710-2053
|
||||
status=SUCCESS
|
||||
agent_flow=AI_OPS
|
||||
has_answer=1
|
||||
has_aiops_eval=1
|
||||
step_count=1
|
||||
tool_call_count=0
|
||||
```
|
||||
|
||||
`agent_step` grouped by run:
|
||||
|
||||
```text
|
||||
run-84b8c02b-4d1e-4b24-ac88-222c88c8db26 | step_rows=1
|
||||
```
|
||||
|
||||
`tool_invocation` grouped by run:
|
||||
|
||||
```text
|
||||
(empty; the successful E2E did not invoke evidence tools)
|
||||
```
|
||||
|
||||
Log evidence from `phase5-mvn-20260710-205204.out.log`:
|
||||
|
||||
- AIOps request was received with the expected `sessionId` and `runId`.
|
||||
- `AiOpsService` started analysis and invoked the supervisor agent.
|
||||
- AIOps orchestration completed and final report extraction ran.
|
||||
- Exact trace was queried with the same `sessionId + runId`.
|
||||
|
||||
## Final Gate
|
||||
|
||||
Final Phase 5 gate:
|
||||
|
||||
```text
|
||||
mvn -q clean test-compile
|
||||
mvn -q "-Dtest=AiOpsServiceTest,ChatControllerTest,AgentLoggingHookTest,ToolInvocationRecorderTest,AiOpsRuleEvaluationServiceTest,DiagnosisTraceServiceTest" test
|
||||
openspec validate session-run-trace-isolation --strict
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Result: passed.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Phase 5 satisfies AIOps run creation, SSE run metadata, run-scoped execution context propagation, run-scoped final report/evaluation/metrics writes, focused tests, and Maven E2E DB/log verification.
|
||||
Reference in New Issue
Block a user