SuperBizAgent-java/openspec/changes/session-run-trace-isolation/phase-5-evidence.md

Phase 5 Evidence: AIOps Run Isolation

Scope

Phase 5 implements AIOps run isolation:

• /api/ai_ops allocates a runId before execution.
• The SSE stream keeps event name message and emits a JSON SseMessage with type=metadata, sessionId, and runId before content.
• AiOpsService creates diagnosis_run rows with agent_flow=AI_OPS.
• AIOps Agent hooks and tool recording receive sessionId + runId through RunnableConfig.metadata and SessionContextHolder.
• AIOps status, final report, metrics, and self_evaluation.aiops_rule_evaluation write to the current diagnosis_run.
• Historical DiagnosisSession final-report persistence remains available only through the legacy overload.

Focused Tests

Focused tests:

mvn -q "-Dtest=AiOpsServiceTest,ChatControllerTest,AgentLoggingHookTest,ToolInvocationRecorderTest,AiOpsRuleEvaluationServiceTest" test

Result: passed.

Coverage:

• AIOps final report updates diagnosis_run.answer and diagnosis_run.self_evaluation.
• Rule evaluation reads tool_invocation rows by run_id.
• AIOps run metrics count agent_step and tool_invocation rows by run_id.
• The same AIOps sessionId can create distinct runId values.
• SSE metadata uses type=metadata and carries sessionId/runId.
• Existing hook and tool-recorder tests cover runId propagation into agent_step and tool_invocation.

E2E Runtime

Maven startup:

mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo

Captured artifacts:

• target/e2e/phase5-mvn-20260710-204732.out.log
• target/e2e/phase5-mvn-20260710-204732.err.log
• target/e2e/phase5-aiops-request.json
• target/e2e/phase5-aiops-sse-response.txt
• target/e2e/phase5-mvn-20260710-205204.out.log
• target/e2e/phase5-mvn-20260710-205204.err.log
• target/e2e/phase5-aiops-request-2053.json
• target/e2e/phase5-aiops-sse-response-2053.txt
• target/e2e/phase5-aiops-trace-2053.json

The Maven process was stopped after evidence collection.

First E2E attempt exposed an existing AIOps runtime integration bug:

AI Ops 流程失败: mainAgent (ReactAgent) must be provided for supervisor agent

Diagnosis result:

• feedback loop: fixed /api/ai_ops request with mvp-demo profile;
• root cause: current SupervisorAgent dependency validates that mainAgent(ReactAgent) is set;
• fix: AiOpsService.buildSupervisorAgent(...) now sets Planner as mainAgent and Executor as sub-agent;
• regression coverage: AiOpsServiceTest.buildSupervisorAgentSetsPlannerAsMainAgent.

Successful E2E:

sessionId = e2e-phase5-aiops-codex-20260710-2053
runId     = run-84b8c02b-4d1e-4b24-ac88-222c88c8db26

SSE response:

event:message
data:{"type":"metadata","data":null,"sessionId":"e2e-phase5-aiops-codex-20260710-2053","runId":"run-84b8c02b-4d1e-4b24-ac88-222c88c8db26"}

...

event:message
data:{"type":"done","data":null,"sessionId":null,"runId":null}

Exact trace check:

GET /api/diagnosis/e2e-phase5-aiops-codex-20260710-2053/trace?runId=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26

Observed:

code=200
runId=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
agentFlow=AI_OPS
summary.hasAiOpsRuleEvaluation=true
summary.persistedStepCount=1
summary.persistedToolCallCount=0

The E2E model did not call evidence tools. This is captured as rule-evaluation WARN rather than a run-isolation failure.

Database Inspection

Queried through scripts/query_mysql.py.

diagnosis_run:

run_id=run-84b8c02b-4d1e-4b24-ac88-222c88c8db26
session_id=e2e-phase5-aiops-codex-20260710-2053
status=SUCCESS
agent_flow=AI_OPS
has_answer=1
has_aiops_eval=1
step_count=1
tool_call_count=0

agent_step grouped by run:

run-84b8c02b-4d1e-4b24-ac88-222c88c8db26 | step_rows=1

tool_invocation grouped by run:

(empty; the successful E2E did not invoke evidence tools)

Log evidence from phase5-mvn-20260710-205204.out.log:

• AIOps request was received with the expected sessionId and runId.
• AiOpsService started analysis and invoked the supervisor agent.
• AIOps orchestration completed and final report extraction ran.
• Exact trace was queried with the same sessionId + runId.

Final Gate

Final Phase 5 gate:

mvn -q clean test-compile
mvn -q "-Dtest=AiOpsServiceTest,ChatControllerTest,AgentLoggingHookTest,ToolInvocationRecorderTest,AiOpsRuleEvaluationServiceTest,DiagnosisTraceServiceTest" test
openspec validate session-run-trace-isolation --strict
git diff --check

Result: passed.

Conclusion

Phase 5 satisfies AIOps run creation, SSE run metadata, run-scoped execution context propagation, run-scoped final report/evaluation/metrics writes, focused tests, and Maven E2E DB/log verification.