137 lines
4.9 KiB
Markdown
137 lines
4.9 KiB
Markdown
# Phase 3 Evidence: Trace Read Path and Run Listing
|
|
|
|
## Scope
|
|
|
|
Phase 3 implements run-scoped trace reads and lightweight run listing:
|
|
|
|
- `GET /api/diagnosis/{sessionId}/trace` resolves the latest run by `diagnosis_run.created_at DESC, id DESC`.
|
|
- `GET /api/diagnosis/{sessionId}/trace?runId=...` returns the exact run after validating run/session ownership.
|
|
- Trace responses include resolved run metadata and run-scoped step/tool rows.
|
|
- `GET /api/chat/session/{sessionId}/runs` returns lightweight run summaries without expanding trace detail rows.
|
|
|
|
## Verification Commands
|
|
|
|
- `mvn -q clean "-Dtest=DiagnosisTraceServiceTest" test`
|
|
- `mvn -q "-Dtest=DiagnosisTraceServiceTest,DiagnosisTraceEvaluatorTest,ChatControllerTest" test`
|
|
- `openspec validate session-run-trace-isolation --strict`
|
|
|
|
After the document review follow-up, OpenSpec strict validation was run again:
|
|
|
|
- `openspec validate session-run-trace-isolation --strict`
|
|
|
|
Result: passed.
|
|
|
|
## E2E Runtime
|
|
|
|
Maven startup:
|
|
|
|
```text
|
|
mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo
|
|
```
|
|
|
|
Captured logs and artifacts:
|
|
|
|
- `target/e2e/phase3-mvn-20260710-191131.out.log`
|
|
- `target/e2e/phase3-mvn-20260710-191131.err.log`
|
|
- `target/e2e/phase3-request-round1.json`
|
|
- `target/e2e/phase3-response-round1.json`
|
|
- `target/e2e/phase3-request-round2.json`
|
|
- `target/e2e/phase3-response-round2.json`
|
|
- `target/e2e/phase3-trace-latest.json`
|
|
- `target/e2e/phase3-trace-first.json`
|
|
- `target/e2e/phase3-trace-second.json`
|
|
- `target/e2e/phase3-runs.json`
|
|
- `target/e2e/phase3-trace-wrong-session.json`
|
|
- `target/e2e/phase3-summary.json`
|
|
|
|
The E2E Maven process was stopped after evidence collection.
|
|
|
|
Note: `logs/application.log` was checked, but it did not contain the Phase 3 E2E session entries and its last write time was earlier than this E2E run. The Phase 3 runtime application logs were captured in the Maven stdout artifact above.
|
|
|
|
## E2E Summary
|
|
|
|
Session:
|
|
|
|
```text
|
|
sessionId = e2e-phase3-codex-20260710-1912
|
|
run1 = run-fdccbe21-e050-4f92-a741-a062aef59644
|
|
run2 = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
|
```
|
|
|
|
Observed behavior from `phase3-summary.json`:
|
|
|
|
```text
|
|
distinctRunIds = true
|
|
latestResolvedRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
|
firstTraceRunId = run-fdccbe21-e050-4f92-a741-a062aef59644
|
|
firstSteps = 9
|
|
firstTools = 13
|
|
secondTraceRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
|
secondSteps = 2
|
|
secondTools = 0
|
|
runListCount = 2
|
|
runListFirst = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
|
wrongSessionStatus = 400
|
|
```
|
|
|
|
Log evidence from `phase3-mvn-20260710-191131.out.log`:
|
|
|
|
- Round 1 request was received for `e2e-phase3-codex-20260710-1912`.
|
|
- Redis session was created for the first round and reused by the second round.
|
|
- Message pair count reached `2` after round 2.
|
|
- Latest trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
|
|
- Exact first trace request returned `run-fdccbe21-e050-4f92-a741-a062aef59644`.
|
|
- Exact second trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
|
|
- Wrong-session exact trace returned HTTP 400 with `runId does not belong to sessionId`.
|
|
|
|
## Database Inspection
|
|
|
|
Queried through `scripts/query_mysql.py`.
|
|
|
|
`diagnosis_run` rows:
|
|
|
|
```text
|
|
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | SUCCESS | CHAT | step_count=2 | tool_call_count=0 | created_at=2026-07-10 19:15:40
|
|
run-fdccbe21-e050-4f92-a741-a062aef59644 | SUCCESS | CHAT | step_count=9 | tool_call_count=13 | created_at=2026-07-10 19:12:33
|
|
```
|
|
|
|
`agent_step` rows grouped by run:
|
|
|
|
```text
|
|
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | step_rows=2 | min_step=0 | max_step=1
|
|
run-fdccbe21-e050-4f92-a741-a062aef59644 | step_rows=9 | min_step=0 | max_step=4
|
|
```
|
|
|
|
`tool_invocation` rows grouped by run:
|
|
|
|
```text
|
|
run-fdccbe21-e050-4f92-a741-a062aef59644 | tool_rows=13
|
|
```
|
|
|
|
Missing run id checks for this session:
|
|
|
|
```text
|
|
agent_step missing run_id = 0
|
|
tool_invocation missing run_id = 0
|
|
```
|
|
|
|
`chat_session` metadata:
|
|
|
|
```text
|
|
session_id=e2e-phase3-codex-20260710-1912 | status=ACTIVE | message_pair_count=2
|
|
```
|
|
|
|
## Document Review Follow-up
|
|
|
|
Before closing Phase 3, the OpenSpec docs were tightened for upcoming phases:
|
|
|
|
- AIOps SSE metadata shape: keep SSE event name `message`, use JSON `SseMessage` with `type=metadata`, preserve existing content message shape.
|
|
- Feedback DTO contract: request `runId` is preferred; response includes bound `runId` and `fallbackToLatestRun`; wrong-session run binding fails instead of updating either run.
|
|
- Run-list API: returns `ApiResponse<List<RunSummary>>`, returns an empty list for an existing session with no runs, and uses existing missing-session error behavior when no session/run data exists.
|
|
|
|
OpenSpec strict validation passed after these document changes.
|
|
|
|
## Conclusion
|
|
|
|
Phase 3 satisfies the run-scoped trace read and run-list contract. Same-session multi-turn E2E proves latest-run compatibility, exact-run replay, run-list ordering, run/session ownership rejection, and no missing `run_id` rows for new trace data.
|