feat(trace): add run-scoped trace reads

This commit is contained in:
zhuyongxin
2026-07-10 20:07:23 +08:00
parent 26d5529280
commit 027aed1eeb
11 changed files with 620 additions and 111 deletions
@@ -143,6 +143,12 @@ Audit conclusions:
- Clarified that `chat_session.expires_at` is nullable directory metadata / best-effort TTL snapshot, not mandatory persisted conversation history. - Clarified that `chat_session.expires_at` is nullable directory metadata / best-effort TTL snapshot, not mandatory persisted conversation history.
- Clarified document review findings before continuing Phase 2: OpenSpec task phases are authoritative over the older active issue phase sketch, and AIOps rule evaluation is stored in `diagnosis_run.self_evaluation.aiops_rule_evaluation`, not a separate table. - Clarified document review findings before continuing Phase 2: OpenSpec task phases are authoritative over the older active issue phase sketch, and AIOps rule evaluation is stored in `diagnosis_run.self_evaluation.aiops_rule_evaluation`, not a separate table.
## Document Review Follow-up Before Phase 3 Gate
- Clarified AIOps SSE compatibility before Phase 5: keep SSE event name `message`, emit a JSON `SseMessage` with `type=metadata`, and preserve existing content message shape for report streaming.
- Clarified Feedback API before Phase 4: request `runId` is preferred, response always includes the bound `runId` and `fallbackToLatestRun`, and wrong-session run binding uses the existing failed feedback response path.
- Clarified run-list API before Phase 3 gate: `GET /api/chat/session/{sessionId}/runs` returns `ApiResponse<List<RunSummary>>`, returns an empty list for an existing session with no runs, and uses existing missing-session error behavior when no session/run data exists.
## Phase 2 Apply Notes ## Phase 2 Apply Notes
- Capability source: `openspec-apply-change` + sm-flow apply protocol. `codebase-retrieval` and LSP tools were not available in this session, so call-chain confirmation used OpenSpec context, `rg`, targeted file reads, compilation, focused tests, E2E, DB inspection, and logs. - Capability source: `openspec-apply-change` + sm-flow apply protocol. `codebase-retrieval` and LSP tools were not available in this session, so call-chain confirmation used OpenSpec context, `rg`, targeted file reads, compilation, focused tests, E2E, DB inspection, and logs.
@@ -151,3 +157,14 @@ Audit conclusions:
- Switched Chat run completion, failure, self-evaluation, metrics, verifier support reads, gatekeeper validation, and evidence scoring to run-scoped data. - Switched Chat run completion, failure, self-evaluation, metrics, verifier support reads, gatekeeper validation, and evidence scoring to run-scoped data.
- Added `/api/chat` response `runId` and focused tests for valid run creation, invalid request no-run behavior, run-scoped trace consumers, and same-session multi-turn run creation. - Added `/api/chat` response `runId` and focused tests for valid run creation, invalid request no-run behavior, run-scoped trace consumers, and same-session multi-turn run creation.
- Phase 2 gate evidence is recorded in `phase-2-evidence.md`. - Phase 2 gate evidence is recorded in `phase-2-evidence.md`.
## Phase 3 Apply Notes
- Capability source: `openspec-apply-change` + sm-flow apply protocol. The committed OpenSpec remained the execution source; a document review follow-up tightened DTO/SSE/listing contracts before the Phase 3 gate.
- Implemented latest-run trace resolution using `diagnosis_run.created_at DESC, id DESC`.
- Implemented exact trace lookup for `sessionId + runId` with run/session ownership validation.
- Changed trace details to read `agent_step` and `tool_invocation` by `run_id`, while retaining a legacy `diagnosis_session` fallback for historical compatibility.
- Added trace response fields for resolved `runId`, chat session metadata, run metadata, and per-row `runId`.
- Added lightweight `GET /api/chat/session/{sessionId}/runs` backed by `diagnosis_run` summaries.
- Added focused tests for latest trace, exact first trace, exact second trace, wrong-session rejection, missing session, read-only trace behavior, run listing, and legacy fallback.
- Phase 3 E2E used Maven startup with profile `mvp-demo`; evidence is recorded in `phase-3-evidence.md`.
@@ -98,10 +98,10 @@ Interface level: L4.
- Database contract changes: new tables, new columns, backfill, indexes, and later non-null expectations for new writes. - Database contract changes: new tables, new columns, backfill, indexes, and later non-null expectations for new writes.
- `/api/chat` response adds `runId`. - `/api/chat` response adds `runId`.
- `/api/ai_ops` SSE emits a compatible metadata message before report content. The metadata payload includes `sessionId` and `runId`; report content continues to stream through the existing content message shape. - `/api/ai_ops` SSE emits a compatible metadata message before report content. It keeps the existing SSE event name `message` and sends a JSON `SseMessage` with `type=metadata`; the metadata payload includes `sessionId` and `runId`. Report content continues to stream through the existing `type=content` message shape.
- Trace API accepts optional `runId`. - Trace API accepts optional `runId`.
- Feedback request accepts preferred `runId` and returns fallback binding metadata when omitted. - Feedback request accepts preferred `runId`. Feedback response includes the actual bound `runId` and `fallbackToLatestRun`; wrong-session `runId`, missing session, and missing run use the existing failed feedback response path with HTTP 400 from `FeedbackController`.
- New run summary API: `GET /api/chat/session/{sessionId}/runs`. - New run summary API: `GET /api/chat/session/{sessionId}/runs`, returned through the existing `ApiResponse<List<RunSummary>>` wrapper. A session with metadata but no runs returns an empty list; a missing session returns the existing not-found/error behavior.
Compatibility: Compatibility:
@@ -0,0 +1,136 @@
# Phase 3 Evidence: Trace Read Path and Run Listing
## Scope
Phase 3 implements run-scoped trace reads and lightweight run listing:
- `GET /api/diagnosis/{sessionId}/trace` resolves the latest run by `diagnosis_run.created_at DESC, id DESC`.
- `GET /api/diagnosis/{sessionId}/trace?runId=...` returns the exact run after validating run/session ownership.
- Trace responses include resolved run metadata and run-scoped step/tool rows.
- `GET /api/chat/session/{sessionId}/runs` returns lightweight run summaries without expanding trace detail rows.
## Verification Commands
- `mvn -q clean "-Dtest=DiagnosisTraceServiceTest" test`
- `mvn -q "-Dtest=DiagnosisTraceServiceTest,DiagnosisTraceEvaluatorTest,ChatControllerTest" test`
- `openspec validate session-run-trace-isolation --strict`
After the document review follow-up, OpenSpec strict validation was run again:
- `openspec validate session-run-trace-isolation --strict`
Result: passed.
## E2E Runtime
Maven startup:
```text
mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo
```
Captured logs and artifacts:
- `target/e2e/phase3-mvn-20260710-191131.out.log`
- `target/e2e/phase3-mvn-20260710-191131.err.log`
- `target/e2e/phase3-request-round1.json`
- `target/e2e/phase3-response-round1.json`
- `target/e2e/phase3-request-round2.json`
- `target/e2e/phase3-response-round2.json`
- `target/e2e/phase3-trace-latest.json`
- `target/e2e/phase3-trace-first.json`
- `target/e2e/phase3-trace-second.json`
- `target/e2e/phase3-runs.json`
- `target/e2e/phase3-trace-wrong-session.json`
- `target/e2e/phase3-summary.json`
The E2E Maven process was stopped after evidence collection.
Note: `logs/application.log` was checked, but it did not contain the Phase 3 E2E session entries and its last write time was earlier than this E2E run. The Phase 3 runtime application logs were captured in the Maven stdout artifact above.
## E2E Summary
Session:
```text
sessionId = e2e-phase3-codex-20260710-1912
run1 = run-fdccbe21-e050-4f92-a741-a062aef59644
run2 = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
```
Observed behavior from `phase3-summary.json`:
```text
distinctRunIds = true
latestResolvedRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
firstTraceRunId = run-fdccbe21-e050-4f92-a741-a062aef59644
firstSteps = 9
firstTools = 13
secondTraceRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
secondSteps = 2
secondTools = 0
runListCount = 2
runListFirst = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
wrongSessionStatus = 400
```
Log evidence from `phase3-mvn-20260710-191131.out.log`:
- Round 1 request was received for `e2e-phase3-codex-20260710-1912`.
- Redis session was created for the first round and reused by the second round.
- Message pair count reached `2` after round 2.
- Latest trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
- Exact first trace request returned `run-fdccbe21-e050-4f92-a741-a062aef59644`.
- Exact second trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
- Wrong-session exact trace returned HTTP 400 with `runId does not belong to sessionId`.
## Database Inspection
Queried through `scripts/query_mysql.py`.
`diagnosis_run` rows:
```text
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | SUCCESS | CHAT | step_count=2 | tool_call_count=0 | created_at=2026-07-10 19:15:40
run-fdccbe21-e050-4f92-a741-a062aef59644 | SUCCESS | CHAT | step_count=9 | tool_call_count=13 | created_at=2026-07-10 19:12:33
```
`agent_step` rows grouped by run:
```text
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | step_rows=2 | min_step=0 | max_step=1
run-fdccbe21-e050-4f92-a741-a062aef59644 | step_rows=9 | min_step=0 | max_step=4
```
`tool_invocation` rows grouped by run:
```text
run-fdccbe21-e050-4f92-a741-a062aef59644 | tool_rows=13
```
Missing run id checks for this session:
```text
agent_step missing run_id = 0
tool_invocation missing run_id = 0
```
`chat_session` metadata:
```text
session_id=e2e-phase3-codex-20260710-1912 | status=ACTIVE | message_pair_count=2
```
## Document Review Follow-up
Before closing Phase 3, the OpenSpec docs were tightened for upcoming phases:
- AIOps SSE metadata shape: keep SSE event name `message`, use JSON `SseMessage` with `type=metadata`, preserve existing content message shape.
- Feedback DTO contract: request `runId` is preferred; response includes bound `runId` and `fallbackToLatestRun`; wrong-session run binding fails instead of updating either run.
- Run-list API: returns `ApiResponse<List<RunSummary>>`, returns an empty list for an existing session with no runs, and uses existing missing-session error behavior when no session/run data exists.
OpenSpec strict validation passed after these document changes.
## Conclusion
Phase 3 satisfies the run-scoped trace read and run-list contract. Same-session multi-turn E2E proves latest-run compatibility, exact-run replay, run-list ordering, run/session ownership rejection, and no missing `run_id` rows for new trace data.
@@ -72,6 +72,8 @@ Level: L4 database/API contract migration with compatibility behavior.
- New query parameter: `GET /api/diagnosis/{sessionId}/trace?runId=...`. - New query parameter: `GET /api/diagnosis/{sessionId}/trace?runId=...`.
- New API: `GET /api/chat/session/{sessionId}/runs`. - New API: `GET /api/chat/session/{sessionId}/runs`.
- Feedback request gains optional/preferred `runId`. - Feedback request gains optional/preferred `runId`.
- Feedback response returns bound `runId` and `fallbackToLatestRun`.
- `/api/ai_ops` keeps SSE event name `message` and emits a `type=metadata` JSON message containing `sessionId` and `runId` before content.
- Database contract changes include new tables and new `run_id` columns. - Database contract changes include new tables and new `run_id` columns.
- Old callers that only pass `sessionId` remain compatible by binding to latest run, but this fallback must be observable. - Old callers that only pass `sessionId` remain compatible by binding to latest run, but this fallback must be observable.
@@ -72,10 +72,18 @@ The system SHALL provide a lightweight run-list API for a Chat Session.
#### Scenario: Run list returns summaries #### Scenario: Run list returns summaries
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs` - **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs`
- **THEN** the system SHALL return run summaries from `diagnosis_run` - **THEN** the system SHALL return run summaries from `diagnosis_run` inside the existing API response wrapper
- **AND** each summary SHALL include `runId`, `sessionId`, `query`, `status`, `agentFlow`, `answerPreview`, `stepCount`, `toolCallCount`, `createdAt`, and `updatedAt` - **AND** each summary SHALL include `runId`, `sessionId`, `query`, `status`, `agentFlow`, `answerPreview`, `stepCount`, `toolCallCount`, `createdAt`, and `updatedAt`
- **AND** the response SHALL NOT expand `agent_step` or `tool_invocation` detail rows - **AND** the response SHALL NOT expand `agent_step` or `tool_invocation` detail rows
#### Scenario: Run list handles session without runs
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs` for an existing `chat_session` with no runs
- **THEN** the system SHALL return a successful empty list
#### Scenario: Run list rejects missing session
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs` for a session that does not exist in `chat_session` or `diagnosis_run`
- **THEN** the system SHALL use the existing not-found/error response behavior
### Requirement: Feedback SHALL bind to diagnosis runs ### Requirement: Feedback SHALL bind to diagnosis runs
The system SHALL bind new feedback to a diagnosis run rather than an ambiguous multi-turn session. The system SHALL bind new feedback to a diagnosis run rather than an ambiguous multi-turn session.
@@ -83,6 +91,8 @@ The system SHALL bind new feedback to a diagnosis run rather than an ambiguous m
- **WHEN** a feedback request includes `sessionId` and `runId` - **WHEN** a feedback request includes `sessionId` and `runId`
- **THEN** the system SHALL validate that the run belongs to the session - **THEN** the system SHALL validate that the run belongs to the session
- **AND** it SHALL update feedback on that run - **AND** it SHALL update feedback on that run
- **AND** the response SHALL include the actual bound `runId`
- **AND** the response SHALL include `fallbackToLatestRun=false`
#### Scenario: Feedback without runId falls back observably #### Scenario: Feedback without runId falls back observably
- **WHEN** a legacy feedback request includes `sessionId` but omits `runId` - **WHEN** a legacy feedback request includes `sessionId` but omits `runId`
@@ -90,6 +100,10 @@ The system SHALL bind new feedback to a diagnosis run rather than an ambiguous m
- **AND** the response SHALL include `fallbackToLatestRun=true` - **AND** the response SHALL include `fallbackToLatestRun=true`
- **AND** the response SHALL include the actual bound `runId` - **AND** the response SHALL include the actual bound `runId`
#### Scenario: Feedback rejects run from another session
- **WHEN** a feedback request includes a `runId` that belongs to a different `sessionId`
- **THEN** the system SHALL return a failed feedback response instead of updating either run
#### Scenario: Useful feedback creates case from run #### Scenario: Useful feedback creates case from run
- **WHEN** feedback for a run is `useful` - **WHEN** feedback for a run is `useful`
- **THEN** the system SHALL create or reuse a `case_library` row using that run's query and answer - **THEN** the system SHALL create or reuse a `case_library` row using that run's query and answer
@@ -107,8 +121,11 @@ The system SHALL create and expose a diagnosis run for every valid `/api/ai_ops`
#### Scenario: AIOps SSE exposes runId #### Scenario: AIOps SSE exposes runId
- **WHEN** `/api/ai_ops` streams response metadata to the caller - **WHEN** `/api/ai_ops` streams response metadata to the caller
- **THEN** the stream SHALL send a compatible metadata message before report content - **THEN** the stream SHALL send a compatible metadata message before report content
- **AND** the SSE event name SHALL remain `message`
- **AND** the message type SHALL be `metadata`
- **AND** the metadata payload SHALL expose the resolved `sessionId` - **AND** the metadata payload SHALL expose the resolved `sessionId`
- **AND** the metadata payload SHALL expose the created `runId` - **AND** the metadata payload SHALL expose the created `runId`
- **AND** report content SHALL continue to use the existing content message shape
### Requirement: Migration SHALL preserve historical trace access ### Requirement: Migration SHALL preserve historical trace access
The system SHALL migrate historical diagnosis data into compatibility runs without deleting the old `diagnosis_session` table. The system SHALL migrate historical diagnosis data into compatibility runs without deleting the old `diagnosis_session` table.
@@ -22,13 +22,13 @@
## 3. Trace Read Path and Run Listing ## 3. Trace Read Path and Run Listing
- [ ] 3.1 Change `DiagnosisTraceService` to resolve latest run by `diagnosis_run.created_at DESC, id DESC` when `runId` is omitted. - [x] 3.1 Change `DiagnosisTraceService` to resolve latest run by `diagnosis_run.created_at DESC, id DESC` when `runId` is omitted.
- [ ] 3.2 Add exact trace lookup for `sessionId + runId`, including validation that the run belongs to the session. - [x] 3.2 Add exact trace lookup for `sessionId + runId`, including validation that the run belongs to the session.
- [ ] 3.3 Change trace response DTOs to include resolved `runId` and run summary fields. - [x] 3.3 Change trace response DTOs to include resolved `runId` and run summary fields.
- [ ] 3.4 Add `GET /api/chat/session/{sessionId}/runs` returning lightweight run summaries without expanding trace detail rows. - [x] 3.4 Add `GET /api/chat/session/{sessionId}/runs` returning lightweight run summaries without expanding trace detail rows.
- [ ] 3.5 Preserve read-only trace behavior for latest-run and exact-run requests. - [x] 3.5 Preserve read-only trace behavior for latest-run and exact-run requests.
- [ ] 3.6 Add tests for latest run, exact first run, exact second run, wrong-session run rejection, missing session, and read-only behavior. - [x] 3.6 Add tests for latest run, exact first run, exact second run, wrong-session run rejection, missing session, and read-only behavior.
- [ ] 3.7 Phase 3 gate: run focused trace tests and same-session E2E trace checks, update task status, archive phase evidence, and commit before starting Phase 4. - [x] 3.7 Phase 3 gate: run focused trace tests and same-session E2E trace checks, update task status, archive phase evidence, and commit before starting Phase 4.
## 4. Feedback and Case Library Run Binding ## 4. Feedback and Case Library Run Binding
@@ -5,8 +5,10 @@ import lombok.Getter;
import lombok.Setter; import lombok.Setter;
import com.superbiz.agent.domain.model.SessionContext; import com.superbiz.agent.domain.model.SessionContext;
import com.superbiz.agent.dto.AIOpsRequest; import com.superbiz.agent.dto.AIOpsRequest;
import com.superbiz.agent.dto.DiagnosisTraceResponse;
import com.superbiz.agent.service.AiOpsService; import com.superbiz.agent.service.AiOpsService;
import com.superbiz.agent.service.ChatService; import com.superbiz.agent.service.ChatService;
import com.superbiz.agent.service.DiagnosisTraceService;
import com.superbiz.agent.service.session.SessionManager; import com.superbiz.agent.service.session.SessionManager;
import org.slf4j.Logger; import org.slf4j.Logger;
import org.slf4j.LoggerFactory; import org.slf4j.LoggerFactory;
@@ -43,6 +45,9 @@ public class ChatController {
@Autowired @Autowired
private ChatService chatService; private ChatService chatService;
@Autowired
private DiagnosisTraceService diagnosisTraceService;
@Autowired @Autowired
private SessionManager sessionManager; private SessionManager sessionManager;
@@ -325,6 +330,12 @@ public class ChatController {
} }
} }
@GetMapping("/chat/session/{sessionId}/runs")
public ResponseEntity<ApiResponse<List<DiagnosisTraceResponse.RunSummary>>> listSessionRuns(
@PathVariable String sessionId) {
return ResponseEntity.ok(ApiResponse.success(diagnosisTraceService.listRunSummaries(sessionId)));
}
// ==================== 辅助方法 ==================== // ==================== 辅助方法 ====================
private SessionContext getOrCreateSession(String sessionId) { private SessionContext getOrCreateSession(String sessionId) {
@@ -7,6 +7,7 @@ import lombok.RequiredArgsConstructor;
import org.springframework.http.ResponseEntity; import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable; import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController; import org.springframework.web.bind.annotation.RestController;
@@ -18,7 +19,10 @@ public class DiagnosisTraceController {
private final DiagnosisTraceService diagnosisTraceService; private final DiagnosisTraceService diagnosisTraceService;
@GetMapping("/{sessionId}/trace") @GetMapping("/{sessionId}/trace")
public ResponseEntity<Result<DiagnosisTraceResponse>> getTrace(@PathVariable String sessionId) { public ResponseEntity<Result<DiagnosisTraceResponse>> getTrace(
return ResponseEntity.ok(Result.success(diagnosisTraceService.getTrace(sessionId))); @PathVariable String sessionId,
@RequestParam(required = false) String runId
) {
return ResponseEntity.ok(Result.success(diagnosisTraceService.getTrace(sessionId, runId)));
} }
} }
@@ -15,11 +15,28 @@ import java.util.Map;
@AllArgsConstructor @AllArgsConstructor
public class DiagnosisTraceResponse { public class DiagnosisTraceResponse {
private String runId;
private ChatSessionTrace chatSession;
private SessionTrace session; private SessionTrace session;
private RunTrace run;
private List<AgentStepTrace> steps; private List<AgentStepTrace> steps;
private List<ToolInvocationTrace> toolInvocations; private List<ToolInvocationTrace> toolInvocations;
private TraceSummary summary; private TraceSummary summary;
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class ChatSessionTrace {
private Long id;
private String sessionId;
private String status;
private Integer messagePairCount;
private LocalDateTime createdAt;
private LocalDateTime lastActiveAt;
private LocalDateTime expiresAt;
}
@Data @Data
@Builder @Builder
@NoArgsConstructor @NoArgsConstructor
@@ -42,6 +59,29 @@ public class DiagnosisTraceResponse {
private LocalDateTime updatedAt; private LocalDateTime updatedAt;
} }
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class RunTrace {
private Long id;
private String runId;
private String sessionId;
private String query;
private String status;
private String agentFlow;
private Integer totalDurationMs;
private Integer totalTokenCount;
private Integer stepCount;
private Integer toolCallCount;
private String answer;
private String selfEvaluationRaw;
private Map<String, Object> selfEvaluation;
private String feedback;
private LocalDateTime createdAt;
private LocalDateTime updatedAt;
}
@Data @Data
@Builder @Builder
@NoArgsConstructor @NoArgsConstructor
@@ -49,6 +89,7 @@ public class DiagnosisTraceResponse {
public static class AgentStepTrace { public static class AgentStepTrace {
private Long id; private Long id;
private String sessionId; private String sessionId;
private String runId;
private Integer stepIndex; private Integer stepIndex;
private String agentName; private String agentName;
private String modelInput; private String modelInput;
@@ -67,6 +108,7 @@ public class DiagnosisTraceResponse {
public static class ToolInvocationTrace { public static class ToolInvocationTrace {
private Long id; private Long id;
private String sessionId; private String sessionId;
private String runId;
private Long stepId; private Long stepId;
private String toolName; private String toolName;
private String inputParamsRaw; private String inputParamsRaw;
@@ -92,6 +134,7 @@ public class DiagnosisTraceResponse {
@NoArgsConstructor @NoArgsConstructor
@AllArgsConstructor @AllArgsConstructor
public static class TraceSummary { public static class TraceSummary {
private String resolvedRunId;
private int persistedStepCount; private int persistedStepCount;
private int returnedStepCount; private int returnedStepCount;
private int persistedToolCallCount; private int persistedToolCallCount;
@@ -100,4 +143,21 @@ public class DiagnosisTraceResponse {
private boolean hasAiOpsRuleEvaluation; private boolean hasAiOpsRuleEvaluation;
private boolean hasFeedback; private boolean hasFeedback;
} }
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public static class RunSummary {
private String runId;
private String sessionId;
private String query;
private String status;
private String agentFlow;
private String answerPreview;
private Integer stepCount;
private Integer toolCallCount;
private LocalDateTime createdAt;
private LocalDateTime updatedAt;
}
} }
@@ -3,11 +3,15 @@ package com.superbiz.agent.service;
import com.fasterxml.jackson.core.type.TypeReference; import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import com.superbiz.agent.domain.entity.AgentStep; import com.superbiz.agent.domain.entity.AgentStep;
import com.superbiz.agent.domain.entity.ChatSession;
import com.superbiz.agent.domain.entity.DiagnosisRun;
import com.superbiz.agent.domain.entity.DiagnosisSession; import com.superbiz.agent.domain.entity.DiagnosisSession;
import com.superbiz.agent.domain.entity.ToolInvocation; import com.superbiz.agent.domain.entity.ToolInvocation;
import com.superbiz.agent.dto.DiagnosisTraceResponse; import com.superbiz.agent.dto.DiagnosisTraceResponse;
import com.superbiz.agent.exception.SessionNotFoundException; import com.superbiz.agent.exception.SessionNotFoundException;
import com.superbiz.agent.repository.AgentStepRepository; import com.superbiz.agent.repository.AgentStepRepository;
import com.superbiz.agent.repository.ChatSessionRepository;
import com.superbiz.agent.repository.DiagnosisRunRepository;
import com.superbiz.agent.repository.DiagnosisSessionRepository; import com.superbiz.agent.repository.DiagnosisSessionRepository;
import com.superbiz.agent.repository.ToolInvocationRepository; import com.superbiz.agent.repository.ToolInvocationRepository;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -25,18 +29,76 @@ public class DiagnosisTraceService {
}; };
private final DiagnosisSessionRepository diagnosisSessionRepository; private final DiagnosisSessionRepository diagnosisSessionRepository;
private final ChatSessionRepository chatSessionRepository;
private final DiagnosisRunRepository diagnosisRunRepository;
private final AgentStepRepository agentStepRepository; private final AgentStepRepository agentStepRepository;
private final ToolInvocationRepository toolInvocationRepository; private final ToolInvocationRepository toolInvocationRepository;
private final ObjectMapper objectMapper; private final ObjectMapper objectMapper;
public DiagnosisTraceResponse getTrace(String sessionId) { public DiagnosisTraceResponse getTrace(String sessionId) {
return getTrace(sessionId, null);
}
public DiagnosisTraceResponse getTrace(String sessionId, String runId) {
if (runId != null && !runId.isBlank()) {
DiagnosisRun run = diagnosisRunRepository.findBySessionIdAndRunId(sessionId, runId)
.orElseThrow(() -> buildRunLookupException(sessionId, runId));
return buildRunTraceResponse(sessionId, run);
}
return diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId)
.map(run -> buildRunTraceResponse(sessionId, run))
.orElseGet(() -> buildLegacyTraceResponse(sessionId));
}
public List<DiagnosisTraceResponse.RunSummary> listRunSummaries(String sessionId) {
List<DiagnosisRun> runs = diagnosisRunRepository.findBySessionIdOrderByCreatedAtDescIdDesc(sessionId);
if (runs.isEmpty() && chatSessionRepository.findBySessionId(sessionId).isEmpty()) {
throw new SessionNotFoundException(sessionId);
}
return runs.stream()
.map(this::toRunSummary)
.toList();
}
private RuntimeException buildRunLookupException(String sessionId, String runId) {
if (diagnosisRunRepository.findByRunId(runId).isPresent()) {
return new IllegalArgumentException("runId does not belong to sessionId: " + runId);
}
return new SessionNotFoundException(sessionId, "Run not found: " + runId);
}
private DiagnosisTraceResponse buildRunTraceResponse(String sessionId, DiagnosisRun run) {
List<AgentStep> steps = orderStepsForTrace(agentStepRepository.findByRunIdOrderByStepIndex(run.getRunId()));
List<ToolInvocation> toolInvocations = toolInvocationRepository.findByRunIdOrderByIdAsc(run.getRunId());
DiagnosisTraceResponse.RunTrace runTrace = toRunTrace(run);
return DiagnosisTraceResponse.builder()
.runId(run.getRunId())
.chatSession(chatSessionRepository.findBySessionId(sessionId)
.map(this::toChatSessionTrace)
.orElse(null))
.session(toSessionTrace(runTrace))
.run(runTrace)
.steps(steps.stream().map(this::toAgentStepTrace).toList())
.toolInvocations(toolInvocations.stream().map(this::toToolInvocationTrace).toList())
.summary(toSummary(runTrace, steps, toolInvocations))
.build();
}
private DiagnosisTraceResponse buildLegacyTraceResponse(String sessionId) {
DiagnosisSession session = diagnosisSessionRepository.findBySessionId(sessionId) DiagnosisSession session = diagnosisSessionRepository.findBySessionId(sessionId)
.orElseThrow(() -> new SessionNotFoundException(sessionId)); .orElseThrow(() -> new SessionNotFoundException(sessionId));
List<AgentStep> steps = orderStepsForTrace(agentStepRepository.findBySessionId(sessionId)); List<AgentStep> steps = orderStepsForTrace(agentStepRepository.findBySessionId(sessionId));
List<ToolInvocation> toolInvocations = toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId); List<ToolInvocation> toolInvocations = toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId);
return DiagnosisTraceResponse.builder() return DiagnosisTraceResponse.builder()
.runId(null)
.chatSession(chatSessionRepository.findBySessionId(sessionId)
.map(this::toChatSessionTrace)
.orElse(null))
.session(toSessionTrace(session)) .session(toSessionTrace(session))
.run(null)
.steps(steps.stream().map(this::toAgentStepTrace).toList()) .steps(steps.stream().map(this::toAgentStepTrace).toList())
.toolInvocations(toolInvocations.stream().map(this::toToolInvocationTrace).toList()) .toolInvocations(toolInvocations.stream().map(this::toToolInvocationTrace).toList())
.summary(toSummary(session, steps, toolInvocations)) .summary(toSummary(session, steps, toolInvocations))
@@ -52,6 +114,18 @@ public class DiagnosisTraceService {
.toList(); .toList();
} }
private DiagnosisTraceResponse.ChatSessionTrace toChatSessionTrace(ChatSession chatSession) {
return DiagnosisTraceResponse.ChatSessionTrace.builder()
.id(chatSession.getId())
.sessionId(chatSession.getSessionId())
.status(chatSession.getStatus())
.messagePairCount(chatSession.getMessagePairCount())
.createdAt(chatSession.getCreatedAt())
.lastActiveAt(chatSession.getLastActiveAt())
.expiresAt(chatSession.getExpiresAt())
.build();
}
private DiagnosisTraceResponse.SessionTrace toSessionTrace(DiagnosisSession session) { private DiagnosisTraceResponse.SessionTrace toSessionTrace(DiagnosisSession session) {
return DiagnosisTraceResponse.SessionTrace.builder() return DiagnosisTraceResponse.SessionTrace.builder()
.id(session.getId()) .id(session.getId())
@@ -72,10 +146,52 @@ public class DiagnosisTraceService {
.build(); .build();
} }
private DiagnosisTraceResponse.SessionTrace toSessionTrace(DiagnosisTraceResponse.RunTrace run) {
return DiagnosisTraceResponse.SessionTrace.builder()
.id(run.getId())
.sessionId(run.getSessionId())
.query(run.getQuery())
.status(run.getStatus())
.agentFlow(run.getAgentFlow())
.totalDurationMs(run.getTotalDurationMs())
.totalTokenCount(run.getTotalTokenCount())
.stepCount(run.getStepCount())
.toolCallCount(run.getToolCallCount())
.answer(run.getAnswer())
.selfEvaluationRaw(run.getSelfEvaluationRaw())
.selfEvaluation(run.getSelfEvaluation())
.feedback(run.getFeedback())
.createdAt(run.getCreatedAt())
.updatedAt(run.getUpdatedAt())
.build();
}
private DiagnosisTraceResponse.RunTrace toRunTrace(DiagnosisRun run) {
return DiagnosisTraceResponse.RunTrace.builder()
.id(run.getId())
.runId(run.getRunId())
.sessionId(run.getSessionId())
.query(run.getQuery())
.status(run.getStatus())
.agentFlow(run.getAgentFlow())
.totalDurationMs(run.getTotalDurationMs())
.totalTokenCount(run.getTotalTokenCount())
.stepCount(run.getStepCount())
.toolCallCount(run.getToolCallCount())
.answer(run.getAnswer())
.selfEvaluationRaw(run.getSelfEvaluation())
.selfEvaluation(parseJsonObject(run.getSelfEvaluation()))
.feedback(run.getFeedback())
.createdAt(run.getCreatedAt())
.updatedAt(run.getUpdatedAt())
.build();
}
private DiagnosisTraceResponse.AgentStepTrace toAgentStepTrace(AgentStep step) { private DiagnosisTraceResponse.AgentStepTrace toAgentStepTrace(AgentStep step) {
return DiagnosisTraceResponse.AgentStepTrace.builder() return DiagnosisTraceResponse.AgentStepTrace.builder()
.id(step.getId()) .id(step.getId())
.sessionId(step.getSessionId()) .sessionId(step.getSessionId())
.runId(step.getRunId())
.stepIndex(step.getStepIndex()) .stepIndex(step.getStepIndex())
.agentName(step.getAgentName()) .agentName(step.getAgentName())
.modelInput(step.getModelInput()) .modelInput(step.getModelInput())
@@ -92,6 +208,7 @@ public class DiagnosisTraceService {
return DiagnosisTraceResponse.ToolInvocationTrace.builder() return DiagnosisTraceResponse.ToolInvocationTrace.builder()
.id(invocation.getId()) .id(invocation.getId())
.sessionId(invocation.getSessionId()) .sessionId(invocation.getSessionId())
.runId(invocation.getRunId())
.stepId(invocation.getStepId()) .stepId(invocation.getStepId())
.toolName(invocation.getToolName()) .toolName(invocation.getToolName())
.inputParamsRaw(invocation.getInputParams()) .inputParamsRaw(invocation.getInputParams())
@@ -113,6 +230,21 @@ public class DiagnosisTraceService {
.build(); .build();
} }
private DiagnosisTraceResponse.RunSummary toRunSummary(DiagnosisRun run) {
return DiagnosisTraceResponse.RunSummary.builder()
.runId(run.getRunId())
.sessionId(run.getSessionId())
.query(run.getQuery())
.status(run.getStatus())
.agentFlow(run.getAgentFlow())
.answerPreview(preview(run.getAnswer(), 160))
.stepCount(run.getStepCount())
.toolCallCount(run.getToolCallCount())
.createdAt(run.getCreatedAt())
.updatedAt(run.getUpdatedAt())
.build();
}
private DiagnosisTraceResponse.TraceSummary toSummary( private DiagnosisTraceResponse.TraceSummary toSummary(
DiagnosisSession session, DiagnosisSession session,
List<AgentStep> steps, List<AgentStep> steps,
@@ -120,6 +252,7 @@ public class DiagnosisTraceService {
) { ) {
Map<String, Object> selfEvaluation = parseJsonObject(session.getSelfEvaluation()); Map<String, Object> selfEvaluation = parseJsonObject(session.getSelfEvaluation());
return DiagnosisTraceResponse.TraceSummary.builder() return DiagnosisTraceResponse.TraceSummary.builder()
.resolvedRunId(null)
.persistedStepCount(defaultInt(session.getStepCount())) .persistedStepCount(defaultInt(session.getStepCount()))
.returnedStepCount(steps.size()) .returnedStepCount(steps.size())
.persistedToolCallCount(defaultInt(session.getToolCallCount())) .persistedToolCallCount(defaultInt(session.getToolCallCount()))
@@ -130,6 +263,24 @@ public class DiagnosisTraceService {
.build(); .build();
} }
private DiagnosisTraceResponse.TraceSummary toSummary(
DiagnosisTraceResponse.RunTrace run,
List<AgentStep> steps,
List<ToolInvocation> toolInvocations
) {
Map<String, Object> selfEvaluation = run.getSelfEvaluation();
return DiagnosisTraceResponse.TraceSummary.builder()
.resolvedRunId(run.getRunId())
.persistedStepCount(defaultInt(run.getStepCount()))
.returnedStepCount(steps.size())
.persistedToolCallCount(defaultInt(run.getToolCallCount()))
.returnedToolCallCount(toolInvocations.size())
.hasVerifierEvaluation(selfEvaluation != null && selfEvaluation.containsKey("verifier_evaluation"))
.hasAiOpsRuleEvaluation(selfEvaluation != null && selfEvaluation.containsKey("aiops_rule_evaluation"))
.hasFeedback(run.getFeedback() != null && !run.getFeedback().isBlank())
.build();
}
private Map<String, Object> parseJsonObject(String json) { private Map<String, Object> parseJsonObject(String json) {
if (json == null || json.isBlank()) { if (json == null || json.isBlank()) {
return null; return null;
@@ -144,4 +295,14 @@ public class DiagnosisTraceService {
private int defaultInt(Integer value) { private int defaultInt(Integer value) {
return value == null ? 0 : value; return value == null ? 0 : value;
} }
private String preview(String text, int maxLength) {
if (text == null) {
return null;
}
if (text.length() <= maxLength) {
return text;
}
return text.substring(0, maxLength);
}
} }
@@ -2,11 +2,15 @@ package com.superbiz.agent.service;
import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.ObjectMapper;
import com.superbiz.agent.domain.entity.AgentStep; import com.superbiz.agent.domain.entity.AgentStep;
import com.superbiz.agent.domain.entity.ChatSession;
import com.superbiz.agent.domain.entity.DiagnosisRun;
import com.superbiz.agent.domain.entity.DiagnosisSession; import com.superbiz.agent.domain.entity.DiagnosisSession;
import com.superbiz.agent.domain.entity.ToolInvocation; import com.superbiz.agent.domain.entity.ToolInvocation;
import com.superbiz.agent.dto.DiagnosisTraceResponse; import com.superbiz.agent.dto.DiagnosisTraceResponse;
import com.superbiz.agent.exception.SessionNotFoundException; import com.superbiz.agent.exception.SessionNotFoundException;
import com.superbiz.agent.repository.AgentStepRepository; import com.superbiz.agent.repository.AgentStepRepository;
import com.superbiz.agent.repository.ChatSessionRepository;
import com.superbiz.agent.repository.DiagnosisRunRepository;
import com.superbiz.agent.repository.DiagnosisSessionRepository; import com.superbiz.agent.repository.DiagnosisSessionRepository;
import com.superbiz.agent.repository.ToolInvocationRepository; import com.superbiz.agent.repository.ToolInvocationRepository;
import org.junit.jupiter.api.Test; import org.junit.jupiter.api.Test;
@@ -15,148 +19,245 @@ import java.time.LocalDateTime;
import java.util.List; import java.util.List;
import java.util.Optional; import java.util.Optional;
import static org.junit.jupiter.api.Assertions.*; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.Mockito.*; import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.verifyNoInteractions;
import static org.mockito.Mockito.when;
class DiagnosisTraceServiceTest { class DiagnosisTraceServiceTest {
private final DiagnosisSessionRepository diagnosisSessionRepository = mock(DiagnosisSessionRepository.class); private final DiagnosisSessionRepository diagnosisSessionRepository = mock(DiagnosisSessionRepository.class);
private final ChatSessionRepository chatSessionRepository = mock(ChatSessionRepository.class);
private final DiagnosisRunRepository diagnosisRunRepository = mock(DiagnosisRunRepository.class);
private final AgentStepRepository agentStepRepository = mock(AgentStepRepository.class); private final AgentStepRepository agentStepRepository = mock(AgentStepRepository.class);
private final ToolInvocationRepository toolInvocationRepository = mock(ToolInvocationRepository.class); private final ToolInvocationRepository toolInvocationRepository = mock(ToolInvocationRepository.class);
private final DiagnosisTraceService service = new DiagnosisTraceService( private final DiagnosisTraceService service = new DiagnosisTraceService(
diagnosisSessionRepository, diagnosisSessionRepository,
chatSessionRepository,
diagnosisRunRepository,
agentStepRepository, agentStepRepository,
toolInvocationRepository, toolInvocationRepository,
new ObjectMapper() new ObjectMapper()
); );
@Test @Test
void getTraceAggregatesSessionStepsAndTools() { void getTraceWithoutRunIdResolvesLatestRun() {
String sessionId = "trace-session-001"; String sessionId = "trace-session-latest";
LocalDateTime now = LocalDateTime.of(2026, 7, 3, 14, 30); LocalDateTime base = LocalDateTime.of(2026, 7, 10, 10, 0);
DiagnosisSession session = DiagnosisSession.builder() DiagnosisRun latest = run(2L, sessionId, "run-latest", "second question", base.plusMinutes(1));
.id(1L) AgentStep step = step(20L, sessionId, "run-latest", 0, "composer", base.plusMinutes(1));
.sessionId(sessionId) ToolInvocation invocation = invocation(30L, sessionId, "run-latest", "query_metrics", base.plusMinutes(1));
.query("payment timeout")
.status("SUCCESS")
.agentFlow("COMPLEX")
.totalDurationMs(1200)
.totalTokenCount(300)
.stepCount(2)
.toolCallCount(1)
.answer("restart payment gateway pool")
.selfEvaluation("{\"verifier_evaluation\":{\"verdict\":\"PASS\"},\"aiops_rule_evaluation\":{\"verdict\":\"WARN\"}}")
.feedback("useful")
.createdAt(now)
.updatedAt(now)
.build();
AgentStep step = AgentStep.builder()
.id(10L)
.sessionId(sessionId)
.stepIndex(1)
.agentName("chat_executor")
.modelInput("input")
.modelOutput("output")
.thought("executor finished")
.hasToolCall(true)
.durationMs(500)
.tokenCount(100)
.createdAt(now)
.build();
ToolInvocation invocation = ToolInvocation.builder()
.id(20L)
.sessionId(sessionId)
.stepId(10L)
.toolName("lookup_knowledge")
.inputParams("{\"query\":\"ERR_TIMEOUT\"}")
.outputPreview("payment timeout doc")
.outputLength(19)
.retrievalLayer("L0")
.l0MatchCount(1)
.l1MatchCount(0)
.isTruncated(false)
.relevanceLevel("HIGHLY_RELEVANT")
.dedupReason("FIRST_HIT")
.retrievalDetails("{\"documents\":[\"payment-errors.md\"]}")
.durationMs(80)
.success(true)
.createdAt(now)
.build();
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(session)); when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
when(agentStepRepository.findBySessionId(sessionId)).thenReturn(List.of(step)); .thenReturn(Optional.of(latest));
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of(invocation)); when(chatSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(chatSession(sessionId)));
when(agentStepRepository.findByRunIdOrderByStepIndex("run-latest")).thenReturn(List.of(step));
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-latest")).thenReturn(List.of(invocation));
DiagnosisTraceResponse response = service.getTrace(sessionId); DiagnosisTraceResponse response = service.getTrace(sessionId);
assertEquals(sessionId, response.getSession().getSessionId()); assertEquals("run-latest", response.getRunId());
assertEquals("payment timeout", response.getSession().getQuery()); assertEquals("run-latest", response.getRun().getRunId());
assertEquals("PASS", ((java.util.Map<?, ?>) response.getSession() assertEquals("run-latest", response.getSummary().getResolvedRunId());
.getSelfEvaluation() assertEquals("second question", response.getSession().getQuery());
.get("verifier_evaluation")).get("verdict")); assertEquals("run-latest", response.getSteps().get(0).getRunId());
assertEquals(1, response.getSteps().size()); assertEquals("run-latest", response.getToolInvocations().get(0).getRunId());
assertEquals("chat_executor", response.getSteps().get(0).getAgentName()); assertEquals(1, response.getChatSession().getMessagePairCount());
assertEquals(1, response.getToolInvocations().size());
assertEquals("ERR_TIMEOUT", response.getToolInvocations().get(0).getInputParams().get("query"));
assertEquals(2, response.getSummary().getPersistedStepCount());
assertEquals(1, response.getSummary().getReturnedStepCount());
assertEquals(1, response.getSummary().getPersistedToolCallCount());
assertEquals(1, response.getSummary().getReturnedToolCallCount());
assertTrue(response.getSummary().isHasVerifierEvaluation());
assertTrue(response.getSummary().isHasAiOpsRuleEvaluation());
assertTrue(response.getSummary().isHasFeedback());
} }
@Test @Test
void getTraceOrdersStepsByCreationTimeAndIdNotPerAgentStepIndex() { void getTraceWithRunIdReturnsExactFirstRun() {
String sessionId = "trace-session-ordered"; String sessionId = "trace-session-exact";
LocalDateTime base = LocalDateTime.of(2026, 7, 7, 0, 0); DiagnosisRun first = run(1L, sessionId, "run-first", "first question",
DiagnosisSession session = DiagnosisSession.builder() LocalDateTime.of(2026, 7, 10, 10, 0));
.id(1L) AgentStep firstStep = step(10L, sessionId, "run-first", 0, "planner", first.getCreatedAt());
.sessionId(sessionId)
.query("mysql pool issue")
.status("SUCCESS")
.stepCount(4)
.toolCallCount(0)
.createdAt(base)
.updatedAt(base)
.build();
AgentStep planner = step(1L, sessionId, 0, "planner", base.plusSeconds(1));
AgentStep executor0 = step(2L, sessionId, 0, "executor", base.plusSeconds(2));
AgentStep executor1 = step(3L, sessionId, 1, "executor", base.plusSeconds(3));
AgentStep verifier = step(4L, sessionId, 0, "verifier", base.plusSeconds(4));
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(session)); when(diagnosisRunRepository.findBySessionIdAndRunId(sessionId, "run-first"))
when(agentStepRepository.findBySessionId(sessionId)) .thenReturn(Optional.of(first));
.thenReturn(List.of(planner, executor0, verifier, executor1)); when(agentStepRepository.findByRunIdOrderByStepIndex("run-first")).thenReturn(List.of(firstStep));
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of()); when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-first")).thenReturn(List.of());
DiagnosisTraceResponse response = service.getTrace(sessionId); DiagnosisTraceResponse response = service.getTrace(sessionId, "run-first");
assertEquals(List.of("planner", "executor", "executor", "verifier"), assertEquals("run-first", response.getRunId());
assertEquals("first question", response.getRun().getQuery());
assertEquals(List.of("planner"),
response.getSteps().stream().map(DiagnosisTraceResponse.AgentStepTrace::getAgentName).toList()); response.getSteps().stream().map(DiagnosisTraceResponse.AgentStepTrace::getAgentName).toList());
assertEquals(List.of(0, 0, 1, 0), verify(diagnosisRunRepository, never()).findFirstBySessionIdOrderByCreatedAtDescIdDesc(any());
response.getSteps().stream().map(DiagnosisTraceResponse.AgentStepTrace::getStepIndex).toList()); }
@Test
void getTraceWithRunIdReturnsExactSecondRun() {
String sessionId = "trace-session-exact";
DiagnosisRun second = run(2L, sessionId, "run-second", "second question",
LocalDateTime.of(2026, 7, 10, 10, 1));
ToolInvocation secondTool = invocation(20L, sessionId, "run-second", "query_logs", second.getCreatedAt());
when(diagnosisRunRepository.findBySessionIdAndRunId(sessionId, "run-second"))
.thenReturn(Optional.of(second));
when(agentStepRepository.findByRunIdOrderByStepIndex("run-second")).thenReturn(List.of());
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-second")).thenReturn(List.of(secondTool));
DiagnosisTraceResponse response = service.getTrace(sessionId, "run-second");
assertEquals("run-second", response.getRunId());
assertEquals("second question", response.getSession().getQuery());
assertEquals(List.of("query_logs"),
response.getToolInvocations().stream().map(DiagnosisTraceResponse.ToolInvocationTrace::getToolName).toList());
}
@Test
void getTraceRejectsRunFromAnotherSession() {
String runId = "run-other-session";
when(diagnosisRunRepository.findBySessionIdAndRunId("path-session", runId)).thenReturn(Optional.empty());
when(diagnosisRunRepository.findByRunId(runId)).thenReturn(Optional.of(run(
1L, "actual-session", runId, "query", LocalDateTime.of(2026, 7, 10, 10, 0))));
IllegalArgumentException error = assertThrows(IllegalArgumentException.class,
() -> service.getTrace("path-session", runId));
assertTrue(error.getMessage().contains("does not belong"));
verifyNoInteractions(agentStepRepository, toolInvocationRepository);
} }
@Test @Test
void getTraceThrowsWhenSessionMissing() { void getTraceThrowsWhenSessionMissing() {
String sessionId = "missing-session"; String sessionId = "missing-session";
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
.thenReturn(Optional.empty());
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.empty()); when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.empty());
assertThrows(SessionNotFoundException.class, () -> service.getTrace(sessionId)); assertThrows(SessionNotFoundException.class, () -> service.getTrace(sessionId));
verify(diagnosisRunRepository).findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId);
verify(diagnosisSessionRepository).findBySessionId(sessionId); verify(diagnosisSessionRepository).findBySessionId(sessionId);
verifyNoInteractions(agentStepRepository, toolInvocationRepository); verifyNoInteractions(agentStepRepository, toolInvocationRepository);
} }
private AgentStep step(Long id, String sessionId, int stepIndex, String agentName, LocalDateTime createdAt) { @Test
void getTraceIsReadOnly() {
String sessionId = "trace-session-readonly";
DiagnosisRun latest = run(3L, sessionId, "run-readonly", "readonly question",
LocalDateTime.of(2026, 7, 10, 10, 2));
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
.thenReturn(Optional.of(latest));
when(agentStepRepository.findByRunIdOrderByStepIndex("run-readonly")).thenReturn(List.of());
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-readonly")).thenReturn(List.of());
service.getTrace(sessionId);
verify(chatSessionRepository, never()).save(any());
verify(diagnosisRunRepository, never()).save(any());
verify(diagnosisSessionRepository, never()).save(any());
verify(agentStepRepository, never()).save(any());
verify(toolInvocationRepository, never()).save(any());
}
@Test
void listRunSummariesDoesNotExpandTraceDetails() {
String sessionId = "trace-session-runs";
DiagnosisRun second = run(2L, sessionId, "run-second", "second question",
LocalDateTime.of(2026, 7, 10, 10, 1));
second.setAnswer("answer ".repeat(40));
DiagnosisRun first = run(1L, sessionId, "run-first", "first question",
LocalDateTime.of(2026, 7, 10, 10, 0));
when(diagnosisRunRepository.findBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
.thenReturn(List.of(second, first));
List<DiagnosisTraceResponse.RunSummary> summaries = service.listRunSummaries(sessionId);
assertEquals(List.of("run-second", "run-first"),
summaries.stream().map(DiagnosisTraceResponse.RunSummary::getRunId).toList());
assertEquals("second question", summaries.get(0).getQuery());
assertTrue(summaries.get(0).getAnswerPreview().length() <= 160);
verifyNoInteractions(agentStepRepository, toolInvocationRepository);
}
@Test
void legacyTraceFallbackKeepsHistoricalSessionReadable() {
String sessionId = "legacy-session";
DiagnosisSession legacy = DiagnosisSession.builder()
.id(1L)
.sessionId(sessionId)
.query("legacy question")
.status("SUCCESS")
.stepCount(0)
.toolCallCount(0)
.createdAt(LocalDateTime.of(2026, 7, 10, 9, 0))
.updatedAt(LocalDateTime.of(2026, 7, 10, 9, 1))
.build();
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
.thenReturn(Optional.empty());
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(legacy));
when(agentStepRepository.findBySessionId(sessionId)).thenReturn(List.of());
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of());
DiagnosisTraceResponse response = service.getTrace(sessionId);
assertNull(response.getRunId());
assertNull(response.getRun());
assertEquals("legacy question", response.getSession().getQuery());
}
private DiagnosisRun run(Long id, String sessionId, String runId, String query, LocalDateTime createdAt) {
return DiagnosisRun.builder()
.id(id)
.sessionId(sessionId)
.runId(runId)
.query(query)
.status("SUCCESS")
.agentFlow("CHAT")
.answer("answer for " + runId)
.selfEvaluation("{\"verifier_evaluation\":{\"verdict\":\"PASS\"}}")
.feedback("useful")
.stepCount(1)
.toolCallCount(1)
.createdAt(createdAt)
.updatedAt(createdAt.plusSeconds(1))
.build();
}
private ChatSession chatSession(String sessionId) {
return ChatSession.builder()
.id(99L)
.sessionId(sessionId)
.status("ACTIVE")
.messagePairCount(1)
.createdAt(LocalDateTime.of(2026, 7, 10, 9, 0))
.lastActiveAt(LocalDateTime.of(2026, 7, 10, 10, 0))
.build();
}
private AgentStep step(Long id, String sessionId, String runId, int stepIndex, String agentName, LocalDateTime createdAt) {
return AgentStep.builder() return AgentStep.builder()
.id(id) .id(id)
.sessionId(sessionId) .sessionId(sessionId)
.runId(runId)
.stepIndex(stepIndex) .stepIndex(stepIndex)
.agentName(agentName) .agentName(agentName)
.createdAt(createdAt) .createdAt(createdAt)
.build(); .build();
} }
private ToolInvocation invocation(Long id, String sessionId, String runId, String toolName, LocalDateTime createdAt) {
return ToolInvocation.builder()
.id(id)
.sessionId(sessionId)
.runId(runId)
.toolName(toolName)
.inputParams("{\"query\":\"timeout\"}")
.retrievalDetails("{\"evidence_refs\":[]}")
.success(true)
.createdAt(createdAt)
.build();
}
} }