feat(trace): add run-scoped trace reads
This commit is contained in:
zhuyongxin
@@ -143,6 +143,12 @@ Audit conclusions:
|
||||
- Clarified that `chat_session.expires_at` is nullable directory metadata / best-effort TTL snapshot, not mandatory persisted conversation history.
|
||||
- Clarified document review findings before continuing Phase 2: OpenSpec task phases are authoritative over the older active issue phase sketch, and AIOps rule evaluation is stored in `diagnosis_run.self_evaluation.aiops_rule_evaluation`, not a separate table.
|
||||
|
||||
## Document Review Follow-up Before Phase 3 Gate
|
||||
|
||||
- Clarified AIOps SSE compatibility before Phase 5: keep SSE event name `message`, emit a JSON `SseMessage` with `type=metadata`, and preserve existing content message shape for report streaming.
|
||||
- Clarified Feedback API before Phase 4: request `runId` is preferred, response always includes the bound `runId` and `fallbackToLatestRun`, and wrong-session run binding uses the existing failed feedback response path.
|
||||
- Clarified run-list API before Phase 3 gate: `GET /api/chat/session/{sessionId}/runs` returns `ApiResponse<List<RunSummary>>`, returns an empty list for an existing session with no runs, and uses existing missing-session error behavior when no session/run data exists.
|
||||
|
||||
## Phase 2 Apply Notes
|
||||
|
||||
- Capability source: `openspec-apply-change` + sm-flow apply protocol. `codebase-retrieval` and LSP tools were not available in this session, so call-chain confirmation used OpenSpec context, `rg`, targeted file reads, compilation, focused tests, E2E, DB inspection, and logs.
|
||||
@@ -151,3 +157,14 @@ Audit conclusions:
|
||||
- Switched Chat run completion, failure, self-evaluation, metrics, verifier support reads, gatekeeper validation, and evidence scoring to run-scoped data.
|
||||
- Added `/api/chat` response `runId` and focused tests for valid run creation, invalid request no-run behavior, run-scoped trace consumers, and same-session multi-turn run creation.
|
||||
- Phase 2 gate evidence is recorded in `phase-2-evidence.md`.
|
||||
|
||||
## Phase 3 Apply Notes
|
||||
|
||||
- Capability source: `openspec-apply-change` + sm-flow apply protocol. The committed OpenSpec remained the execution source; a document review follow-up tightened DTO/SSE/listing contracts before the Phase 3 gate.
|
||||
- Implemented latest-run trace resolution using `diagnosis_run.created_at DESC, id DESC`.
|
||||
- Implemented exact trace lookup for `sessionId + runId` with run/session ownership validation.
|
||||
- Changed trace details to read `agent_step` and `tool_invocation` by `run_id`, while retaining a legacy `diagnosis_session` fallback for historical compatibility.
|
||||
- Added trace response fields for resolved `runId`, chat session metadata, run metadata, and per-row `runId`.
|
||||
- Added lightweight `GET /api/chat/session/{sessionId}/runs` backed by `diagnosis_run` summaries.
|
||||
- Added focused tests for latest trace, exact first trace, exact second trace, wrong-session rejection, missing session, read-only trace behavior, run listing, and legacy fallback.
|
||||
- Phase 3 E2E used Maven startup with profile `mvp-demo`; evidence is recorded in `phase-3-evidence.md`.
|
||||
|
||||
@@ -98,10 +98,10 @@ Interface level: L4.
|
||||
|
||||
- Database contract changes: new tables, new columns, backfill, indexes, and later non-null expectations for new writes.
|
||||
- `/api/chat` response adds `runId`.
|
||||
- `/api/ai_ops` SSE emits a compatible metadata message before report content. The metadata payload includes `sessionId` and `runId`; report content continues to stream through the existing content message shape.
|
||||
- `/api/ai_ops` SSE emits a compatible metadata message before report content. It keeps the existing SSE event name `message` and sends a JSON `SseMessage` with `type=metadata`; the metadata payload includes `sessionId` and `runId`. Report content continues to stream through the existing `type=content` message shape.
|
||||
- Trace API accepts optional `runId`.
|
||||
- Feedback request accepts preferred `runId` and returns fallback binding metadata when omitted.
|
||||
- New run summary API: `GET /api/chat/session/{sessionId}/runs`.
|
||||
- Feedback request accepts preferred `runId`. Feedback response includes the actual bound `runId` and `fallbackToLatestRun`; wrong-session `runId`, missing session, and missing run use the existing failed feedback response path with HTTP 400 from `FeedbackController`.
|
||||
- New run summary API: `GET /api/chat/session/{sessionId}/runs`, returned through the existing `ApiResponse<List<RunSummary>>` wrapper. A session with metadata but no runs returns an empty list; a missing session returns the existing not-found/error behavior.
|
||||
|
||||
Compatibility:
|
||||
|
||||
|
||||
@@ -0,0 +1,136 @@
|
||||
# Phase 3 Evidence: Trace Read Path and Run Listing
|
||||
|
||||
## Scope
|
||||
|
||||
Phase 3 implements run-scoped trace reads and lightweight run listing:
|
||||
|
||||
- `GET /api/diagnosis/{sessionId}/trace` resolves the latest run by `diagnosis_run.created_at DESC, id DESC`.
|
||||
- `GET /api/diagnosis/{sessionId}/trace?runId=...` returns the exact run after validating run/session ownership.
|
||||
- Trace responses include resolved run metadata and run-scoped step/tool rows.
|
||||
- `GET /api/chat/session/{sessionId}/runs` returns lightweight run summaries without expanding trace detail rows.
|
||||
|
||||
## Verification Commands
|
||||
|
||||
- `mvn -q clean "-Dtest=DiagnosisTraceServiceTest" test`
|
||||
- `mvn -q "-Dtest=DiagnosisTraceServiceTest,DiagnosisTraceEvaluatorTest,ChatControllerTest" test`
|
||||
- `openspec validate session-run-trace-isolation --strict`
|
||||
|
||||
After the document review follow-up, OpenSpec strict validation was run again:
|
||||
|
||||
- `openspec validate session-run-trace-isolation --strict`
|
||||
|
||||
Result: passed.
|
||||
|
||||
## E2E Runtime
|
||||
|
||||
Maven startup:
|
||||
|
||||
```text
|
||||
mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo
|
||||
```
|
||||
|
||||
Captured logs and artifacts:
|
||||
|
||||
- `target/e2e/phase3-mvn-20260710-191131.out.log`
|
||||
- `target/e2e/phase3-mvn-20260710-191131.err.log`
|
||||
- `target/e2e/phase3-request-round1.json`
|
||||
- `target/e2e/phase3-response-round1.json`
|
||||
- `target/e2e/phase3-request-round2.json`
|
||||
- `target/e2e/phase3-response-round2.json`
|
||||
- `target/e2e/phase3-trace-latest.json`
|
||||
- `target/e2e/phase3-trace-first.json`
|
||||
- `target/e2e/phase3-trace-second.json`
|
||||
- `target/e2e/phase3-runs.json`
|
||||
- `target/e2e/phase3-trace-wrong-session.json`
|
||||
- `target/e2e/phase3-summary.json`
|
||||
|
||||
The E2E Maven process was stopped after evidence collection.
|
||||
|
||||
Note: `logs/application.log` was checked, but it did not contain the Phase 3 E2E session entries and its last write time was earlier than this E2E run. The Phase 3 runtime application logs were captured in the Maven stdout artifact above.
|
||||
|
||||
## E2E Summary
|
||||
|
||||
Session:
|
||||
|
||||
```text
|
||||
sessionId = e2e-phase3-codex-20260710-1912
|
||||
run1 = run-fdccbe21-e050-4f92-a741-a062aef59644
|
||||
run2 = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
||||
```
|
||||
|
||||
Observed behavior from `phase3-summary.json`:
|
||||
|
||||
```text
|
||||
distinctRunIds = true
|
||||
latestResolvedRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
||||
firstTraceRunId = run-fdccbe21-e050-4f92-a741-a062aef59644
|
||||
firstSteps = 9
|
||||
firstTools = 13
|
||||
secondTraceRunId = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
||||
secondSteps = 2
|
||||
secondTools = 0
|
||||
runListCount = 2
|
||||
runListFirst = run-a9b883ab-9cab-4eec-accf-38129bd2eb94
|
||||
wrongSessionStatus = 400
|
||||
```
|
||||
|
||||
Log evidence from `phase3-mvn-20260710-191131.out.log`:
|
||||
|
||||
- Round 1 request was received for `e2e-phase3-codex-20260710-1912`.
|
||||
- Redis session was created for the first round and reused by the second round.
|
||||
- Message pair count reached `2` after round 2.
|
||||
- Latest trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
|
||||
- Exact first trace request returned `run-fdccbe21-e050-4f92-a741-a062aef59644`.
|
||||
- Exact second trace request returned `run-a9b883ab-9cab-4eec-accf-38129bd2eb94`.
|
||||
- Wrong-session exact trace returned HTTP 400 with `runId does not belong to sessionId`.
|
||||
|
||||
## Database Inspection
|
||||
|
||||
Queried through `scripts/query_mysql.py`.
|
||||
|
||||
`diagnosis_run` rows:
|
||||
|
||||
```text
|
||||
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | SUCCESS | CHAT | step_count=2 | tool_call_count=0 | created_at=2026-07-10 19:15:40
|
||||
run-fdccbe21-e050-4f92-a741-a062aef59644 | SUCCESS | CHAT | step_count=9 | tool_call_count=13 | created_at=2026-07-10 19:12:33
|
||||
```
|
||||
|
||||
`agent_step` rows grouped by run:
|
||||
|
||||
```text
|
||||
run-a9b883ab-9cab-4eec-accf-38129bd2eb94 | step_rows=2 | min_step=0 | max_step=1
|
||||
run-fdccbe21-e050-4f92-a741-a062aef59644 | step_rows=9 | min_step=0 | max_step=4
|
||||
```
|
||||
|
||||
`tool_invocation` rows grouped by run:
|
||||
|
||||
```text
|
||||
run-fdccbe21-e050-4f92-a741-a062aef59644 | tool_rows=13
|
||||
```
|
||||
|
||||
Missing run id checks for this session:
|
||||
|
||||
```text
|
||||
agent_step missing run_id = 0
|
||||
tool_invocation missing run_id = 0
|
||||
```
|
||||
|
||||
`chat_session` metadata:
|
||||
|
||||
```text
|
||||
session_id=e2e-phase3-codex-20260710-1912 | status=ACTIVE | message_pair_count=2
|
||||
```
|
||||
|
||||
## Document Review Follow-up
|
||||
|
||||
Before closing Phase 3, the OpenSpec docs were tightened for upcoming phases:
|
||||
|
||||
- AIOps SSE metadata shape: keep SSE event name `message`, use JSON `SseMessage` with `type=metadata`, preserve existing content message shape.
|
||||
- Feedback DTO contract: request `runId` is preferred; response includes bound `runId` and `fallbackToLatestRun`; wrong-session run binding fails instead of updating either run.
|
||||
- Run-list API: returns `ApiResponse<List<RunSummary>>`, returns an empty list for an existing session with no runs, and uses existing missing-session error behavior when no session/run data exists.
|
||||
|
||||
OpenSpec strict validation passed after these document changes.
|
||||
|
||||
## Conclusion
|
||||
|
||||
Phase 3 satisfies the run-scoped trace read and run-list contract. Same-session multi-turn E2E proves latest-run compatibility, exact-run replay, run-list ordering, run/session ownership rejection, and no missing `run_id` rows for new trace data.
|
||||
@@ -72,6 +72,8 @@ Level: L4 database/API contract migration with compatibility behavior.
|
||||
- New query parameter: `GET /api/diagnosis/{sessionId}/trace?runId=...`.
|
||||
- New API: `GET /api/chat/session/{sessionId}/runs`.
|
||||
- Feedback request gains optional/preferred `runId`.
|
||||
- Feedback response returns bound `runId` and `fallbackToLatestRun`.
|
||||
- `/api/ai_ops` keeps SSE event name `message` and emits a `type=metadata` JSON message containing `sessionId` and `runId` before content.
|
||||
- Database contract changes include new tables and new `run_id` columns.
|
||||
- Old callers that only pass `sessionId` remain compatible by binding to latest run, but this fallback must be observable.
|
||||
|
||||
|
||||
+18
-1
@@ -72,10 +72,18 @@ The system SHALL provide a lightweight run-list API for a Chat Session.
|
||||
|
||||
#### Scenario: Run list returns summaries
|
||||
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs`
|
||||
- **THEN** the system SHALL return run summaries from `diagnosis_run`
|
||||
- **THEN** the system SHALL return run summaries from `diagnosis_run` inside the existing API response wrapper
|
||||
- **AND** each summary SHALL include `runId`, `sessionId`, `query`, `status`, `agentFlow`, `answerPreview`, `stepCount`, `toolCallCount`, `createdAt`, and `updatedAt`
|
||||
- **AND** the response SHALL NOT expand `agent_step` or `tool_invocation` detail rows
|
||||
|
||||
#### Scenario: Run list handles session without runs
|
||||
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs` for an existing `chat_session` with no runs
|
||||
- **THEN** the system SHALL return a successful empty list
|
||||
|
||||
#### Scenario: Run list rejects missing session
|
||||
- **WHEN** a caller requests `GET /api/chat/session/{sessionId}/runs` for a session that does not exist in `chat_session` or `diagnosis_run`
|
||||
- **THEN** the system SHALL use the existing not-found/error response behavior
|
||||
|
||||
### Requirement: Feedback SHALL bind to diagnosis runs
|
||||
The system SHALL bind new feedback to a diagnosis run rather than an ambiguous multi-turn session.
|
||||
|
||||
@@ -83,6 +91,8 @@ The system SHALL bind new feedback to a diagnosis run rather than an ambiguous m
|
||||
- **WHEN** a feedback request includes `sessionId` and `runId`
|
||||
- **THEN** the system SHALL validate that the run belongs to the session
|
||||
- **AND** it SHALL update feedback on that run
|
||||
- **AND** the response SHALL include the actual bound `runId`
|
||||
- **AND** the response SHALL include `fallbackToLatestRun=false`
|
||||
|
||||
#### Scenario: Feedback without runId falls back observably
|
||||
- **WHEN** a legacy feedback request includes `sessionId` but omits `runId`
|
||||
@@ -90,6 +100,10 @@ The system SHALL bind new feedback to a diagnosis run rather than an ambiguous m
|
||||
- **AND** the response SHALL include `fallbackToLatestRun=true`
|
||||
- **AND** the response SHALL include the actual bound `runId`
|
||||
|
||||
#### Scenario: Feedback rejects run from another session
|
||||
- **WHEN** a feedback request includes a `runId` that belongs to a different `sessionId`
|
||||
- **THEN** the system SHALL return a failed feedback response instead of updating either run
|
||||
|
||||
#### Scenario: Useful feedback creates case from run
|
||||
- **WHEN** feedback for a run is `useful`
|
||||
- **THEN** the system SHALL create or reuse a `case_library` row using that run's query and answer
|
||||
@@ -107,8 +121,11 @@ The system SHALL create and expose a diagnosis run for every valid `/api/ai_ops`
|
||||
#### Scenario: AIOps SSE exposes runId
|
||||
- **WHEN** `/api/ai_ops` streams response metadata to the caller
|
||||
- **THEN** the stream SHALL send a compatible metadata message before report content
|
||||
- **AND** the SSE event name SHALL remain `message`
|
||||
- **AND** the message type SHALL be `metadata`
|
||||
- **AND** the metadata payload SHALL expose the resolved `sessionId`
|
||||
- **AND** the metadata payload SHALL expose the created `runId`
|
||||
- **AND** report content SHALL continue to use the existing content message shape
|
||||
|
||||
### Requirement: Migration SHALL preserve historical trace access
|
||||
The system SHALL migrate historical diagnosis data into compatibility runs without deleting the old `diagnosis_session` table.
|
||||
|
||||
@@ -22,13 +22,13 @@
|
||||
|
||||
## 3. Trace Read Path and Run Listing
|
||||
|
||||
- [ ] 3.1 Change `DiagnosisTraceService` to resolve latest run by `diagnosis_run.created_at DESC, id DESC` when `runId` is omitted.
|
||||
- [ ] 3.2 Add exact trace lookup for `sessionId + runId`, including validation that the run belongs to the session.
|
||||
- [ ] 3.3 Change trace response DTOs to include resolved `runId` and run summary fields.
|
||||
- [ ] 3.4 Add `GET /api/chat/session/{sessionId}/runs` returning lightweight run summaries without expanding trace detail rows.
|
||||
- [ ] 3.5 Preserve read-only trace behavior for latest-run and exact-run requests.
|
||||
- [ ] 3.6 Add tests for latest run, exact first run, exact second run, wrong-session run rejection, missing session, and read-only behavior.
|
||||
- [ ] 3.7 Phase 3 gate: run focused trace tests and same-session E2E trace checks, update task status, archive phase evidence, and commit before starting Phase 4.
|
||||
- [x] 3.1 Change `DiagnosisTraceService` to resolve latest run by `diagnosis_run.created_at DESC, id DESC` when `runId` is omitted.
|
||||
- [x] 3.2 Add exact trace lookup for `sessionId + runId`, including validation that the run belongs to the session.
|
||||
- [x] 3.3 Change trace response DTOs to include resolved `runId` and run summary fields.
|
||||
- [x] 3.4 Add `GET /api/chat/session/{sessionId}/runs` returning lightweight run summaries without expanding trace detail rows.
|
||||
- [x] 3.5 Preserve read-only trace behavior for latest-run and exact-run requests.
|
||||
- [x] 3.6 Add tests for latest run, exact first run, exact second run, wrong-session run rejection, missing session, and read-only behavior.
|
||||
- [x] 3.7 Phase 3 gate: run focused trace tests and same-session E2E trace checks, update task status, archive phase evidence, and commit before starting Phase 4.
|
||||
|
||||
## 4. Feedback and Case Library Run Binding
|
||||
|
||||
|
||||
@@ -5,8 +5,10 @@ import lombok.Getter;
|
||||
import lombok.Setter;
|
||||
import com.superbiz.agent.domain.model.SessionContext;
|
||||
import com.superbiz.agent.dto.AIOpsRequest;
|
||||
import com.superbiz.agent.dto.DiagnosisTraceResponse;
|
||||
import com.superbiz.agent.service.AiOpsService;
|
||||
import com.superbiz.agent.service.ChatService;
|
||||
import com.superbiz.agent.service.DiagnosisTraceService;
|
||||
import com.superbiz.agent.service.session.SessionManager;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
@@ -43,6 +45,9 @@ public class ChatController {
|
||||
@Autowired
|
||||
private ChatService chatService;
|
||||
|
||||
@Autowired
|
||||
private DiagnosisTraceService diagnosisTraceService;
|
||||
|
||||
@Autowired
|
||||
private SessionManager sessionManager;
|
||||
|
||||
@@ -325,6 +330,12 @@ public class ChatController {
|
||||
}
|
||||
}
|
||||
|
||||
@GetMapping("/chat/session/{sessionId}/runs")
|
||||
public ResponseEntity<ApiResponse<List<DiagnosisTraceResponse.RunSummary>>> listSessionRuns(
|
||||
@PathVariable String sessionId) {
|
||||
return ResponseEntity.ok(ApiResponse.success(diagnosisTraceService.listRunSummaries(sessionId)));
|
||||
}
|
||||
|
||||
// ==================== 辅助方法 ====================
|
||||
|
||||
private SessionContext getOrCreateSession(String sessionId) {
|
||||
|
||||
@@ -7,6 +7,7 @@ import lombok.RequiredArgsConstructor;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PathVariable;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
@@ -18,7 +19,10 @@ public class DiagnosisTraceController {
|
||||
private final DiagnosisTraceService diagnosisTraceService;
|
||||
|
||||
@GetMapping("/{sessionId}/trace")
|
||||
public ResponseEntity<Result<DiagnosisTraceResponse>> getTrace(@PathVariable String sessionId) {
|
||||
return ResponseEntity.ok(Result.success(diagnosisTraceService.getTrace(sessionId)));
|
||||
public ResponseEntity<Result<DiagnosisTraceResponse>> getTrace(
|
||||
@PathVariable String sessionId,
|
||||
@RequestParam(required = false) String runId
|
||||
) {
|
||||
return ResponseEntity.ok(Result.success(diagnosisTraceService.getTrace(sessionId, runId)));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,11 +15,28 @@ import java.util.Map;
|
||||
@AllArgsConstructor
|
||||
public class DiagnosisTraceResponse {
|
||||
|
||||
private String runId;
|
||||
private ChatSessionTrace chatSession;
|
||||
private SessionTrace session;
|
||||
private RunTrace run;
|
||||
private List<AgentStepTrace> steps;
|
||||
private List<ToolInvocationTrace> toolInvocations;
|
||||
private TraceSummary summary;
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public static class ChatSessionTrace {
|
||||
private Long id;
|
||||
private String sessionId;
|
||||
private String status;
|
||||
private Integer messagePairCount;
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime lastActiveAt;
|
||||
private LocalDateTime expiresAt;
|
||||
}
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@@ -42,6 +59,29 @@ public class DiagnosisTraceResponse {
|
||||
private LocalDateTime updatedAt;
|
||||
}
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public static class RunTrace {
|
||||
private Long id;
|
||||
private String runId;
|
||||
private String sessionId;
|
||||
private String query;
|
||||
private String status;
|
||||
private String agentFlow;
|
||||
private Integer totalDurationMs;
|
||||
private Integer totalTokenCount;
|
||||
private Integer stepCount;
|
||||
private Integer toolCallCount;
|
||||
private String answer;
|
||||
private String selfEvaluationRaw;
|
||||
private Map<String, Object> selfEvaluation;
|
||||
private String feedback;
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime updatedAt;
|
||||
}
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@@ -49,6 +89,7 @@ public class DiagnosisTraceResponse {
|
||||
public static class AgentStepTrace {
|
||||
private Long id;
|
||||
private String sessionId;
|
||||
private String runId;
|
||||
private Integer stepIndex;
|
||||
private String agentName;
|
||||
private String modelInput;
|
||||
@@ -67,6 +108,7 @@ public class DiagnosisTraceResponse {
|
||||
public static class ToolInvocationTrace {
|
||||
private Long id;
|
||||
private String sessionId;
|
||||
private String runId;
|
||||
private Long stepId;
|
||||
private String toolName;
|
||||
private String inputParamsRaw;
|
||||
@@ -92,6 +134,7 @@ public class DiagnosisTraceResponse {
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public static class TraceSummary {
|
||||
private String resolvedRunId;
|
||||
private int persistedStepCount;
|
||||
private int returnedStepCount;
|
||||
private int persistedToolCallCount;
|
||||
@@ -100,4 +143,21 @@ public class DiagnosisTraceResponse {
|
||||
private boolean hasAiOpsRuleEvaluation;
|
||||
private boolean hasFeedback;
|
||||
}
|
||||
|
||||
@Data
|
||||
@Builder
|
||||
@NoArgsConstructor
|
||||
@AllArgsConstructor
|
||||
public static class RunSummary {
|
||||
private String runId;
|
||||
private String sessionId;
|
||||
private String query;
|
||||
private String status;
|
||||
private String agentFlow;
|
||||
private String answerPreview;
|
||||
private Integer stepCount;
|
||||
private Integer toolCallCount;
|
||||
private LocalDateTime createdAt;
|
||||
private LocalDateTime updatedAt;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,11 +3,15 @@ package com.superbiz.agent.service;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.domain.entity.AgentStep;
|
||||
import com.superbiz.agent.domain.entity.ChatSession;
|
||||
import com.superbiz.agent.domain.entity.DiagnosisRun;
|
||||
import com.superbiz.agent.domain.entity.DiagnosisSession;
|
||||
import com.superbiz.agent.domain.entity.ToolInvocation;
|
||||
import com.superbiz.agent.dto.DiagnosisTraceResponse;
|
||||
import com.superbiz.agent.exception.SessionNotFoundException;
|
||||
import com.superbiz.agent.repository.AgentStepRepository;
|
||||
import com.superbiz.agent.repository.ChatSessionRepository;
|
||||
import com.superbiz.agent.repository.DiagnosisRunRepository;
|
||||
import com.superbiz.agent.repository.DiagnosisSessionRepository;
|
||||
import com.superbiz.agent.repository.ToolInvocationRepository;
|
||||
import lombok.RequiredArgsConstructor;
|
||||
@@ -25,18 +29,76 @@ public class DiagnosisTraceService {
|
||||
};
|
||||
|
||||
private final DiagnosisSessionRepository diagnosisSessionRepository;
|
||||
private final ChatSessionRepository chatSessionRepository;
|
||||
private final DiagnosisRunRepository diagnosisRunRepository;
|
||||
private final AgentStepRepository agentStepRepository;
|
||||
private final ToolInvocationRepository toolInvocationRepository;
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public DiagnosisTraceResponse getTrace(String sessionId) {
|
||||
return getTrace(sessionId, null);
|
||||
}
|
||||
|
||||
public DiagnosisTraceResponse getTrace(String sessionId, String runId) {
|
||||
if (runId != null && !runId.isBlank()) {
|
||||
DiagnosisRun run = diagnosisRunRepository.findBySessionIdAndRunId(sessionId, runId)
|
||||
.orElseThrow(() -> buildRunLookupException(sessionId, runId));
|
||||
return buildRunTraceResponse(sessionId, run);
|
||||
}
|
||||
|
||||
return diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId)
|
||||
.map(run -> buildRunTraceResponse(sessionId, run))
|
||||
.orElseGet(() -> buildLegacyTraceResponse(sessionId));
|
||||
}
|
||||
|
||||
public List<DiagnosisTraceResponse.RunSummary> listRunSummaries(String sessionId) {
|
||||
List<DiagnosisRun> runs = diagnosisRunRepository.findBySessionIdOrderByCreatedAtDescIdDesc(sessionId);
|
||||
if (runs.isEmpty() && chatSessionRepository.findBySessionId(sessionId).isEmpty()) {
|
||||
throw new SessionNotFoundException(sessionId);
|
||||
}
|
||||
return runs.stream()
|
||||
.map(this::toRunSummary)
|
||||
.toList();
|
||||
}
|
||||
|
||||
private RuntimeException buildRunLookupException(String sessionId, String runId) {
|
||||
if (diagnosisRunRepository.findByRunId(runId).isPresent()) {
|
||||
return new IllegalArgumentException("runId does not belong to sessionId: " + runId);
|
||||
}
|
||||
return new SessionNotFoundException(sessionId, "Run not found: " + runId);
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse buildRunTraceResponse(String sessionId, DiagnosisRun run) {
|
||||
List<AgentStep> steps = orderStepsForTrace(agentStepRepository.findByRunIdOrderByStepIndex(run.getRunId()));
|
||||
List<ToolInvocation> toolInvocations = toolInvocationRepository.findByRunIdOrderByIdAsc(run.getRunId());
|
||||
DiagnosisTraceResponse.RunTrace runTrace = toRunTrace(run);
|
||||
|
||||
return DiagnosisTraceResponse.builder()
|
||||
.runId(run.getRunId())
|
||||
.chatSession(chatSessionRepository.findBySessionId(sessionId)
|
||||
.map(this::toChatSessionTrace)
|
||||
.orElse(null))
|
||||
.session(toSessionTrace(runTrace))
|
||||
.run(runTrace)
|
||||
.steps(steps.stream().map(this::toAgentStepTrace).toList())
|
||||
.toolInvocations(toolInvocations.stream().map(this::toToolInvocationTrace).toList())
|
||||
.summary(toSummary(runTrace, steps, toolInvocations))
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse buildLegacyTraceResponse(String sessionId) {
|
||||
DiagnosisSession session = diagnosisSessionRepository.findBySessionId(sessionId)
|
||||
.orElseThrow(() -> new SessionNotFoundException(sessionId));
|
||||
List<AgentStep> steps = orderStepsForTrace(agentStepRepository.findBySessionId(sessionId));
|
||||
List<ToolInvocation> toolInvocations = toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId);
|
||||
|
||||
return DiagnosisTraceResponse.builder()
|
||||
.runId(null)
|
||||
.chatSession(chatSessionRepository.findBySessionId(sessionId)
|
||||
.map(this::toChatSessionTrace)
|
||||
.orElse(null))
|
||||
.session(toSessionTrace(session))
|
||||
.run(null)
|
||||
.steps(steps.stream().map(this::toAgentStepTrace).toList())
|
||||
.toolInvocations(toolInvocations.stream().map(this::toToolInvocationTrace).toList())
|
||||
.summary(toSummary(session, steps, toolInvocations))
|
||||
@@ -52,6 +114,18 @@ public class DiagnosisTraceService {
|
||||
.toList();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.ChatSessionTrace toChatSessionTrace(ChatSession chatSession) {
|
||||
return DiagnosisTraceResponse.ChatSessionTrace.builder()
|
||||
.id(chatSession.getId())
|
||||
.sessionId(chatSession.getSessionId())
|
||||
.status(chatSession.getStatus())
|
||||
.messagePairCount(chatSession.getMessagePairCount())
|
||||
.createdAt(chatSession.getCreatedAt())
|
||||
.lastActiveAt(chatSession.getLastActiveAt())
|
||||
.expiresAt(chatSession.getExpiresAt())
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.SessionTrace toSessionTrace(DiagnosisSession session) {
|
||||
return DiagnosisTraceResponse.SessionTrace.builder()
|
||||
.id(session.getId())
|
||||
@@ -72,10 +146,52 @@ public class DiagnosisTraceService {
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.SessionTrace toSessionTrace(DiagnosisTraceResponse.RunTrace run) {
|
||||
return DiagnosisTraceResponse.SessionTrace.builder()
|
||||
.id(run.getId())
|
||||
.sessionId(run.getSessionId())
|
||||
.query(run.getQuery())
|
||||
.status(run.getStatus())
|
||||
.agentFlow(run.getAgentFlow())
|
||||
.totalDurationMs(run.getTotalDurationMs())
|
||||
.totalTokenCount(run.getTotalTokenCount())
|
||||
.stepCount(run.getStepCount())
|
||||
.toolCallCount(run.getToolCallCount())
|
||||
.answer(run.getAnswer())
|
||||
.selfEvaluationRaw(run.getSelfEvaluationRaw())
|
||||
.selfEvaluation(run.getSelfEvaluation())
|
||||
.feedback(run.getFeedback())
|
||||
.createdAt(run.getCreatedAt())
|
||||
.updatedAt(run.getUpdatedAt())
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.RunTrace toRunTrace(DiagnosisRun run) {
|
||||
return DiagnosisTraceResponse.RunTrace.builder()
|
||||
.id(run.getId())
|
||||
.runId(run.getRunId())
|
||||
.sessionId(run.getSessionId())
|
||||
.query(run.getQuery())
|
||||
.status(run.getStatus())
|
||||
.agentFlow(run.getAgentFlow())
|
||||
.totalDurationMs(run.getTotalDurationMs())
|
||||
.totalTokenCount(run.getTotalTokenCount())
|
||||
.stepCount(run.getStepCount())
|
||||
.toolCallCount(run.getToolCallCount())
|
||||
.answer(run.getAnswer())
|
||||
.selfEvaluationRaw(run.getSelfEvaluation())
|
||||
.selfEvaluation(parseJsonObject(run.getSelfEvaluation()))
|
||||
.feedback(run.getFeedback())
|
||||
.createdAt(run.getCreatedAt())
|
||||
.updatedAt(run.getUpdatedAt())
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.AgentStepTrace toAgentStepTrace(AgentStep step) {
|
||||
return DiagnosisTraceResponse.AgentStepTrace.builder()
|
||||
.id(step.getId())
|
||||
.sessionId(step.getSessionId())
|
||||
.runId(step.getRunId())
|
||||
.stepIndex(step.getStepIndex())
|
||||
.agentName(step.getAgentName())
|
||||
.modelInput(step.getModelInput())
|
||||
@@ -92,6 +208,7 @@ public class DiagnosisTraceService {
|
||||
return DiagnosisTraceResponse.ToolInvocationTrace.builder()
|
||||
.id(invocation.getId())
|
||||
.sessionId(invocation.getSessionId())
|
||||
.runId(invocation.getRunId())
|
||||
.stepId(invocation.getStepId())
|
||||
.toolName(invocation.getToolName())
|
||||
.inputParamsRaw(invocation.getInputParams())
|
||||
@@ -113,6 +230,21 @@ public class DiagnosisTraceService {
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.RunSummary toRunSummary(DiagnosisRun run) {
|
||||
return DiagnosisTraceResponse.RunSummary.builder()
|
||||
.runId(run.getRunId())
|
||||
.sessionId(run.getSessionId())
|
||||
.query(run.getQuery())
|
||||
.status(run.getStatus())
|
||||
.agentFlow(run.getAgentFlow())
|
||||
.answerPreview(preview(run.getAnswer(), 160))
|
||||
.stepCount(run.getStepCount())
|
||||
.toolCallCount(run.getToolCallCount())
|
||||
.createdAt(run.getCreatedAt())
|
||||
.updatedAt(run.getUpdatedAt())
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.TraceSummary toSummary(
|
||||
DiagnosisSession session,
|
||||
List<AgentStep> steps,
|
||||
@@ -120,6 +252,7 @@ public class DiagnosisTraceService {
|
||||
) {
|
||||
Map<String, Object> selfEvaluation = parseJsonObject(session.getSelfEvaluation());
|
||||
return DiagnosisTraceResponse.TraceSummary.builder()
|
||||
.resolvedRunId(null)
|
||||
.persistedStepCount(defaultInt(session.getStepCount()))
|
||||
.returnedStepCount(steps.size())
|
||||
.persistedToolCallCount(defaultInt(session.getToolCallCount()))
|
||||
@@ -130,6 +263,24 @@ public class DiagnosisTraceService {
|
||||
.build();
|
||||
}
|
||||
|
||||
private DiagnosisTraceResponse.TraceSummary toSummary(
|
||||
DiagnosisTraceResponse.RunTrace run,
|
||||
List<AgentStep> steps,
|
||||
List<ToolInvocation> toolInvocations
|
||||
) {
|
||||
Map<String, Object> selfEvaluation = run.getSelfEvaluation();
|
||||
return DiagnosisTraceResponse.TraceSummary.builder()
|
||||
.resolvedRunId(run.getRunId())
|
||||
.persistedStepCount(defaultInt(run.getStepCount()))
|
||||
.returnedStepCount(steps.size())
|
||||
.persistedToolCallCount(defaultInt(run.getToolCallCount()))
|
||||
.returnedToolCallCount(toolInvocations.size())
|
||||
.hasVerifierEvaluation(selfEvaluation != null && selfEvaluation.containsKey("verifier_evaluation"))
|
||||
.hasAiOpsRuleEvaluation(selfEvaluation != null && selfEvaluation.containsKey("aiops_rule_evaluation"))
|
||||
.hasFeedback(run.getFeedback() != null && !run.getFeedback().isBlank())
|
||||
.build();
|
||||
}
|
||||
|
||||
private Map<String, Object> parseJsonObject(String json) {
|
||||
if (json == null || json.isBlank()) {
|
||||
return null;
|
||||
@@ -144,4 +295,14 @@ public class DiagnosisTraceService {
|
||||
private int defaultInt(Integer value) {
|
||||
return value == null ? 0 : value;
|
||||
}
|
||||
|
||||
private String preview(String text, int maxLength) {
|
||||
if (text == null) {
|
||||
return null;
|
||||
}
|
||||
if (text.length() <= maxLength) {
|
||||
return text;
|
||||
}
|
||||
return text.substring(0, maxLength);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,11 +2,15 @@ package com.superbiz.agent.service;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.domain.entity.AgentStep;
|
||||
import com.superbiz.agent.domain.entity.ChatSession;
|
||||
import com.superbiz.agent.domain.entity.DiagnosisRun;
|
||||
import com.superbiz.agent.domain.entity.DiagnosisSession;
|
||||
import com.superbiz.agent.domain.entity.ToolInvocation;
|
||||
import com.superbiz.agent.dto.DiagnosisTraceResponse;
|
||||
import com.superbiz.agent.exception.SessionNotFoundException;
|
||||
import com.superbiz.agent.repository.AgentStepRepository;
|
||||
import com.superbiz.agent.repository.ChatSessionRepository;
|
||||
import com.superbiz.agent.repository.DiagnosisRunRepository;
|
||||
import com.superbiz.agent.repository.DiagnosisSessionRepository;
|
||||
import com.superbiz.agent.repository.ToolInvocationRepository;
|
||||
import org.junit.jupiter.api.Test;
|
||||
@@ -15,148 +19,245 @@ import java.time.LocalDateTime;
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
import static org.mockito.Mockito.*;
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertNull;
|
||||
import static org.junit.jupiter.api.Assertions.assertThrows;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
import static org.mockito.ArgumentMatchers.any;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.never;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.verifyNoInteractions;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
class DiagnosisTraceServiceTest {
|
||||
|
||||
private final DiagnosisSessionRepository diagnosisSessionRepository = mock(DiagnosisSessionRepository.class);
|
||||
private final ChatSessionRepository chatSessionRepository = mock(ChatSessionRepository.class);
|
||||
private final DiagnosisRunRepository diagnosisRunRepository = mock(DiagnosisRunRepository.class);
|
||||
private final AgentStepRepository agentStepRepository = mock(AgentStepRepository.class);
|
||||
private final ToolInvocationRepository toolInvocationRepository = mock(ToolInvocationRepository.class);
|
||||
private final DiagnosisTraceService service = new DiagnosisTraceService(
|
||||
diagnosisSessionRepository,
|
||||
chatSessionRepository,
|
||||
diagnosisRunRepository,
|
||||
agentStepRepository,
|
||||
toolInvocationRepository,
|
||||
new ObjectMapper()
|
||||
);
|
||||
|
||||
@Test
|
||||
void getTraceAggregatesSessionStepsAndTools() {
|
||||
String sessionId = "trace-session-001";
|
||||
LocalDateTime now = LocalDateTime.of(2026, 7, 3, 14, 30);
|
||||
DiagnosisSession session = DiagnosisSession.builder()
|
||||
.id(1L)
|
||||
.sessionId(sessionId)
|
||||
.query("payment timeout")
|
||||
.status("SUCCESS")
|
||||
.agentFlow("COMPLEX")
|
||||
.totalDurationMs(1200)
|
||||
.totalTokenCount(300)
|
||||
.stepCount(2)
|
||||
.toolCallCount(1)
|
||||
.answer("restart payment gateway pool")
|
||||
.selfEvaluation("{\"verifier_evaluation\":{\"verdict\":\"PASS\"},\"aiops_rule_evaluation\":{\"verdict\":\"WARN\"}}")
|
||||
.feedback("useful")
|
||||
.createdAt(now)
|
||||
.updatedAt(now)
|
||||
.build();
|
||||
AgentStep step = AgentStep.builder()
|
||||
.id(10L)
|
||||
.sessionId(sessionId)
|
||||
.stepIndex(1)
|
||||
.agentName("chat_executor")
|
||||
.modelInput("input")
|
||||
.modelOutput("output")
|
||||
.thought("executor finished")
|
||||
.hasToolCall(true)
|
||||
.durationMs(500)
|
||||
.tokenCount(100)
|
||||
.createdAt(now)
|
||||
.build();
|
||||
ToolInvocation invocation = ToolInvocation.builder()
|
||||
.id(20L)
|
||||
.sessionId(sessionId)
|
||||
.stepId(10L)
|
||||
.toolName("lookup_knowledge")
|
||||
.inputParams("{\"query\":\"ERR_TIMEOUT\"}")
|
||||
.outputPreview("payment timeout doc")
|
||||
.outputLength(19)
|
||||
.retrievalLayer("L0")
|
||||
.l0MatchCount(1)
|
||||
.l1MatchCount(0)
|
||||
.isTruncated(false)
|
||||
.relevanceLevel("HIGHLY_RELEVANT")
|
||||
.dedupReason("FIRST_HIT")
|
||||
.retrievalDetails("{\"documents\":[\"payment-errors.md\"]}")
|
||||
.durationMs(80)
|
||||
.success(true)
|
||||
.createdAt(now)
|
||||
.build();
|
||||
void getTraceWithoutRunIdResolvesLatestRun() {
|
||||
String sessionId = "trace-session-latest";
|
||||
LocalDateTime base = LocalDateTime.of(2026, 7, 10, 10, 0);
|
||||
DiagnosisRun latest = run(2L, sessionId, "run-latest", "second question", base.plusMinutes(1));
|
||||
AgentStep step = step(20L, sessionId, "run-latest", 0, "composer", base.plusMinutes(1));
|
||||
ToolInvocation invocation = invocation(30L, sessionId, "run-latest", "query_metrics", base.plusMinutes(1));
|
||||
|
||||
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(session));
|
||||
when(agentStepRepository.findBySessionId(sessionId)).thenReturn(List.of(step));
|
||||
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of(invocation));
|
||||
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
|
||||
.thenReturn(Optional.of(latest));
|
||||
when(chatSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(chatSession(sessionId)));
|
||||
when(agentStepRepository.findByRunIdOrderByStepIndex("run-latest")).thenReturn(List.of(step));
|
||||
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-latest")).thenReturn(List.of(invocation));
|
||||
|
||||
DiagnosisTraceResponse response = service.getTrace(sessionId);
|
||||
|
||||
assertEquals(sessionId, response.getSession().getSessionId());
|
||||
assertEquals("payment timeout", response.getSession().getQuery());
|
||||
assertEquals("PASS", ((java.util.Map<?, ?>) response.getSession()
|
||||
.getSelfEvaluation()
|
||||
.get("verifier_evaluation")).get("verdict"));
|
||||
assertEquals(1, response.getSteps().size());
|
||||
assertEquals("chat_executor", response.getSteps().get(0).getAgentName());
|
||||
assertEquals(1, response.getToolInvocations().size());
|
||||
assertEquals("ERR_TIMEOUT", response.getToolInvocations().get(0).getInputParams().get("query"));
|
||||
assertEquals(2, response.getSummary().getPersistedStepCount());
|
||||
assertEquals(1, response.getSummary().getReturnedStepCount());
|
||||
assertEquals(1, response.getSummary().getPersistedToolCallCount());
|
||||
assertEquals(1, response.getSummary().getReturnedToolCallCount());
|
||||
assertTrue(response.getSummary().isHasVerifierEvaluation());
|
||||
assertTrue(response.getSummary().isHasAiOpsRuleEvaluation());
|
||||
assertTrue(response.getSummary().isHasFeedback());
|
||||
assertEquals("run-latest", response.getRunId());
|
||||
assertEquals("run-latest", response.getRun().getRunId());
|
||||
assertEquals("run-latest", response.getSummary().getResolvedRunId());
|
||||
assertEquals("second question", response.getSession().getQuery());
|
||||
assertEquals("run-latest", response.getSteps().get(0).getRunId());
|
||||
assertEquals("run-latest", response.getToolInvocations().get(0).getRunId());
|
||||
assertEquals(1, response.getChatSession().getMessagePairCount());
|
||||
}
|
||||
|
||||
@Test
|
||||
void getTraceOrdersStepsByCreationTimeAndIdNotPerAgentStepIndex() {
|
||||
String sessionId = "trace-session-ordered";
|
||||
LocalDateTime base = LocalDateTime.of(2026, 7, 7, 0, 0);
|
||||
DiagnosisSession session = DiagnosisSession.builder()
|
||||
.id(1L)
|
||||
.sessionId(sessionId)
|
||||
.query("mysql pool issue")
|
||||
.status("SUCCESS")
|
||||
.stepCount(4)
|
||||
.toolCallCount(0)
|
||||
.createdAt(base)
|
||||
.updatedAt(base)
|
||||
.build();
|
||||
AgentStep planner = step(1L, sessionId, 0, "planner", base.plusSeconds(1));
|
||||
AgentStep executor0 = step(2L, sessionId, 0, "executor", base.plusSeconds(2));
|
||||
AgentStep executor1 = step(3L, sessionId, 1, "executor", base.plusSeconds(3));
|
||||
AgentStep verifier = step(4L, sessionId, 0, "verifier", base.plusSeconds(4));
|
||||
void getTraceWithRunIdReturnsExactFirstRun() {
|
||||
String sessionId = "trace-session-exact";
|
||||
DiagnosisRun first = run(1L, sessionId, "run-first", "first question",
|
||||
LocalDateTime.of(2026, 7, 10, 10, 0));
|
||||
AgentStep firstStep = step(10L, sessionId, "run-first", 0, "planner", first.getCreatedAt());
|
||||
|
||||
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(session));
|
||||
when(agentStepRepository.findBySessionId(sessionId))
|
||||
.thenReturn(List.of(planner, executor0, verifier, executor1));
|
||||
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of());
|
||||
when(diagnosisRunRepository.findBySessionIdAndRunId(sessionId, "run-first"))
|
||||
.thenReturn(Optional.of(first));
|
||||
when(agentStepRepository.findByRunIdOrderByStepIndex("run-first")).thenReturn(List.of(firstStep));
|
||||
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-first")).thenReturn(List.of());
|
||||
|
||||
DiagnosisTraceResponse response = service.getTrace(sessionId);
|
||||
DiagnosisTraceResponse response = service.getTrace(sessionId, "run-first");
|
||||
|
||||
assertEquals(List.of("planner", "executor", "executor", "verifier"),
|
||||
assertEquals("run-first", response.getRunId());
|
||||
assertEquals("first question", response.getRun().getQuery());
|
||||
assertEquals(List.of("planner"),
|
||||
response.getSteps().stream().map(DiagnosisTraceResponse.AgentStepTrace::getAgentName).toList());
|
||||
assertEquals(List.of(0, 0, 1, 0),
|
||||
response.getSteps().stream().map(DiagnosisTraceResponse.AgentStepTrace::getStepIndex).toList());
|
||||
verify(diagnosisRunRepository, never()).findFirstBySessionIdOrderByCreatedAtDescIdDesc(any());
|
||||
}
|
||||
|
||||
@Test
|
||||
void getTraceWithRunIdReturnsExactSecondRun() {
|
||||
String sessionId = "trace-session-exact";
|
||||
DiagnosisRun second = run(2L, sessionId, "run-second", "second question",
|
||||
LocalDateTime.of(2026, 7, 10, 10, 1));
|
||||
ToolInvocation secondTool = invocation(20L, sessionId, "run-second", "query_logs", second.getCreatedAt());
|
||||
|
||||
when(diagnosisRunRepository.findBySessionIdAndRunId(sessionId, "run-second"))
|
||||
.thenReturn(Optional.of(second));
|
||||
when(agentStepRepository.findByRunIdOrderByStepIndex("run-second")).thenReturn(List.of());
|
||||
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-second")).thenReturn(List.of(secondTool));
|
||||
|
||||
DiagnosisTraceResponse response = service.getTrace(sessionId, "run-second");
|
||||
|
||||
assertEquals("run-second", response.getRunId());
|
||||
assertEquals("second question", response.getSession().getQuery());
|
||||
assertEquals(List.of("query_logs"),
|
||||
response.getToolInvocations().stream().map(DiagnosisTraceResponse.ToolInvocationTrace::getToolName).toList());
|
||||
}
|
||||
|
||||
@Test
|
||||
void getTraceRejectsRunFromAnotherSession() {
|
||||
String runId = "run-other-session";
|
||||
when(diagnosisRunRepository.findBySessionIdAndRunId("path-session", runId)).thenReturn(Optional.empty());
|
||||
when(diagnosisRunRepository.findByRunId(runId)).thenReturn(Optional.of(run(
|
||||
1L, "actual-session", runId, "query", LocalDateTime.of(2026, 7, 10, 10, 0))));
|
||||
|
||||
IllegalArgumentException error = assertThrows(IllegalArgumentException.class,
|
||||
() -> service.getTrace("path-session", runId));
|
||||
|
||||
assertTrue(error.getMessage().contains("does not belong"));
|
||||
verifyNoInteractions(agentStepRepository, toolInvocationRepository);
|
||||
}
|
||||
|
||||
@Test
|
||||
void getTraceThrowsWhenSessionMissing() {
|
||||
String sessionId = "missing-session";
|
||||
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
|
||||
.thenReturn(Optional.empty());
|
||||
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.empty());
|
||||
|
||||
assertThrows(SessionNotFoundException.class, () -> service.getTrace(sessionId));
|
||||
|
||||
verify(diagnosisRunRepository).findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId);
|
||||
verify(diagnosisSessionRepository).findBySessionId(sessionId);
|
||||
verifyNoInteractions(agentStepRepository, toolInvocationRepository);
|
||||
}
|
||||
|
||||
private AgentStep step(Long id, String sessionId, int stepIndex, String agentName, LocalDateTime createdAt) {
|
||||
@Test
|
||||
void getTraceIsReadOnly() {
|
||||
String sessionId = "trace-session-readonly";
|
||||
DiagnosisRun latest = run(3L, sessionId, "run-readonly", "readonly question",
|
||||
LocalDateTime.of(2026, 7, 10, 10, 2));
|
||||
|
||||
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
|
||||
.thenReturn(Optional.of(latest));
|
||||
when(agentStepRepository.findByRunIdOrderByStepIndex("run-readonly")).thenReturn(List.of());
|
||||
when(toolInvocationRepository.findByRunIdOrderByIdAsc("run-readonly")).thenReturn(List.of());
|
||||
|
||||
service.getTrace(sessionId);
|
||||
|
||||
verify(chatSessionRepository, never()).save(any());
|
||||
verify(diagnosisRunRepository, never()).save(any());
|
||||
verify(diagnosisSessionRepository, never()).save(any());
|
||||
verify(agentStepRepository, never()).save(any());
|
||||
verify(toolInvocationRepository, never()).save(any());
|
||||
}
|
||||
|
||||
@Test
|
||||
void listRunSummariesDoesNotExpandTraceDetails() {
|
||||
String sessionId = "trace-session-runs";
|
||||
DiagnosisRun second = run(2L, sessionId, "run-second", "second question",
|
||||
LocalDateTime.of(2026, 7, 10, 10, 1));
|
||||
second.setAnswer("answer ".repeat(40));
|
||||
DiagnosisRun first = run(1L, sessionId, "run-first", "first question",
|
||||
LocalDateTime.of(2026, 7, 10, 10, 0));
|
||||
|
||||
when(diagnosisRunRepository.findBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
|
||||
.thenReturn(List.of(second, first));
|
||||
|
||||
List<DiagnosisTraceResponse.RunSummary> summaries = service.listRunSummaries(sessionId);
|
||||
|
||||
assertEquals(List.of("run-second", "run-first"),
|
||||
summaries.stream().map(DiagnosisTraceResponse.RunSummary::getRunId).toList());
|
||||
assertEquals("second question", summaries.get(0).getQuery());
|
||||
assertTrue(summaries.get(0).getAnswerPreview().length() <= 160);
|
||||
verifyNoInteractions(agentStepRepository, toolInvocationRepository);
|
||||
}
|
||||
|
||||
@Test
|
||||
void legacyTraceFallbackKeepsHistoricalSessionReadable() {
|
||||
String sessionId = "legacy-session";
|
||||
DiagnosisSession legacy = DiagnosisSession.builder()
|
||||
.id(1L)
|
||||
.sessionId(sessionId)
|
||||
.query("legacy question")
|
||||
.status("SUCCESS")
|
||||
.stepCount(0)
|
||||
.toolCallCount(0)
|
||||
.createdAt(LocalDateTime.of(2026, 7, 10, 9, 0))
|
||||
.updatedAt(LocalDateTime.of(2026, 7, 10, 9, 1))
|
||||
.build();
|
||||
|
||||
when(diagnosisRunRepository.findFirstBySessionIdOrderByCreatedAtDescIdDesc(sessionId))
|
||||
.thenReturn(Optional.empty());
|
||||
when(diagnosisSessionRepository.findBySessionId(sessionId)).thenReturn(Optional.of(legacy));
|
||||
when(agentStepRepository.findBySessionId(sessionId)).thenReturn(List.of());
|
||||
when(toolInvocationRepository.findBySessionIdOrderByIdAsc(sessionId)).thenReturn(List.of());
|
||||
|
||||
DiagnosisTraceResponse response = service.getTrace(sessionId);
|
||||
|
||||
assertNull(response.getRunId());
|
||||
assertNull(response.getRun());
|
||||
assertEquals("legacy question", response.getSession().getQuery());
|
||||
}
|
||||
|
||||
private DiagnosisRun run(Long id, String sessionId, String runId, String query, LocalDateTime createdAt) {
|
||||
return DiagnosisRun.builder()
|
||||
.id(id)
|
||||
.sessionId(sessionId)
|
||||
.runId(runId)
|
||||
.query(query)
|
||||
.status("SUCCESS")
|
||||
.agentFlow("CHAT")
|
||||
.answer("answer for " + runId)
|
||||
.selfEvaluation("{\"verifier_evaluation\":{\"verdict\":\"PASS\"}}")
|
||||
.feedback("useful")
|
||||
.stepCount(1)
|
||||
.toolCallCount(1)
|
||||
.createdAt(createdAt)
|
||||
.updatedAt(createdAt.plusSeconds(1))
|
||||
.build();
|
||||
}
|
||||
|
||||
private ChatSession chatSession(String sessionId) {
|
||||
return ChatSession.builder()
|
||||
.id(99L)
|
||||
.sessionId(sessionId)
|
||||
.status("ACTIVE")
|
||||
.messagePairCount(1)
|
||||
.createdAt(LocalDateTime.of(2026, 7, 10, 9, 0))
|
||||
.lastActiveAt(LocalDateTime.of(2026, 7, 10, 10, 0))
|
||||
.build();
|
||||
}
|
||||
|
||||
private AgentStep step(Long id, String sessionId, String runId, int stepIndex, String agentName, LocalDateTime createdAt) {
|
||||
return AgentStep.builder()
|
||||
.id(id)
|
||||
.sessionId(sessionId)
|
||||
.runId(runId)
|
||||
.stepIndex(stepIndex)
|
||||
.agentName(agentName)
|
||||
.createdAt(createdAt)
|
||||
.build();
|
||||
}
|
||||
|
||||
private ToolInvocation invocation(Long id, String sessionId, String runId, String toolName, LocalDateTime createdAt) {
|
||||
return ToolInvocation.builder()
|
||||
.id(id)
|
||||
.sessionId(sessionId)
|
||||
.runId(runId)
|
||||
.toolName(toolName)
|
||||
.inputParams("{\"query\":\"timeout\"}")
|
||||
.retrievalDetails("{\"evidence_refs\":[]}")
|
||||
.success(true)
|
||||
.createdAt(createdAt)
|
||||
.build();
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user