feat(trace): add diagnosis trace workbench
This commit is contained in:
aruo
@@ -0,0 +1,70 @@
|
|||||||
|
# Trace UI workbench design
|
||||||
|
|
||||||
|
## Subject and audience
|
||||||
|
|
||||||
|
Subject: a single-session diagnosis trace audit workbench.
|
||||||
|
|
||||||
|
Audience: backend and AIOps engineers reviewing an MVP diagnosis run after a
|
||||||
|
demo, incident drill, or regression check.
|
||||||
|
|
||||||
|
Single job: turn one trace response into an inspectable ledger that exposes
|
||||||
|
agent flow, evidence, verifier judgement, and skill boundaries without reading
|
||||||
|
raw JSON first.
|
||||||
|
|
||||||
|
## Frontend approach
|
||||||
|
|
||||||
|
- Static `trace.html`, `trace.css`, and `trace.js`.
|
||||||
|
- No dependency on a package manager, bundler, external fonts, or remote icons.
|
||||||
|
- Fetches `/api/diagnosis/{sessionId}/trace` and renders client-side.
|
||||||
|
- Accepts `?sessionId=...` and keeps the loaded session id in the URL.
|
||||||
|
|
||||||
|
## Visual direction
|
||||||
|
|
||||||
|
Palette:
|
||||||
|
|
||||||
|
- `#111827` ink rail for the trace frame.
|
||||||
|
- `#f7f3ea` warm ledger surface.
|
||||||
|
- `#1f7a6b` evidence green.
|
||||||
|
- `#b5472f` rejection red.
|
||||||
|
- `#c58a19` warning amber.
|
||||||
|
- `#5b6472` operational gray.
|
||||||
|
|
||||||
|
Type:
|
||||||
|
|
||||||
|
- UI/body: `Inter, ui-sans-serif, system-ui`.
|
||||||
|
- Data and labels: `ui-monospace, SFMono-Regular, Consolas`.
|
||||||
|
|
||||||
|
Layout:
|
||||||
|
|
||||||
|
```text
|
||||||
|
+--------------------------------------------------------------------+
|
||||||
|
| Session id input | Load | status / verdict / counts strip |
|
||||||
|
+------------------+----------------------+--------------------------+
|
||||||
|
| Agent trace rail | Evidence tool ledger | Inspector |
|
||||||
|
| ordered steps | filters + calls | verifier + RAG + skill |
|
||||||
|
+------------------+----------------------+--------------------------+
|
||||||
|
```
|
||||||
|
|
||||||
|
Signature element: a trace rail that treats each agent step as a ledger entry
|
||||||
|
with ordered markers, duration, token count, and expandable raw model excerpts.
|
||||||
|
|
||||||
|
## Data mapping
|
||||||
|
|
||||||
|
- `data.session`: summary strip, final answer, self-evaluation source.
|
||||||
|
- `data.steps`: ordered trace rail.
|
||||||
|
- `data.toolInvocations`: evidence ledger and selected tool inspector.
|
||||||
|
- `data.session.selfEvaluation.verifier_evaluation`: verdict, groundedness,
|
||||||
|
facts table, and verifier `tool_trace_summary`.
|
||||||
|
- `data.toolInvocations[*].retrievalDetails`: RAG query transform, retrieval
|
||||||
|
trace, context pack, rerank trace, and evidence blocks.
|
||||||
|
- `steps[*].modelOutput`: best-effort extraction of `selected_skill`.
|
||||||
|
- `steps[*].modelInput/modelOutput` and tool names: best-effort skill boundary
|
||||||
|
checks for `read_skill`.
|
||||||
|
|
||||||
|
## Accessibility and states
|
||||||
|
|
||||||
|
- Keyboard-focusable controls.
|
||||||
|
- Loading, missing session id, API error, empty steps, empty tools, and missing
|
||||||
|
verifier states.
|
||||||
|
- Responsive three-pane desktop layout that stacks on narrow screens.
|
||||||
|
- Respects `prefers-reduced-motion`.
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Trace UI workbench
|
||||||
|
|
||||||
|
## Why
|
||||||
|
|
||||||
|
The MVP already persists diagnosis trace data and exposes it through
|
||||||
|
`GET /api/diagnosis/{sessionId}/trace`, but reviewers still need to inspect raw
|
||||||
|
JSON to answer basic audit questions:
|
||||||
|
|
||||||
|
- Which agents ran, in what order, and with what persisted counts?
|
||||||
|
- Which evidence tools executed, how many times, and did they succeed?
|
||||||
|
- Which facts did the Verifier check, and which evidence refs support them?
|
||||||
|
- Did Planner only select skill metadata while Executor handled skill loading?
|
||||||
|
- What RAG retrieval details were used by `lookup_knowledge`?
|
||||||
|
|
||||||
|
This slows down demo review and makes trace quality issues harder to spot.
|
||||||
|
|
||||||
|
## What changes
|
||||||
|
|
||||||
|
- Add a static Trace workbench page served by Spring Boot static resources.
|
||||||
|
- Load a diagnosis trace by session id through the existing read-only Trace API.
|
||||||
|
- Render session summary, agent timeline, evidence tool ledger, verifier facts,
|
||||||
|
skill boundary checks, and RAG retrieval details.
|
||||||
|
- Add a navigation entry from the existing chat page to the Trace workbench.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- No new backend endpoint.
|
||||||
|
- No mutation of diagnosis sessions, agent steps, tool invocations, or feedback.
|
||||||
|
- No new frontend framework or build pipeline.
|
||||||
|
- No change to the persisted trace schema.
|
||||||
|
|
||||||
|
## Impact
|
||||||
|
|
||||||
|
- Frontend-only runtime surface under `src/main/resources/static`.
|
||||||
|
- Uses the existing `Result<T>` API response contract.
|
||||||
|
- Works with existing trace records, including sessions that lack verifier or RAG
|
||||||
|
detail fields.
|
||||||
+36
@@ -0,0 +1,36 @@
|
|||||||
|
## ADDED Requirements
|
||||||
|
|
||||||
|
### Requirement: MVP demo SHALL provide a browser trace workbench
|
||||||
|
The MVP demo SHALL provide a browser-accessible static page for inspecting one
|
||||||
|
diagnosis trace by session id using the existing read-only Trace API.
|
||||||
|
|
||||||
|
#### Scenario: Existing trace renders in the workbench
|
||||||
|
- **WHEN** a reviewer opens the Trace workbench with a session id that exists
|
||||||
|
- **THEN** the page SHALL request `GET /api/diagnosis/{sessionId}/trace`
|
||||||
|
- **AND** it SHALL render session summary, ordered agent steps, ordered tool
|
||||||
|
invocations, verifier evaluation, and final answer when present
|
||||||
|
|
||||||
|
#### Scenario: Trace workbench handles missing or failed traces
|
||||||
|
- **WHEN** the Trace API returns an error or the session id is empty
|
||||||
|
- **THEN** the page SHALL show a clear error or empty state without mutating any
|
||||||
|
diagnosis data
|
||||||
|
|
||||||
|
### Requirement: Trace workbench SHALL expose evidence and skill boundaries
|
||||||
|
The Trace workbench SHALL make tool evidence and skill-loading boundaries visible
|
||||||
|
without requiring raw JSON inspection first.
|
||||||
|
|
||||||
|
#### Scenario: Tool and verifier evidence are inspectable
|
||||||
|
- **WHEN** a trace includes tool invocations and verifier facts
|
||||||
|
- **THEN** the page SHALL show tool invocation counts, success status, tool
|
||||||
|
filters, verifier facts, and evidence references
|
||||||
|
|
||||||
|
#### Scenario: RAG details are inspectable for lookup knowledge calls
|
||||||
|
- **WHEN** a `lookup_knowledge` invocation includes retrieval details
|
||||||
|
- **THEN** the page SHALL show query transform, retrieval trace, context pack,
|
||||||
|
rerank trace, and evidence block data where available
|
||||||
|
|
||||||
|
#### Scenario: Skill boundary checks are visible
|
||||||
|
- **WHEN** a trace includes planner, executor, or verifier steps
|
||||||
|
- **THEN** the page SHALL show best-effort indicators for selected skill,
|
||||||
|
planner `read_skill` text mentions, executor `read_skill` text mentions, and
|
||||||
|
verifier `read_skill` text mentions
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# Tasks
|
||||||
|
|
||||||
|
- [x] 1. Add OpenSpec delta for the Trace UI workbench.
|
||||||
|
- [x] 2. Add static Trace workbench HTML/CSS/JS.
|
||||||
|
- [x] 3. Add a chat-page navigation entry to the Trace workbench.
|
||||||
|
- [x] 4. Verify Java compilation and static page syntax.
|
||||||
|
- [x] 5. Validate the page against a real trace endpoint when a local service is available.
|
||||||
|
- [x] 6. Archive the OpenSpec change after validation.
|
||||||
|
- [x] 7. Commit the completed change.
|
||||||
@@ -1,9 +1,7 @@
|
|||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
Provide a repeatable MVP demo flow that can run a chat diagnosis, expose its persisted execution trace, and submit feedback for the same session id.
|
Provide a repeatable MVP demo flow that can run a chat diagnosis, expose its persisted execution trace, and submit feedback for the same session id.
|
||||||
|
|
||||||
## Requirements
|
## Requirements
|
||||||
|
|
||||||
### Requirement: Diagnosis trace can be queried by session id
|
### Requirement: Diagnosis trace can be queried by session id
|
||||||
The system SHALL expose a read-only HTTP endpoint `GET /api/diagnosis/{sessionId}/trace` that returns the persisted diagnosis trace for the requested session id.
|
The system SHALL expose a read-only HTTP endpoint `GET /api/diagnosis/{sessionId}/trace` that returns the persisted diagnosis trace for the requested session id.
|
||||||
|
|
||||||
@@ -64,3 +62,39 @@ The MVP demo SHALL document which trace fields to inspect for evidence, verifier
|
|||||||
#### Scenario: Checklist maps fields to interview claims
|
#### Scenario: Checklist maps fields to interview claims
|
||||||
- **WHEN** a developer reviews a trace response
|
- **WHEN** a developer reviews a trace response
|
||||||
- **THEN** the checklist SHALL map concrete JSON paths to the claims made in the interview walkthrough
|
- **THEN** the checklist SHALL map concrete JSON paths to the claims made in the interview walkthrough
|
||||||
|
|
||||||
|
### Requirement: MVP demo SHALL provide a browser trace workbench
|
||||||
|
The MVP demo SHALL provide a browser-accessible static page for inspecting one
|
||||||
|
diagnosis trace by session id using the existing read-only Trace API.
|
||||||
|
|
||||||
|
#### Scenario: Existing trace renders in the workbench
|
||||||
|
- **WHEN** a reviewer opens the Trace workbench with a session id that exists
|
||||||
|
- **THEN** the page SHALL request `GET /api/diagnosis/{sessionId}/trace`
|
||||||
|
- **AND** it SHALL render session summary, ordered agent steps, ordered tool
|
||||||
|
invocations, verifier evaluation, and final answer when present
|
||||||
|
|
||||||
|
#### Scenario: Trace workbench handles missing or failed traces
|
||||||
|
- **WHEN** the Trace API returns an error or the session id is empty
|
||||||
|
- **THEN** the page SHALL show a clear error or empty state without mutating any
|
||||||
|
diagnosis data
|
||||||
|
|
||||||
|
### Requirement: Trace workbench SHALL expose evidence and skill boundaries
|
||||||
|
The Trace workbench SHALL make tool evidence and skill-loading boundaries visible
|
||||||
|
without requiring raw JSON inspection first.
|
||||||
|
|
||||||
|
#### Scenario: Tool and verifier evidence are inspectable
|
||||||
|
- **WHEN** a trace includes tool invocations and verifier facts
|
||||||
|
- **THEN** the page SHALL show tool invocation counts, success status, tool
|
||||||
|
filters, verifier facts, and evidence references
|
||||||
|
|
||||||
|
#### Scenario: RAG details are inspectable for lookup knowledge calls
|
||||||
|
- **WHEN** a `lookup_knowledge` invocation includes retrieval details
|
||||||
|
- **THEN** the page SHALL show query transform, retrieval trace, context pack,
|
||||||
|
rerank trace, and evidence block data where available
|
||||||
|
|
||||||
|
#### Scenario: Skill boundary checks are visible
|
||||||
|
- **WHEN** a trace includes planner, executor, or verifier steps
|
||||||
|
- **THEN** the page SHALL show best-effort indicators for selected skill,
|
||||||
|
planner `read_skill` text mentions, executor `read_skill` text mentions, and
|
||||||
|
verifier `read_skill` text mentions
|
||||||
|
|
||||||
|
|||||||
@@ -34,6 +34,14 @@
|
|||||||
<span style="font-size: 14px; font-weight: 500;">文档管理</span>
|
<span style="font-size: 14px; font-weight: 500;">文档管理</span>
|
||||||
</a>
|
</a>
|
||||||
|
|
||||||
|
<a href="trace.html" class="sidebar-btn" style="display: flex; align-items: center; gap: 12px; padding: 12px; border-radius: 12px; color: #202124; text-decoration: none; transition: background 0.3s ease; margin-top: 8px;">
|
||||||
|
<svg viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg" style="width: 20px; height: 20px; flex-shrink: 0;">
|
||||||
|
<path d="M4 19V5M4 5H20M4 12H16M4 19H20" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
|
||||||
|
<path d="M8 5V19M16 5V19" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
|
||||||
|
</svg>
|
||||||
|
<span style="font-size: 14px; font-weight: 500;">Trace Workbench</span>
|
||||||
|
</a>
|
||||||
|
|
||||||
<div class="chat-history-section">
|
<div class="chat-history-section">
|
||||||
<div class="history-header">
|
<div class="history-header">
|
||||||
<span>近期对话</span>
|
<span>近期对话</span>
|
||||||
|
|||||||
@@ -0,0 +1,606 @@
|
|||||||
|
:root {
|
||||||
|
--ink: #111827;
|
||||||
|
--ink-soft: #263241;
|
||||||
|
--paper: #f7f3ea;
|
||||||
|
--line: #d6cbbb;
|
||||||
|
--evidence: #1f7a6b;
|
||||||
|
--evidence-soft: #dfeee9;
|
||||||
|
--reject: #b5472f;
|
||||||
|
--reject-soft: #f4dfd8;
|
||||||
|
--warn: #c58a19;
|
||||||
|
--warn-soft: #f5ead1;
|
||||||
|
--steel: #5b6472;
|
||||||
|
--white: #fffdf8;
|
||||||
|
--shadow: 0 18px 48px rgba(17, 24, 39, 0.16);
|
||||||
|
}
|
||||||
|
|
||||||
|
* {
|
||||||
|
box-sizing: border-box;
|
||||||
|
}
|
||||||
|
|
||||||
|
html,
|
||||||
|
body {
|
||||||
|
min-height: 100%;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
margin: 0;
|
||||||
|
background: var(--paper);
|
||||||
|
color: var(--ink);
|
||||||
|
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||||
|
letter-spacing: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
button,
|
||||||
|
input {
|
||||||
|
font: inherit;
|
||||||
|
}
|
||||||
|
|
||||||
|
button {
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
button:focus-visible,
|
||||||
|
input:focus-visible,
|
||||||
|
a:focus-visible,
|
||||||
|
.tool-row:focus-visible {
|
||||||
|
outline: 3px solid rgba(31, 122, 107, 0.34);
|
||||||
|
outline-offset: 2px;
|
||||||
|
}
|
||||||
|
|
||||||
|
svg {
|
||||||
|
width: 18px;
|
||||||
|
height: 18px;
|
||||||
|
fill: none;
|
||||||
|
stroke: currentColor;
|
||||||
|
stroke-width: 2;
|
||||||
|
stroke-linecap: round;
|
||||||
|
stroke-linejoin: round;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-app {
|
||||||
|
min-height: 100vh;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
padding: 22px;
|
||||||
|
gap: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-header {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: minmax(260px, 1fr) minmax(340px, 560px);
|
||||||
|
gap: 18px;
|
||||||
|
align-items: end;
|
||||||
|
padding: 18px 20px;
|
||||||
|
background: var(--ink);
|
||||||
|
color: var(--white);
|
||||||
|
border-radius: 8px;
|
||||||
|
box-shadow: var(--shadow);
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-back-link {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 8px;
|
||||||
|
color: #cbd5e1;
|
||||||
|
text-decoration: none;
|
||||||
|
font-size: 13px;
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-header h1 {
|
||||||
|
margin: 8px 0 0;
|
||||||
|
font-size: clamp(28px, 4vw, 52px);
|
||||||
|
line-height: 0.95;
|
||||||
|
font-weight: 760;
|
||||||
|
letter-spacing: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-form {
|
||||||
|
display: grid;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-form label,
|
||||||
|
.summary-cell span,
|
||||||
|
.pane-heading p,
|
||||||
|
.tool-meta,
|
||||||
|
.state-line,
|
||||||
|
.field-label {
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 12px;
|
||||||
|
text-transform: uppercase;
|
||||||
|
color: var(--steel);
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-form label {
|
||||||
|
color: #cbd5e1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-control {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: minmax(0, 1fr) auto;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-control input {
|
||||||
|
width: 100%;
|
||||||
|
min-width: 0;
|
||||||
|
height: 42px;
|
||||||
|
padding: 0 12px;
|
||||||
|
border: 1px solid #4b5563;
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #0b1220;
|
||||||
|
color: var(--white);
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-control button {
|
||||||
|
height: 42px;
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
gap: 8px;
|
||||||
|
padding: 0 14px;
|
||||||
|
border: 0;
|
||||||
|
border-radius: 6px;
|
||||||
|
color: var(--white);
|
||||||
|
background: var(--evidence);
|
||||||
|
font-weight: 700;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-control button:disabled {
|
||||||
|
opacity: 0.62;
|
||||||
|
cursor: wait;
|
||||||
|
}
|
||||||
|
|
||||||
|
.state-line {
|
||||||
|
min-height: 34px;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
padding: 8px 12px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: var(--white);
|
||||||
|
}
|
||||||
|
|
||||||
|
.state-line.error {
|
||||||
|
color: var(--reject);
|
||||||
|
border-color: #dfb2a5;
|
||||||
|
background: var(--reject-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.state-line.loading {
|
||||||
|
color: var(--evidence);
|
||||||
|
border-color: #a9d2c8;
|
||||||
|
background: var(--evidence-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-strip {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(7, minmax(120px, 1fr));
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 8px;
|
||||||
|
overflow: hidden;
|
||||||
|
background: var(--white);
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell {
|
||||||
|
min-width: 0;
|
||||||
|
padding: 13px 14px;
|
||||||
|
border-right: 1px solid var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell:last-child {
|
||||||
|
border-right: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell strong {
|
||||||
|
display: block;
|
||||||
|
margin-top: 5px;
|
||||||
|
font-size: 21px;
|
||||||
|
line-height: 1.1;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-grid {
|
||||||
|
min-height: 0;
|
||||||
|
flex: 1;
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: minmax(300px, 1.05fr) minmax(300px, 0.95fr) minmax(340px, 1.1fr);
|
||||||
|
gap: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-pane {
|
||||||
|
min-height: 620px;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 8px;
|
||||||
|
background: var(--white);
|
||||||
|
overflow: hidden;
|
||||||
|
}
|
||||||
|
|
||||||
|
.pane-heading {
|
||||||
|
min-height: 74px;
|
||||||
|
display: flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 14px;
|
||||||
|
padding: 14px 16px;
|
||||||
|
border-bottom: 1px solid var(--line);
|
||||||
|
background: #fbf7ef;
|
||||||
|
}
|
||||||
|
|
||||||
|
.pane-heading p,
|
||||||
|
.pane-heading h2 {
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.pane-heading h2 {
|
||||||
|
margin-top: 3px;
|
||||||
|
font-size: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.pane-count {
|
||||||
|
min-width: 38px;
|
||||||
|
min-height: 30px;
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
padding: 5px 9px;
|
||||||
|
border-radius: 999px;
|
||||||
|
background: var(--ink);
|
||||||
|
color: var(--white);
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-rail,
|
||||||
|
.tool-ledger,
|
||||||
|
.inspector-stack {
|
||||||
|
overflow: auto;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-rail {
|
||||||
|
position: relative;
|
||||||
|
padding: 18px 16px 24px 34px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-rail::before {
|
||||||
|
content: "";
|
||||||
|
position: absolute;
|
||||||
|
left: 23px;
|
||||||
|
top: 0;
|
||||||
|
bottom: 0;
|
||||||
|
width: 3px;
|
||||||
|
background: var(--ink);
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-entry {
|
||||||
|
position: relative;
|
||||||
|
margin-bottom: 14px;
|
||||||
|
padding: 12px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #fffaf1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-entry::before {
|
||||||
|
content: attr(data-index);
|
||||||
|
position: absolute;
|
||||||
|
left: -30px;
|
||||||
|
top: 12px;
|
||||||
|
width: 24px;
|
||||||
|
height: 24px;
|
||||||
|
display: grid;
|
||||||
|
place-items: center;
|
||||||
|
border: 2px solid var(--ink);
|
||||||
|
border-radius: 50%;
|
||||||
|
background: var(--paper);
|
||||||
|
color: var(--ink);
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 11px;
|
||||||
|
font-weight: 800;
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-entry.planner {
|
||||||
|
border-left: 5px solid var(--warn);
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-entry.executor {
|
||||||
|
border-left: 5px solid var(--evidence);
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-entry.verifier {
|
||||||
|
border-left: 5px solid var(--reject);
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-title,
|
||||||
|
.tool-title {
|
||||||
|
display: flex;
|
||||||
|
align-items: flex-start;
|
||||||
|
justify-content: space-between;
|
||||||
|
gap: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.step-title h3,
|
||||||
|
.tool-title h3 {
|
||||||
|
margin: 0;
|
||||||
|
font-size: 16px;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
justify-content: center;
|
||||||
|
min-height: 22px;
|
||||||
|
padding: 3px 8px;
|
||||||
|
border-radius: 999px;
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 11px;
|
||||||
|
font-weight: 700;
|
||||||
|
white-space: nowrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge.ok {
|
||||||
|
color: #0b4f44;
|
||||||
|
background: var(--evidence-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge.warn {
|
||||||
|
color: #754d00;
|
||||||
|
background: var(--warn-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.badge.bad {
|
||||||
|
color: #7c2414;
|
||||||
|
background: var(--reject-soft);
|
||||||
|
}
|
||||||
|
|
||||||
|
.meta-line {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 8px;
|
||||||
|
margin-top: 9px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.meta-pill {
|
||||||
|
display: inline-flex;
|
||||||
|
align-items: center;
|
||||||
|
gap: 5px;
|
||||||
|
padding: 4px 7px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 999px;
|
||||||
|
background: var(--white);
|
||||||
|
color: var(--steel);
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 11px;
|
||||||
|
}
|
||||||
|
|
||||||
|
details {
|
||||||
|
margin-top: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
summary {
|
||||||
|
color: var(--ink-soft);
|
||||||
|
font-weight: 700;
|
||||||
|
cursor: pointer;
|
||||||
|
}
|
||||||
|
|
||||||
|
pre,
|
||||||
|
.answer-box {
|
||||||
|
margin: 8px 0 0;
|
||||||
|
padding: 10px;
|
||||||
|
max-height: 280px;
|
||||||
|
overflow: auto;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #fdf8ee;
|
||||||
|
color: #172033;
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 12px;
|
||||||
|
line-height: 1.55;
|
||||||
|
white-space: pre-wrap;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-filters {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 8px;
|
||||||
|
padding: 12px 14px;
|
||||||
|
border-bottom: 1px solid var(--line);
|
||||||
|
background: #fffaf1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-filter {
|
||||||
|
min-height: 30px;
|
||||||
|
padding: 5px 10px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 999px;
|
||||||
|
background: var(--white);
|
||||||
|
color: var(--ink);
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-filter.active {
|
||||||
|
border-color: var(--ink);
|
||||||
|
background: var(--ink);
|
||||||
|
color: var(--white);
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-ledger {
|
||||||
|
padding: 12px 14px 18px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-row {
|
||||||
|
width: 100%;
|
||||||
|
display: block;
|
||||||
|
margin-bottom: 10px;
|
||||||
|
padding: 12px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: var(--white);
|
||||||
|
color: inherit;
|
||||||
|
text-align: left;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-row:hover,
|
||||||
|
.tool-row.selected {
|
||||||
|
border-color: var(--evidence);
|
||||||
|
background: #f5fbf8;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-row.failed {
|
||||||
|
border-color: #e1b4aa;
|
||||||
|
}
|
||||||
|
|
||||||
|
.tool-meta {
|
||||||
|
display: flex;
|
||||||
|
flex-wrap: wrap;
|
||||||
|
gap: 7px;
|
||||||
|
margin-top: 8px;
|
||||||
|
color: var(--steel);
|
||||||
|
text-transform: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.inspector-stack {
|
||||||
|
padding: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.inspector-section {
|
||||||
|
padding: 0 0 16px;
|
||||||
|
margin-bottom: 16px;
|
||||||
|
border-bottom: 1px solid var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.inspector-section:last-child {
|
||||||
|
margin-bottom: 0;
|
||||||
|
border-bottom: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.inspector-section h3 {
|
||||||
|
margin: 0 0 10px;
|
||||||
|
font-size: 15px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.boundary-grid {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.boundary-item,
|
||||||
|
.fact-row,
|
||||||
|
.field-row {
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
background: #fffaf1;
|
||||||
|
}
|
||||||
|
|
||||||
|
.boundary-item {
|
||||||
|
padding: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.boundary-item strong {
|
||||||
|
display: block;
|
||||||
|
margin-top: 4px;
|
||||||
|
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||||
|
font-size: 14px;
|
||||||
|
overflow-wrap: anywhere;
|
||||||
|
}
|
||||||
|
|
||||||
|
.fact-row {
|
||||||
|
padding: 10px;
|
||||||
|
margin-bottom: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.fact-row p {
|
||||||
|
margin: 0 0 8px;
|
||||||
|
line-height: 1.45;
|
||||||
|
}
|
||||||
|
|
||||||
|
.field-grid {
|
||||||
|
display: grid;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.field-row {
|
||||||
|
padding: 10px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.field-label {
|
||||||
|
display: block;
|
||||||
|
margin-bottom: 5px;
|
||||||
|
color: var(--steel);
|
||||||
|
}
|
||||||
|
|
||||||
|
.empty-panel {
|
||||||
|
padding: 14px;
|
||||||
|
border: 1px dashed var(--line);
|
||||||
|
border-radius: 6px;
|
||||||
|
color: var(--steel);
|
||||||
|
background: #fffaf1;
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 1180px) {
|
||||||
|
.summary-strip {
|
||||||
|
grid-template-columns: repeat(4, minmax(120px, 1fr));
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell:nth-child(4) {
|
||||||
|
border-right: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-grid {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-pane {
|
||||||
|
min-height: 420px;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (max-width: 760px) {
|
||||||
|
.trace-app {
|
||||||
|
padding: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.trace-header {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
|
||||||
|
.session-control {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-strip {
|
||||||
|
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell,
|
||||||
|
.summary-cell:nth-child(4) {
|
||||||
|
border-right: 1px solid var(--line);
|
||||||
|
}
|
||||||
|
|
||||||
|
.summary-cell:nth-child(2n) {
|
||||||
|
border-right: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.boundary-grid {
|
||||||
|
grid-template-columns: 1fr;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@media (prefers-reduced-motion: reduce) {
|
||||||
|
*,
|
||||||
|
*::before,
|
||||||
|
*::after {
|
||||||
|
scroll-behavior: auto !important;
|
||||||
|
transition: none !important;
|
||||||
|
animation: none !important;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,129 @@
|
|||||||
|
<!DOCTYPE html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8">
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||||
|
<title>Trace Workbench - SuperBizAgent</title>
|
||||||
|
<link rel="stylesheet" href="trace.css">
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<main class="trace-app">
|
||||||
|
<header class="trace-header">
|
||||||
|
<div>
|
||||||
|
<a class="trace-back-link" href="index.html" aria-label="Back to chat">
|
||||||
|
<svg viewBox="0 0 24 24" aria-hidden="true">
|
||||||
|
<path d="M19 12H5M12 19L5 12L12 5" />
|
||||||
|
</svg>
|
||||||
|
<span>Chat</span>
|
||||||
|
</a>
|
||||||
|
<h1>Trace workbench</h1>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<form class="session-form" id="sessionForm">
|
||||||
|
<label for="sessionIdInput">Session id</label>
|
||||||
|
<div class="session-control">
|
||||||
|
<input id="sessionIdInput" name="sessionId" autocomplete="off" spellcheck="false" placeholder="e2e-mysql-skill-rag-20260706-2355">
|
||||||
|
<button type="submit" id="loadButton">
|
||||||
|
<svg viewBox="0 0 24 24" aria-hidden="true">
|
||||||
|
<path d="M5 12H19M13 6L19 12L13 18" />
|
||||||
|
</svg>
|
||||||
|
<span>Load</span>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<section class="state-line" id="stateLine" role="status">Enter a session id to load a persisted diagnosis trace.</section>
|
||||||
|
|
||||||
|
<section class="summary-strip" id="summaryStrip" aria-label="Trace summary">
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Status</span>
|
||||||
|
<strong id="summaryStatus">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Verdict</span>
|
||||||
|
<strong id="summaryVerdict">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Groundedness</span>
|
||||||
|
<strong id="summaryScore">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Steps</span>
|
||||||
|
<strong id="summarySteps">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Tools</span>
|
||||||
|
<strong id="summaryTools">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Duration</span>
|
||||||
|
<strong id="summaryDuration">-</strong>
|
||||||
|
</div>
|
||||||
|
<div class="summary-cell">
|
||||||
|
<span>Tokens</span>
|
||||||
|
<strong id="summaryTokens">-</strong>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="trace-grid">
|
||||||
|
<section class="trace-pane agent-pane" aria-labelledby="agentPaneTitle">
|
||||||
|
<div class="pane-heading">
|
||||||
|
<div>
|
||||||
|
<p>Agent rail</p>
|
||||||
|
<h2 id="agentPaneTitle">Persisted steps</h2>
|
||||||
|
</div>
|
||||||
|
<span class="pane-count" id="stepCount">0</span>
|
||||||
|
</div>
|
||||||
|
<div class="trace-rail" id="stepsRail"></div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="trace-pane evidence-pane" aria-labelledby="evidencePaneTitle">
|
||||||
|
<div class="pane-heading">
|
||||||
|
<div>
|
||||||
|
<p>Evidence ledger</p>
|
||||||
|
<h2 id="evidencePaneTitle">Tool invocations</h2>
|
||||||
|
</div>
|
||||||
|
<span class="pane-count" id="toolCount">0</span>
|
||||||
|
</div>
|
||||||
|
<div class="tool-filters" id="toolFilters"></div>
|
||||||
|
<div class="tool-ledger" id="toolLedger"></div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="trace-pane inspector-pane" aria-labelledby="inspectorPaneTitle">
|
||||||
|
<div class="pane-heading">
|
||||||
|
<div>
|
||||||
|
<p>Inspector</p>
|
||||||
|
<h2 id="inspectorPaneTitle">Verifier and RAG</h2>
|
||||||
|
</div>
|
||||||
|
<span class="pane-count" id="inspectorMode">idle</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="inspector-stack">
|
||||||
|
<section class="inspector-section" aria-labelledby="skillBoundaryTitle">
|
||||||
|
<h3 id="skillBoundaryTitle">Skill boundary</h3>
|
||||||
|
<div class="boundary-grid" id="skillBoundary"></div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="inspector-section" aria-labelledby="verifierTitle">
|
||||||
|
<h3 id="verifierTitle">Verifier facts</h3>
|
||||||
|
<div id="verifierFacts"></div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="inspector-section" aria-labelledby="toolInspectorTitle">
|
||||||
|
<h3 id="toolInspectorTitle">Selected tool</h3>
|
||||||
|
<div id="toolInspector"></div>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="inspector-section" aria-labelledby="answerTitle">
|
||||||
|
<h3 id="answerTitle">Final answer</h3>
|
||||||
|
<div class="answer-box" id="finalAnswer">-</div>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
</section>
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<script src="trace.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
@@ -0,0 +1,574 @@
|
|||||||
|
class TraceWorkbench {
|
||||||
|
constructor() {
|
||||||
|
this.trace = null;
|
||||||
|
this.selectedToolId = null;
|
||||||
|
this.activeFilter = 'all';
|
||||||
|
|
||||||
|
this.sessionForm = document.getElementById('sessionForm');
|
||||||
|
this.sessionIdInput = document.getElementById('sessionIdInput');
|
||||||
|
this.loadButton = document.getElementById('loadButton');
|
||||||
|
this.stateLine = document.getElementById('stateLine');
|
||||||
|
this.stepsRail = document.getElementById('stepsRail');
|
||||||
|
this.toolLedger = document.getElementById('toolLedger');
|
||||||
|
this.toolFilters = document.getElementById('toolFilters');
|
||||||
|
this.skillBoundary = document.getElementById('skillBoundary');
|
||||||
|
this.verifierFacts = document.getElementById('verifierFacts');
|
||||||
|
this.toolInspector = document.getElementById('toolInspector');
|
||||||
|
this.finalAnswer = document.getElementById('finalAnswer');
|
||||||
|
|
||||||
|
this.bindEvents();
|
||||||
|
this.bootstrapFromUrl();
|
||||||
|
}
|
||||||
|
|
||||||
|
bindEvents() {
|
||||||
|
this.sessionForm.addEventListener('submit', (event) => {
|
||||||
|
event.preventDefault();
|
||||||
|
this.loadTrace(this.sessionIdInput.value.trim());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
bootstrapFromUrl() {
|
||||||
|
const params = new URLSearchParams(window.location.search);
|
||||||
|
const sessionId = params.get('sessionId');
|
||||||
|
if (sessionId) {
|
||||||
|
this.sessionIdInput.value = sessionId;
|
||||||
|
this.loadTrace(sessionId);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
this.renderEmpty();
|
||||||
|
}
|
||||||
|
|
||||||
|
async loadTrace(sessionId) {
|
||||||
|
if (!sessionId) {
|
||||||
|
this.setState('Enter a session id before loading.', 'error');
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.setState(`Loading ${sessionId}...`, 'loading');
|
||||||
|
this.loadButton.disabled = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await fetch(`/api/diagnosis/${encodeURIComponent(sessionId)}/trace`);
|
||||||
|
const data = await this.handleResponse(response);
|
||||||
|
this.trace = data;
|
||||||
|
this.selectedToolId = data.toolInvocations && data.toolInvocations.length
|
||||||
|
? data.toolInvocations[0].id
|
||||||
|
: null;
|
||||||
|
this.activeFilter = 'all';
|
||||||
|
this.updateUrl(sessionId);
|
||||||
|
this.renderTrace();
|
||||||
|
this.setState(`Loaded ${sessionId}.`, '');
|
||||||
|
} catch (error) {
|
||||||
|
this.trace = null;
|
||||||
|
this.renderEmpty();
|
||||||
|
this.setState(error.message, 'error');
|
||||||
|
} finally {
|
||||||
|
this.loadButton.disabled = false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async handleResponse(response) {
|
||||||
|
let payload = null;
|
||||||
|
try {
|
||||||
|
payload = await response.json();
|
||||||
|
} catch (error) {
|
||||||
|
throw new Error(`HTTP ${response.status}: response is not JSON`);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!response.ok || payload.code !== 200) {
|
||||||
|
throw new Error(payload.message || `HTTP ${response.status}: trace request failed`);
|
||||||
|
}
|
||||||
|
|
||||||
|
return payload.data;
|
||||||
|
}
|
||||||
|
|
||||||
|
updateUrl(sessionId) {
|
||||||
|
const url = new URL(window.location.href);
|
||||||
|
url.searchParams.set('sessionId', sessionId);
|
||||||
|
window.history.replaceState({}, '', url.toString());
|
||||||
|
}
|
||||||
|
|
||||||
|
renderTrace() {
|
||||||
|
const session = this.trace.session || {};
|
||||||
|
const verifier = this.getVerifier();
|
||||||
|
const summary = this.trace.summary || {};
|
||||||
|
const steps = this.trace.steps || [];
|
||||||
|
const tools = this.trace.toolInvocations || [];
|
||||||
|
|
||||||
|
this.text('summaryStatus', session.status || '-');
|
||||||
|
this.text('summaryVerdict', verifier.verdict || '-');
|
||||||
|
this.text('summaryScore', this.formatScore(verifier.groundedness_score));
|
||||||
|
this.text('summarySteps', `${summary.returnedStepCount ?? steps.length}/${summary.persistedStepCount ?? session.stepCount ?? '-'}`);
|
||||||
|
this.text('summaryTools', `${summary.returnedToolCallCount ?? tools.length}/${summary.persistedToolCallCount ?? session.toolCallCount ?? '-'}`);
|
||||||
|
this.text('summaryDuration', this.formatDuration(session.totalDurationMs));
|
||||||
|
this.text('summaryTokens', this.formatNumber(session.totalTokenCount));
|
||||||
|
this.text('stepCount', steps.length);
|
||||||
|
this.text('toolCount', tools.length);
|
||||||
|
this.text('inspectorMode', tools.length ? 'tool' : 'trace');
|
||||||
|
|
||||||
|
this.renderSteps(steps);
|
||||||
|
this.renderToolFilters(tools);
|
||||||
|
this.renderTools(tools);
|
||||||
|
this.renderSkillBoundary();
|
||||||
|
this.renderVerifier(verifier);
|
||||||
|
this.renderToolInspector();
|
||||||
|
this.finalAnswer.textContent = session.answer || 'No final answer persisted.';
|
||||||
|
}
|
||||||
|
|
||||||
|
renderEmpty() {
|
||||||
|
this.text('summaryStatus', '-');
|
||||||
|
this.text('summaryVerdict', '-');
|
||||||
|
this.text('summaryScore', '-');
|
||||||
|
this.text('summarySteps', '-');
|
||||||
|
this.text('summaryTools', '-');
|
||||||
|
this.text('summaryDuration', '-');
|
||||||
|
this.text('summaryTokens', '-');
|
||||||
|
this.text('stepCount', '0');
|
||||||
|
this.text('toolCount', '0');
|
||||||
|
this.text('inspectorMode', 'idle');
|
||||||
|
|
||||||
|
this.stepsRail.innerHTML = '<div class="empty-panel">No trace loaded.</div>';
|
||||||
|
this.toolFilters.innerHTML = '';
|
||||||
|
this.toolLedger.innerHTML = '<div class="empty-panel">No tool invocations loaded.</div>';
|
||||||
|
this.skillBoundary.innerHTML = this.renderBoundaryItems({
|
||||||
|
selectedSkill: '-',
|
||||||
|
plannerReadSkill: 0,
|
||||||
|
executorReadSkill: 0,
|
||||||
|
verifierReadSkill: 0
|
||||||
|
});
|
||||||
|
this.verifierFacts.innerHTML = '<div class="empty-panel">No verifier evaluation loaded.</div>';
|
||||||
|
this.toolInspector.innerHTML = '<div class="empty-panel">Select a tool invocation after loading a trace.</div>';
|
||||||
|
this.finalAnswer.textContent = '-';
|
||||||
|
}
|
||||||
|
|
||||||
|
renderSteps(steps) {
|
||||||
|
if (!steps.length) {
|
||||||
|
this.stepsRail.innerHTML = '<div class="empty-panel">No persisted agent steps.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.stepsRail.innerHTML = steps.map((step, index) => {
|
||||||
|
const agent = (step.agentName || 'agent').toLowerCase();
|
||||||
|
const cls = this.agentClass(agent);
|
||||||
|
return `
|
||||||
|
<article class="step-entry ${cls}" data-index="${this.escape(String(index + 1).padStart(2, '0'))}">
|
||||||
|
<div class="step-title">
|
||||||
|
<h3>${this.escape(step.agentName || 'agent')}</h3>
|
||||||
|
${this.renderBadge(step.hasToolCall ? 'tool' : 'model', step.hasToolCall ? 'warn' : 'ok')}
|
||||||
|
</div>
|
||||||
|
<div class="meta-line">
|
||||||
|
<span class="meta-pill">step ${this.escape(step.stepIndex ?? index)}</span>
|
||||||
|
<span class="meta-pill">${this.escape(this.formatDuration(step.durationMs))}</span>
|
||||||
|
<span class="meta-pill">${this.escape(this.formatNumber(step.tokenCount))} tokens</span>
|
||||||
|
<span class="meta-pill">${this.escape(this.formatDate(step.createdAt))}</span>
|
||||||
|
</div>
|
||||||
|
${this.renderTextDetail('Model input', step.modelInput)}
|
||||||
|
${this.renderTextDetail('Model output', step.modelOutput)}
|
||||||
|
${this.renderTextDetail('Thought', step.thought)}
|
||||||
|
</article>
|
||||||
|
`;
|
||||||
|
}).join('');
|
||||||
|
}
|
||||||
|
|
||||||
|
renderToolFilters(tools) {
|
||||||
|
if (!tools.length) {
|
||||||
|
this.toolFilters.innerHTML = '';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const groups = this.groupByTool(tools);
|
||||||
|
const filters = [
|
||||||
|
{ key: 'all', label: `all ${tools.length}` },
|
||||||
|
{ key: 'success', label: `success ${tools.filter(tool => tool.success !== false).length}` },
|
||||||
|
{ key: 'failed', label: `failed ${tools.filter(tool => tool.success === false).length}` },
|
||||||
|
...Object.entries(groups).map(([name, count]) => ({ key: `tool:${name}`, label: `${name} ${count}` }))
|
||||||
|
];
|
||||||
|
|
||||||
|
this.toolFilters.innerHTML = filters.map(filter => `
|
||||||
|
<button class="tool-filter ${filter.key === this.activeFilter ? 'active' : ''}" data-filter="${this.escape(filter.key)}">
|
||||||
|
${this.escape(filter.label)}
|
||||||
|
</button>
|
||||||
|
`).join('');
|
||||||
|
|
||||||
|
this.toolFilters.querySelectorAll('button').forEach((button) => {
|
||||||
|
button.addEventListener('click', () => {
|
||||||
|
this.activeFilter = button.dataset.filter;
|
||||||
|
this.renderToolFilters(this.trace.toolInvocations || []);
|
||||||
|
this.renderTools(this.trace.toolInvocations || []);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
renderTools(tools) {
|
||||||
|
const filtered = tools.filter(tool => this.matchesFilter(tool));
|
||||||
|
if (!filtered.length) {
|
||||||
|
this.toolLedger.innerHTML = '<div class="empty-panel">No tool invocations match this filter.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.toolLedger.innerHTML = filtered.map((tool, index) => {
|
||||||
|
const failed = tool.success === false;
|
||||||
|
const selected = String(tool.id) === String(this.selectedToolId);
|
||||||
|
return `
|
||||||
|
<button class="tool-row ${failed ? 'failed' : ''} ${selected ? 'selected' : ''}" data-tool-id="${this.escape(tool.id)}">
|
||||||
|
<div class="tool-title">
|
||||||
|
<h3>${this.escape(tool.toolName || 'tool')}</h3>
|
||||||
|
${this.renderBadge(failed ? 'failed' : 'ok', failed ? 'bad' : 'ok')}
|
||||||
|
</div>
|
||||||
|
<div class="tool-meta">
|
||||||
|
<span>#${index + 1}</span>
|
||||||
|
<span>id ${this.escape(tool.id ?? '-')}</span>
|
||||||
|
<span>step ${this.escape(tool.stepId ?? '-')}</span>
|
||||||
|
<span>${this.escape(this.formatDuration(tool.durationMs))}</span>
|
||||||
|
${tool.retrievalLayer ? `<span>${this.escape(tool.retrievalLayer)}</span>` : ''}
|
||||||
|
${tool.relevanceLevel ? `<span>${this.escape(tool.relevanceLevel)}</span>` : ''}
|
||||||
|
</div>
|
||||||
|
</button>
|
||||||
|
`;
|
||||||
|
}).join('');
|
||||||
|
|
||||||
|
this.toolLedger.querySelectorAll('.tool-row').forEach((row) => {
|
||||||
|
row.addEventListener('click', () => {
|
||||||
|
this.selectedToolId = row.dataset.toolId;
|
||||||
|
this.renderTools(this.trace.toolInvocations || []);
|
||||||
|
this.renderToolInspector();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
renderSkillBoundary() {
|
||||||
|
const steps = this.trace.steps || [];
|
||||||
|
const selectedSkill = this.extractSelectedSkill(steps);
|
||||||
|
const counts = {
|
||||||
|
selectedSkill: selectedSkill || '-',
|
||||||
|
plannerReadSkill: this.countReadSkill(steps, 'planner'),
|
||||||
|
executorReadSkill: this.countReadSkill(steps, 'executor'),
|
||||||
|
verifierReadSkill: this.countReadSkill(steps, 'verifier')
|
||||||
|
};
|
||||||
|
this.skillBoundary.innerHTML = this.renderBoundaryItems(counts);
|
||||||
|
}
|
||||||
|
|
||||||
|
renderBoundaryItems(counts) {
|
||||||
|
return `
|
||||||
|
<div class="boundary-item">
|
||||||
|
<span class="field-label">Selected skill</span>
|
||||||
|
<strong>${this.escape(counts.selectedSkill)}</strong>
|
||||||
|
</div>
|
||||||
|
<div class="boundary-item">
|
||||||
|
<span class="field-label">Planner mentions</span>
|
||||||
|
<strong>${this.escape(counts.plannerReadSkill)}</strong>
|
||||||
|
</div>
|
||||||
|
<div class="boundary-item">
|
||||||
|
<span class="field-label">Executor mentions</span>
|
||||||
|
<strong>${this.escape(counts.executorReadSkill)}</strong>
|
||||||
|
</div>
|
||||||
|
<div class="boundary-item">
|
||||||
|
<span class="field-label">Verifier mentions</span>
|
||||||
|
<strong>${this.escape(counts.verifierReadSkill)}</strong>
|
||||||
|
</div>
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
renderVerifier(verifier) {
|
||||||
|
if (!verifier || !Object.keys(verifier).length) {
|
||||||
|
this.verifierFacts.innerHTML = '<div class="empty-panel">No verifier evaluation in selfEvaluation.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const facts = Array.isArray(verifier.facts_checked) ? verifier.facts_checked : [];
|
||||||
|
const traceSummary = Array.isArray(verifier.tool_trace_summary) ? verifier.tool_trace_summary : [];
|
||||||
|
const header = `
|
||||||
|
<div class="meta-line">
|
||||||
|
${this.renderBadge(verifier.verdict || 'verdict -', this.verdictClass(verifier.verdict))}
|
||||||
|
<span class="meta-pill">score ${this.escape(this.formatScore(verifier.groundedness_score))}</span>
|
||||||
|
<span class="meta-pill">facts ${facts.length}</span>
|
||||||
|
<span class="meta-pill">trace refs ${traceSummary.length}</span>
|
||||||
|
</div>
|
||||||
|
`;
|
||||||
|
|
||||||
|
if (!facts.length) {
|
||||||
|
this.verifierFacts.innerHTML = header + '<div class="empty-panel">Verifier returned no facts_checked rows.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.verifierFacts.innerHTML = header + facts.map((fact) => {
|
||||||
|
const refs = Array.isArray(fact.evidence_refs) ? fact.evidence_refs : [];
|
||||||
|
return `
|
||||||
|
<article class="fact-row">
|
||||||
|
<p>${this.escape(fact.fact || '-')}</p>
|
||||||
|
<div class="meta-line">
|
||||||
|
${this.renderBadge(fact.verification || '-', this.factClass(fact.verification))}
|
||||||
|
<span class="meta-pill">${fact.is_critical ? 'critical' : 'supporting'}</span>
|
||||||
|
<span class="meta-pill">refs ${refs.length}</span>
|
||||||
|
</div>
|
||||||
|
${fact.detail ? `<pre>${this.escape(fact.detail)}</pre>` : ''}
|
||||||
|
${refs.length ? `<pre>${this.escape(this.prettyJson(refs))}</pre>` : ''}
|
||||||
|
</article>
|
||||||
|
`;
|
||||||
|
}).join('');
|
||||||
|
}
|
||||||
|
|
||||||
|
renderToolInspector() {
|
||||||
|
const tools = this.trace ? (this.trace.toolInvocations || []) : [];
|
||||||
|
const tool = tools.find(item => String(item.id) === String(this.selectedToolId));
|
||||||
|
if (!tool) {
|
||||||
|
this.toolInspector.innerHTML = '<div class="empty-panel">No tool invocation selected.</div>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const retrieval = tool.retrievalDetails || {};
|
||||||
|
const ragFields = [
|
||||||
|
['Input params', tool.inputParams || this.parseJson(tool.inputParamsRaw)],
|
||||||
|
['Output preview', tool.outputPreview || '-'],
|
||||||
|
['Retrieval layer', tool.retrievalLayer || '-'],
|
||||||
|
['L0 matches', tool.l0MatchCount ?? '-'],
|
||||||
|
['L1 matches', tool.l1MatchCount ?? '-'],
|
||||||
|
['Relevance', tool.relevanceLevel || '-'],
|
||||||
|
['Dedup reason', tool.dedupReason || '-'],
|
||||||
|
['Query transform', retrieval.query_transform],
|
||||||
|
['Retrieval trace', retrieval.retrieval_trace],
|
||||||
|
['Context pack', retrieval.context_pack_summary],
|
||||||
|
['Rerank trace', retrieval.rerank_trace],
|
||||||
|
['Evidence blocks', retrieval.evidence_blocks]
|
||||||
|
];
|
||||||
|
|
||||||
|
this.toolInspector.innerHTML = `
|
||||||
|
<div class="meta-line">
|
||||||
|
${this.renderBadge(tool.success === false ? 'failed' : 'ok', tool.success === false ? 'bad' : 'ok')}
|
||||||
|
<span class="meta-pill">${this.escape(tool.toolName || 'tool')}</span>
|
||||||
|
<span class="meta-pill">${this.escape(this.formatDuration(tool.durationMs))}</span>
|
||||||
|
<span class="meta-pill">output ${this.escape(this.formatNumber(tool.outputLength))}</span>
|
||||||
|
${tool.truncated ? '<span class="meta-pill">truncated</span>' : ''}
|
||||||
|
</div>
|
||||||
|
${tool.errorMessage ? `<pre>${this.escape(tool.errorMessage)}</pre>` : ''}
|
||||||
|
<div class="field-grid">
|
||||||
|
${ragFields.map(([label, value]) => this.renderField(label, value)).join('')}
|
||||||
|
</div>
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
renderField(label, value) {
|
||||||
|
const empty = value === undefined || value === null || value === '';
|
||||||
|
return `
|
||||||
|
<div class="field-row">
|
||||||
|
<span class="field-label">${this.escape(label)}</span>
|
||||||
|
<pre>${this.escape(empty ? '-' : this.stringifyValue(value))}</pre>
|
||||||
|
</div>
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
renderTextDetail(label, text) {
|
||||||
|
if (!text) {
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
return `
|
||||||
|
<details>
|
||||||
|
<summary>${this.escape(label)}</summary>
|
||||||
|
<pre>${this.escape(text)}</pre>
|
||||||
|
</details>
|
||||||
|
`;
|
||||||
|
}
|
||||||
|
|
||||||
|
renderBadge(text, cls) {
|
||||||
|
return `<span class="badge ${this.escape(cls || 'ok')}">${this.escape(text)}</span>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
getVerifier() {
|
||||||
|
const evaluation = (this.trace && this.trace.session && this.trace.session.selfEvaluation) || {};
|
||||||
|
if (evaluation.verifier_evaluation) {
|
||||||
|
return evaluation.verifier_evaluation;
|
||||||
|
}
|
||||||
|
if (evaluation.verdict || evaluation.groundedness_score || evaluation.facts_checked) {
|
||||||
|
return evaluation;
|
||||||
|
}
|
||||||
|
return {};
|
||||||
|
}
|
||||||
|
|
||||||
|
extractSelectedSkill(steps) {
|
||||||
|
const plannerText = steps
|
||||||
|
.filter(step => (step.agentName || '').toLowerCase().includes('planner'))
|
||||||
|
.map(step => `${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||||
|
.join('\n');
|
||||||
|
const allText = steps
|
||||||
|
.map(step => `${step.modelInput || ''}\n${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||||
|
.join('\n');
|
||||||
|
const sourceText = `${plannerText}\n${allText}`;
|
||||||
|
const direct = sourceText.match(/"selected_skill"\s*:\s*"([^"]+)"/);
|
||||||
|
if (direct) {
|
||||||
|
return direct[1];
|
||||||
|
}
|
||||||
|
const skillName = sourceText.match(/"skill_name"\s*:\s*"([^"]+)"/);
|
||||||
|
if (skillName) {
|
||||||
|
return skillName[1];
|
||||||
|
}
|
||||||
|
const loose = sourceText.match(/selected_skill\s*[:=]\s*([A-Za-z0-9_.-]+)/);
|
||||||
|
return loose ? loose[1] : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
countReadSkill(steps, agentName) {
|
||||||
|
const text = steps
|
||||||
|
.filter(step => (step.agentName || '').toLowerCase().includes(agentName))
|
||||||
|
.map(step => `${step.modelInput || ''}\n${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||||
|
.join('\n');
|
||||||
|
const matches = text.match(/\bread_skill\b/g);
|
||||||
|
return matches ? matches.length : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
matchesFilter(tool) {
|
||||||
|
if (this.activeFilter === 'all') {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (this.activeFilter === 'success') {
|
||||||
|
return tool.success !== false;
|
||||||
|
}
|
||||||
|
if (this.activeFilter === 'failed') {
|
||||||
|
return tool.success === false;
|
||||||
|
}
|
||||||
|
if (this.activeFilter.startsWith('tool:')) {
|
||||||
|
return tool.toolName === this.activeFilter.slice(5);
|
||||||
|
}
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
groupByTool(tools) {
|
||||||
|
return tools.reduce((groups, tool) => {
|
||||||
|
const name = tool.toolName || 'tool';
|
||||||
|
groups[name] = (groups[name] || 0) + 1;
|
||||||
|
return groups;
|
||||||
|
}, {});
|
||||||
|
}
|
||||||
|
|
||||||
|
agentClass(agent) {
|
||||||
|
if (agent.includes('planner')) {
|
||||||
|
return 'planner';
|
||||||
|
}
|
||||||
|
if (agent.includes('executor')) {
|
||||||
|
return 'executor';
|
||||||
|
}
|
||||||
|
if (agent.includes('verifier')) {
|
||||||
|
return 'verifier';
|
||||||
|
}
|
||||||
|
return '';
|
||||||
|
}
|
||||||
|
|
||||||
|
verdictClass(verdict) {
|
||||||
|
if (!verdict) {
|
||||||
|
return 'warn';
|
||||||
|
}
|
||||||
|
const value = String(verdict).toUpperCase();
|
||||||
|
if (value.includes('PASS')) {
|
||||||
|
return 'ok';
|
||||||
|
}
|
||||||
|
if (value.includes('REJECT')) {
|
||||||
|
return 'bad';
|
||||||
|
}
|
||||||
|
return 'warn';
|
||||||
|
}
|
||||||
|
|
||||||
|
factClass(verification) {
|
||||||
|
if (!verification) {
|
||||||
|
return 'warn';
|
||||||
|
}
|
||||||
|
const value = String(verification).toUpperCase();
|
||||||
|
if (value.includes('SUPPORTED') || value.includes('PASS')) {
|
||||||
|
return 'ok';
|
||||||
|
}
|
||||||
|
if (value.includes('UNSUPPORTED') || value.includes('CONTRADICT')) {
|
||||||
|
return 'bad';
|
||||||
|
}
|
||||||
|
return 'warn';
|
||||||
|
}
|
||||||
|
|
||||||
|
parseJson(raw) {
|
||||||
|
if (!raw || typeof raw !== 'string') {
|
||||||
|
return raw;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return JSON.parse(raw);
|
||||||
|
} catch (error) {
|
||||||
|
return raw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
stringifyValue(value) {
|
||||||
|
if (typeof value === 'string') {
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
return this.prettyJson(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
prettyJson(value) {
|
||||||
|
try {
|
||||||
|
return JSON.stringify(value, null, 2);
|
||||||
|
} catch (error) {
|
||||||
|
return String(value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
formatDuration(ms) {
|
||||||
|
if (ms === undefined || ms === null || ms === '') {
|
||||||
|
return '-';
|
||||||
|
}
|
||||||
|
if (ms < 1000) {
|
||||||
|
return `${ms}ms`;
|
||||||
|
}
|
||||||
|
const seconds = ms / 1000;
|
||||||
|
if (seconds < 60) {
|
||||||
|
return `${seconds.toFixed(1)}s`;
|
||||||
|
}
|
||||||
|
const minutes = Math.floor(seconds / 60);
|
||||||
|
const remaining = Math.round(seconds % 60);
|
||||||
|
return `${minutes}m ${remaining}s`;
|
||||||
|
}
|
||||||
|
|
||||||
|
formatNumber(value) {
|
||||||
|
if (value === undefined || value === null || value === '') {
|
||||||
|
return '-';
|
||||||
|
}
|
||||||
|
return Number(value).toLocaleString('en-US');
|
||||||
|
}
|
||||||
|
|
||||||
|
formatScore(value) {
|
||||||
|
if (value === undefined || value === null || value === '') {
|
||||||
|
return '-';
|
||||||
|
}
|
||||||
|
const number = Number(value);
|
||||||
|
return Number.isFinite(number) ? number.toFixed(2) : String(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
formatDate(value) {
|
||||||
|
if (!value) {
|
||||||
|
return '-';
|
||||||
|
}
|
||||||
|
const date = new Date(value);
|
||||||
|
if (Number.isNaN(date.getTime())) {
|
||||||
|
return String(value);
|
||||||
|
}
|
||||||
|
return date.toLocaleString('en-US', { hour12: false });
|
||||||
|
}
|
||||||
|
|
||||||
|
text(id, value) {
|
||||||
|
const node = document.getElementById(id);
|
||||||
|
if (node) {
|
||||||
|
node.textContent = value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
setState(message, statusClass) {
|
||||||
|
this.stateLine.textContent = message;
|
||||||
|
this.stateLine.className = `state-line ${statusClass || ''}`.trim();
|
||||||
|
}
|
||||||
|
|
||||||
|
escape(value) {
|
||||||
|
return String(value)
|
||||||
|
.replace(/&/g, '&')
|
||||||
|
.replace(/</g, '<')
|
||||||
|
.replace(/>/g, '>')
|
||||||
|
.replace(/"/g, '"')
|
||||||
|
.replace(/'/g, ''');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
document.addEventListener('DOMContentLoaded', () => {
|
||||||
|
new TraceWorkbench();
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user