feat(trace): add diagnosis trace workbench
This commit is contained in:
aruo
@@ -0,0 +1,70 @@
|
||||
# Trace UI workbench design
|
||||
|
||||
## Subject and audience
|
||||
|
||||
Subject: a single-session diagnosis trace audit workbench.
|
||||
|
||||
Audience: backend and AIOps engineers reviewing an MVP diagnosis run after a
|
||||
demo, incident drill, or regression check.
|
||||
|
||||
Single job: turn one trace response into an inspectable ledger that exposes
|
||||
agent flow, evidence, verifier judgement, and skill boundaries without reading
|
||||
raw JSON first.
|
||||
|
||||
## Frontend approach
|
||||
|
||||
- Static `trace.html`, `trace.css`, and `trace.js`.
|
||||
- No dependency on a package manager, bundler, external fonts, or remote icons.
|
||||
- Fetches `/api/diagnosis/{sessionId}/trace` and renders client-side.
|
||||
- Accepts `?sessionId=...` and keeps the loaded session id in the URL.
|
||||
|
||||
## Visual direction
|
||||
|
||||
Palette:
|
||||
|
||||
- `#111827` ink rail for the trace frame.
|
||||
- `#f7f3ea` warm ledger surface.
|
||||
- `#1f7a6b` evidence green.
|
||||
- `#b5472f` rejection red.
|
||||
- `#c58a19` warning amber.
|
||||
- `#5b6472` operational gray.
|
||||
|
||||
Type:
|
||||
|
||||
- UI/body: `Inter, ui-sans-serif, system-ui`.
|
||||
- Data and labels: `ui-monospace, SFMono-Regular, Consolas`.
|
||||
|
||||
Layout:
|
||||
|
||||
```text
|
||||
+--------------------------------------------------------------------+
|
||||
| Session id input | Load | status / verdict / counts strip |
|
||||
+------------------+----------------------+--------------------------+
|
||||
| Agent trace rail | Evidence tool ledger | Inspector |
|
||||
| ordered steps | filters + calls | verifier + RAG + skill |
|
||||
+------------------+----------------------+--------------------------+
|
||||
```
|
||||
|
||||
Signature element: a trace rail that treats each agent step as a ledger entry
|
||||
with ordered markers, duration, token count, and expandable raw model excerpts.
|
||||
|
||||
## Data mapping
|
||||
|
||||
- `data.session`: summary strip, final answer, self-evaluation source.
|
||||
- `data.steps`: ordered trace rail.
|
||||
- `data.toolInvocations`: evidence ledger and selected tool inspector.
|
||||
- `data.session.selfEvaluation.verifier_evaluation`: verdict, groundedness,
|
||||
facts table, and verifier `tool_trace_summary`.
|
||||
- `data.toolInvocations[*].retrievalDetails`: RAG query transform, retrieval
|
||||
trace, context pack, rerank trace, and evidence blocks.
|
||||
- `steps[*].modelOutput`: best-effort extraction of `selected_skill`.
|
||||
- `steps[*].modelInput/modelOutput` and tool names: best-effort skill boundary
|
||||
checks for `read_skill`.
|
||||
|
||||
## Accessibility and states
|
||||
|
||||
- Keyboard-focusable controls.
|
||||
- Loading, missing session id, API error, empty steps, empty tools, and missing
|
||||
verifier states.
|
||||
- Responsive three-pane desktop layout that stacks on narrow screens.
|
||||
- Respects `prefers-reduced-motion`.
|
||||
@@ -0,0 +1,37 @@
|
||||
# Trace UI workbench
|
||||
|
||||
## Why
|
||||
|
||||
The MVP already persists diagnosis trace data and exposes it through
|
||||
`GET /api/diagnosis/{sessionId}/trace`, but reviewers still need to inspect raw
|
||||
JSON to answer basic audit questions:
|
||||
|
||||
- Which agents ran, in what order, and with what persisted counts?
|
||||
- Which evidence tools executed, how many times, and did they succeed?
|
||||
- Which facts did the Verifier check, and which evidence refs support them?
|
||||
- Did Planner only select skill metadata while Executor handled skill loading?
|
||||
- What RAG retrieval details were used by `lookup_knowledge`?
|
||||
|
||||
This slows down demo review and makes trace quality issues harder to spot.
|
||||
|
||||
## What changes
|
||||
|
||||
- Add a static Trace workbench page served by Spring Boot static resources.
|
||||
- Load a diagnosis trace by session id through the existing read-only Trace API.
|
||||
- Render session summary, agent timeline, evidence tool ledger, verifier facts,
|
||||
skill boundary checks, and RAG retrieval details.
|
||||
- Add a navigation entry from the existing chat page to the Trace workbench.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No new backend endpoint.
|
||||
- No mutation of diagnosis sessions, agent steps, tool invocations, or feedback.
|
||||
- No new frontend framework or build pipeline.
|
||||
- No change to the persisted trace schema.
|
||||
|
||||
## Impact
|
||||
|
||||
- Frontend-only runtime surface under `src/main/resources/static`.
|
||||
- Uses the existing `Result<T>` API response contract.
|
||||
- Works with existing trace records, including sessions that lack verifier or RAG
|
||||
detail fields.
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: MVP demo SHALL provide a browser trace workbench
|
||||
The MVP demo SHALL provide a browser-accessible static page for inspecting one
|
||||
diagnosis trace by session id using the existing read-only Trace API.
|
||||
|
||||
#### Scenario: Existing trace renders in the workbench
|
||||
- **WHEN** a reviewer opens the Trace workbench with a session id that exists
|
||||
- **THEN** the page SHALL request `GET /api/diagnosis/{sessionId}/trace`
|
||||
- **AND** it SHALL render session summary, ordered agent steps, ordered tool
|
||||
invocations, verifier evaluation, and final answer when present
|
||||
|
||||
#### Scenario: Trace workbench handles missing or failed traces
|
||||
- **WHEN** the Trace API returns an error or the session id is empty
|
||||
- **THEN** the page SHALL show a clear error or empty state without mutating any
|
||||
diagnosis data
|
||||
|
||||
### Requirement: Trace workbench SHALL expose evidence and skill boundaries
|
||||
The Trace workbench SHALL make tool evidence and skill-loading boundaries visible
|
||||
without requiring raw JSON inspection first.
|
||||
|
||||
#### Scenario: Tool and verifier evidence are inspectable
|
||||
- **WHEN** a trace includes tool invocations and verifier facts
|
||||
- **THEN** the page SHALL show tool invocation counts, success status, tool
|
||||
filters, verifier facts, and evidence references
|
||||
|
||||
#### Scenario: RAG details are inspectable for lookup knowledge calls
|
||||
- **WHEN** a `lookup_knowledge` invocation includes retrieval details
|
||||
- **THEN** the page SHALL show query transform, retrieval trace, context pack,
|
||||
rerank trace, and evidence block data where available
|
||||
|
||||
#### Scenario: Skill boundary checks are visible
|
||||
- **WHEN** a trace includes planner, executor, or verifier steps
|
||||
- **THEN** the page SHALL show best-effort indicators for selected skill,
|
||||
planner `read_skill` text mentions, executor `read_skill` text mentions, and
|
||||
verifier `read_skill` text mentions
|
||||
@@ -0,0 +1,9 @@
|
||||
# Tasks
|
||||
|
||||
- [x] 1. Add OpenSpec delta for the Trace UI workbench.
|
||||
- [x] 2. Add static Trace workbench HTML/CSS/JS.
|
||||
- [x] 3. Add a chat-page navigation entry to the Trace workbench.
|
||||
- [x] 4. Verify Java compilation and static page syntax.
|
||||
- [x] 5. Validate the page against a real trace endpoint when a local service is available.
|
||||
- [x] 6. Archive the OpenSpec change after validation.
|
||||
- [x] 7. Commit the completed change.
|
||||
@@ -1,9 +1,7 @@
|
||||
## Purpose
|
||||
|
||||
Provide a repeatable MVP demo flow that can run a chat diagnosis, expose its persisted execution trace, and submit feedback for the same session id.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement: Diagnosis trace can be queried by session id
|
||||
The system SHALL expose a read-only HTTP endpoint `GET /api/diagnosis/{sessionId}/trace` that returns the persisted diagnosis trace for the requested session id.
|
||||
|
||||
@@ -64,3 +62,39 @@ The MVP demo SHALL document which trace fields to inspect for evidence, verifier
|
||||
#### Scenario: Checklist maps fields to interview claims
|
||||
- **WHEN** a developer reviews a trace response
|
||||
- **THEN** the checklist SHALL map concrete JSON paths to the claims made in the interview walkthrough
|
||||
|
||||
### Requirement: MVP demo SHALL provide a browser trace workbench
|
||||
The MVP demo SHALL provide a browser-accessible static page for inspecting one
|
||||
diagnosis trace by session id using the existing read-only Trace API.
|
||||
|
||||
#### Scenario: Existing trace renders in the workbench
|
||||
- **WHEN** a reviewer opens the Trace workbench with a session id that exists
|
||||
- **THEN** the page SHALL request `GET /api/diagnosis/{sessionId}/trace`
|
||||
- **AND** it SHALL render session summary, ordered agent steps, ordered tool
|
||||
invocations, verifier evaluation, and final answer when present
|
||||
|
||||
#### Scenario: Trace workbench handles missing or failed traces
|
||||
- **WHEN** the Trace API returns an error or the session id is empty
|
||||
- **THEN** the page SHALL show a clear error or empty state without mutating any
|
||||
diagnosis data
|
||||
|
||||
### Requirement: Trace workbench SHALL expose evidence and skill boundaries
|
||||
The Trace workbench SHALL make tool evidence and skill-loading boundaries visible
|
||||
without requiring raw JSON inspection first.
|
||||
|
||||
#### Scenario: Tool and verifier evidence are inspectable
|
||||
- **WHEN** a trace includes tool invocations and verifier facts
|
||||
- **THEN** the page SHALL show tool invocation counts, success status, tool
|
||||
filters, verifier facts, and evidence references
|
||||
|
||||
#### Scenario: RAG details are inspectable for lookup knowledge calls
|
||||
- **WHEN** a `lookup_knowledge` invocation includes retrieval details
|
||||
- **THEN** the page SHALL show query transform, retrieval trace, context pack,
|
||||
rerank trace, and evidence block data where available
|
||||
|
||||
#### Scenario: Skill boundary checks are visible
|
||||
- **WHEN** a trace includes planner, executor, or verifier steps
|
||||
- **THEN** the page SHALL show best-effort indicators for selected skill,
|
||||
planner `read_skill` text mentions, executor `read_skill` text mentions, and
|
||||
verifier `read_skill` text mentions
|
||||
|
||||
|
||||
@@ -34,6 +34,14 @@
|
||||
<span style="font-size: 14px; font-weight: 500;">文档管理</span>
|
||||
</a>
|
||||
|
||||
<a href="trace.html" class="sidebar-btn" style="display: flex; align-items: center; gap: 12px; padding: 12px; border-radius: 12px; color: #202124; text-decoration: none; transition: background 0.3s ease; margin-top: 8px;">
|
||||
<svg viewBox="0 0 24 24" fill="none" xmlns="http://www.w3.org/2000/svg" style="width: 20px; height: 20px; flex-shrink: 0;">
|
||||
<path d="M4 19V5M4 5H20M4 12H16M4 19H20" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<path d="M8 5V19M16 5V19" stroke="currentColor" stroke-width="2" stroke-linecap="round"/>
|
||||
</svg>
|
||||
<span style="font-size: 14px; font-weight: 500;">Trace Workbench</span>
|
||||
</a>
|
||||
|
||||
<div class="chat-history-section">
|
||||
<div class="history-header">
|
||||
<span>近期对话</span>
|
||||
|
||||
@@ -0,0 +1,606 @@
|
||||
:root {
|
||||
--ink: #111827;
|
||||
--ink-soft: #263241;
|
||||
--paper: #f7f3ea;
|
||||
--line: #d6cbbb;
|
||||
--evidence: #1f7a6b;
|
||||
--evidence-soft: #dfeee9;
|
||||
--reject: #b5472f;
|
||||
--reject-soft: #f4dfd8;
|
||||
--warn: #c58a19;
|
||||
--warn-soft: #f5ead1;
|
||||
--steel: #5b6472;
|
||||
--white: #fffdf8;
|
||||
--shadow: 0 18px 48px rgba(17, 24, 39, 0.16);
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
html,
|
||||
body {
|
||||
min-height: 100%;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
background: var(--paper);
|
||||
color: var(--ink);
|
||||
font-family: Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
button,
|
||||
input {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
button {
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
button:focus-visible,
|
||||
input:focus-visible,
|
||||
a:focus-visible,
|
||||
.tool-row:focus-visible {
|
||||
outline: 3px solid rgba(31, 122, 107, 0.34);
|
||||
outline-offset: 2px;
|
||||
}
|
||||
|
||||
svg {
|
||||
width: 18px;
|
||||
height: 18px;
|
||||
fill: none;
|
||||
stroke: currentColor;
|
||||
stroke-width: 2;
|
||||
stroke-linecap: round;
|
||||
stroke-linejoin: round;
|
||||
}
|
||||
|
||||
.trace-app {
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
padding: 22px;
|
||||
gap: 14px;
|
||||
}
|
||||
|
||||
.trace-header {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(260px, 1fr) minmax(340px, 560px);
|
||||
gap: 18px;
|
||||
align-items: end;
|
||||
padding: 18px 20px;
|
||||
background: var(--ink);
|
||||
color: var(--white);
|
||||
border-radius: 8px;
|
||||
box-shadow: var(--shadow);
|
||||
}
|
||||
|
||||
.trace-back-link {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
color: #cbd5e1;
|
||||
text-decoration: none;
|
||||
font-size: 13px;
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
}
|
||||
|
||||
.trace-header h1 {
|
||||
margin: 8px 0 0;
|
||||
font-size: clamp(28px, 4vw, 52px);
|
||||
line-height: 0.95;
|
||||
font-weight: 760;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
.session-form {
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.session-form label,
|
||||
.summary-cell span,
|
||||
.pane-heading p,
|
||||
.tool-meta,
|
||||
.state-line,
|
||||
.field-label {
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 12px;
|
||||
text-transform: uppercase;
|
||||
color: var(--steel);
|
||||
}
|
||||
|
||||
.session-form label {
|
||||
color: #cbd5e1;
|
||||
}
|
||||
|
||||
.session-control {
|
||||
display: grid;
|
||||
grid-template-columns: minmax(0, 1fr) auto;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.session-control input {
|
||||
width: 100%;
|
||||
min-width: 0;
|
||||
height: 42px;
|
||||
padding: 0 12px;
|
||||
border: 1px solid #4b5563;
|
||||
border-radius: 6px;
|
||||
background: #0b1220;
|
||||
color: var(--white);
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
}
|
||||
|
||||
.session-control button {
|
||||
height: 42px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
gap: 8px;
|
||||
padding: 0 14px;
|
||||
border: 0;
|
||||
border-radius: 6px;
|
||||
color: var(--white);
|
||||
background: var(--evidence);
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.session-control button:disabled {
|
||||
opacity: 0.62;
|
||||
cursor: wait;
|
||||
}
|
||||
|
||||
.state-line {
|
||||
min-height: 34px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
padding: 8px 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: var(--white);
|
||||
}
|
||||
|
||||
.state-line.error {
|
||||
color: var(--reject);
|
||||
border-color: #dfb2a5;
|
||||
background: var(--reject-soft);
|
||||
}
|
||||
|
||||
.state-line.loading {
|
||||
color: var(--evidence);
|
||||
border-color: #a9d2c8;
|
||||
background: var(--evidence-soft);
|
||||
}
|
||||
|
||||
.summary-strip {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(7, minmax(120px, 1fr));
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
background: var(--white);
|
||||
}
|
||||
|
||||
.summary-cell {
|
||||
min-width: 0;
|
||||
padding: 13px 14px;
|
||||
border-right: 1px solid var(--line);
|
||||
}
|
||||
|
||||
.summary-cell:last-child {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.summary-cell strong {
|
||||
display: block;
|
||||
margin-top: 5px;
|
||||
font-size: 21px;
|
||||
line-height: 1.1;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.trace-grid {
|
||||
min-height: 0;
|
||||
flex: 1;
|
||||
display: grid;
|
||||
grid-template-columns: minmax(300px, 1.05fr) minmax(300px, 0.95fr) minmax(340px, 1.1fr);
|
||||
gap: 14px;
|
||||
}
|
||||
|
||||
.trace-pane {
|
||||
min-height: 620px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
background: var(--white);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.pane-heading {
|
||||
min-height: 74px;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
gap: 14px;
|
||||
padding: 14px 16px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
background: #fbf7ef;
|
||||
}
|
||||
|
||||
.pane-heading p,
|
||||
.pane-heading h2 {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
.pane-heading h2 {
|
||||
margin-top: 3px;
|
||||
font-size: 18px;
|
||||
}
|
||||
|
||||
.pane-count {
|
||||
min-width: 38px;
|
||||
min-height: 30px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
padding: 5px 9px;
|
||||
border-radius: 999px;
|
||||
background: var(--ink);
|
||||
color: var(--white);
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.trace-rail,
|
||||
.tool-ledger,
|
||||
.inspector-stack {
|
||||
overflow: auto;
|
||||
}
|
||||
|
||||
.trace-rail {
|
||||
position: relative;
|
||||
padding: 18px 16px 24px 34px;
|
||||
}
|
||||
|
||||
.trace-rail::before {
|
||||
content: "";
|
||||
position: absolute;
|
||||
left: 23px;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 3px;
|
||||
background: var(--ink);
|
||||
}
|
||||
|
||||
.step-entry {
|
||||
position: relative;
|
||||
margin-bottom: 14px;
|
||||
padding: 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: #fffaf1;
|
||||
}
|
||||
|
||||
.step-entry::before {
|
||||
content: attr(data-index);
|
||||
position: absolute;
|
||||
left: -30px;
|
||||
top: 12px;
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
border: 2px solid var(--ink);
|
||||
border-radius: 50%;
|
||||
background: var(--paper);
|
||||
color: var(--ink);
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 11px;
|
||||
font-weight: 800;
|
||||
}
|
||||
|
||||
.step-entry.planner {
|
||||
border-left: 5px solid var(--warn);
|
||||
}
|
||||
|
||||
.step-entry.executor {
|
||||
border-left: 5px solid var(--evidence);
|
||||
}
|
||||
|
||||
.step-entry.verifier {
|
||||
border-left: 5px solid var(--reject);
|
||||
}
|
||||
|
||||
.step-title,
|
||||
.tool-title {
|
||||
display: flex;
|
||||
align-items: flex-start;
|
||||
justify-content: space-between;
|
||||
gap: 10px;
|
||||
}
|
||||
|
||||
.step-title h3,
|
||||
.tool-title h3 {
|
||||
margin: 0;
|
||||
font-size: 16px;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.badge {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
min-height: 22px;
|
||||
padding: 3px 8px;
|
||||
border-radius: 999px;
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 11px;
|
||||
font-weight: 700;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.badge.ok {
|
||||
color: #0b4f44;
|
||||
background: var(--evidence-soft);
|
||||
}
|
||||
|
||||
.badge.warn {
|
||||
color: #754d00;
|
||||
background: var(--warn-soft);
|
||||
}
|
||||
|
||||
.badge.bad {
|
||||
color: #7c2414;
|
||||
background: var(--reject-soft);
|
||||
}
|
||||
|
||||
.meta-line {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 8px;
|
||||
margin-top: 9px;
|
||||
}
|
||||
|
||||
.meta-pill {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 5px;
|
||||
padding: 4px 7px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 999px;
|
||||
background: var(--white);
|
||||
color: var(--steel);
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 11px;
|
||||
}
|
||||
|
||||
details {
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
summary {
|
||||
color: var(--ink-soft);
|
||||
font-weight: 700;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
pre,
|
||||
.answer-box {
|
||||
margin: 8px 0 0;
|
||||
padding: 10px;
|
||||
max-height: 280px;
|
||||
overflow: auto;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: #fdf8ee;
|
||||
color: #172033;
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 12px;
|
||||
line-height: 1.55;
|
||||
white-space: pre-wrap;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.tool-filters {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 8px;
|
||||
padding: 12px 14px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
background: #fffaf1;
|
||||
}
|
||||
|
||||
.tool-filter {
|
||||
min-height: 30px;
|
||||
padding: 5px 10px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 999px;
|
||||
background: var(--white);
|
||||
color: var(--ink);
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 12px;
|
||||
}
|
||||
|
||||
.tool-filter.active {
|
||||
border-color: var(--ink);
|
||||
background: var(--ink);
|
||||
color: var(--white);
|
||||
}
|
||||
|
||||
.tool-ledger {
|
||||
padding: 12px 14px 18px;
|
||||
}
|
||||
|
||||
.tool-row {
|
||||
width: 100%;
|
||||
display: block;
|
||||
margin-bottom: 10px;
|
||||
padding: 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: var(--white);
|
||||
color: inherit;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.tool-row:hover,
|
||||
.tool-row.selected {
|
||||
border-color: var(--evidence);
|
||||
background: #f5fbf8;
|
||||
}
|
||||
|
||||
.tool-row.failed {
|
||||
border-color: #e1b4aa;
|
||||
}
|
||||
|
||||
.tool-meta {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 7px;
|
||||
margin-top: 8px;
|
||||
color: var(--steel);
|
||||
text-transform: none;
|
||||
}
|
||||
|
||||
.inspector-stack {
|
||||
padding: 14px;
|
||||
}
|
||||
|
||||
.inspector-section {
|
||||
padding: 0 0 16px;
|
||||
margin-bottom: 16px;
|
||||
border-bottom: 1px solid var(--line);
|
||||
}
|
||||
|
||||
.inspector-section:last-child {
|
||||
margin-bottom: 0;
|
||||
border-bottom: 0;
|
||||
}
|
||||
|
||||
.inspector-section h3 {
|
||||
margin: 0 0 10px;
|
||||
font-size: 15px;
|
||||
}
|
||||
|
||||
.boundary-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.boundary-item,
|
||||
.fact-row,
|
||||
.field-row {
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: #fffaf1;
|
||||
}
|
||||
|
||||
.boundary-item {
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.boundary-item strong {
|
||||
display: block;
|
||||
margin-top: 4px;
|
||||
font-family: ui-monospace, SFMono-Regular, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 14px;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.fact-row {
|
||||
padding: 10px;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.fact-row p {
|
||||
margin: 0 0 8px;
|
||||
line-height: 1.45;
|
||||
}
|
||||
|
||||
.field-grid {
|
||||
display: grid;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.field-row {
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
.field-label {
|
||||
display: block;
|
||||
margin-bottom: 5px;
|
||||
color: var(--steel);
|
||||
}
|
||||
|
||||
.empty-panel {
|
||||
padding: 14px;
|
||||
border: 1px dashed var(--line);
|
||||
border-radius: 6px;
|
||||
color: var(--steel);
|
||||
background: #fffaf1;
|
||||
}
|
||||
|
||||
@media (max-width: 1180px) {
|
||||
.summary-strip {
|
||||
grid-template-columns: repeat(4, minmax(120px, 1fr));
|
||||
}
|
||||
|
||||
.summary-cell:nth-child(4) {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.trace-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.trace-pane {
|
||||
min-height: 420px;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 760px) {
|
||||
.trace-app {
|
||||
padding: 12px;
|
||||
}
|
||||
|
||||
.trace-header {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.session-control {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
|
||||
.summary-strip {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
}
|
||||
|
||||
.summary-cell,
|
||||
.summary-cell:nth-child(4) {
|
||||
border-right: 1px solid var(--line);
|
||||
}
|
||||
|
||||
.summary-cell:nth-child(2n) {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.boundary-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
*,
|
||||
*::before,
|
||||
*::after {
|
||||
scroll-behavior: auto !important;
|
||||
transition: none !important;
|
||||
animation: none !important;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Trace Workbench - SuperBizAgent</title>
|
||||
<link rel="stylesheet" href="trace.css">
|
||||
</head>
|
||||
<body>
|
||||
<main class="trace-app">
|
||||
<header class="trace-header">
|
||||
<div>
|
||||
<a class="trace-back-link" href="index.html" aria-label="Back to chat">
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true">
|
||||
<path d="M19 12H5M12 19L5 12L12 5" />
|
||||
</svg>
|
||||
<span>Chat</span>
|
||||
</a>
|
||||
<h1>Trace workbench</h1>
|
||||
</div>
|
||||
|
||||
<form class="session-form" id="sessionForm">
|
||||
<label for="sessionIdInput">Session id</label>
|
||||
<div class="session-control">
|
||||
<input id="sessionIdInput" name="sessionId" autocomplete="off" spellcheck="false" placeholder="e2e-mysql-skill-rag-20260706-2355">
|
||||
<button type="submit" id="loadButton">
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true">
|
||||
<path d="M5 12H19M13 6L19 12L13 18" />
|
||||
</svg>
|
||||
<span>Load</span>
|
||||
</button>
|
||||
</div>
|
||||
</form>
|
||||
</header>
|
||||
|
||||
<section class="state-line" id="stateLine" role="status">Enter a session id to load a persisted diagnosis trace.</section>
|
||||
|
||||
<section class="summary-strip" id="summaryStrip" aria-label="Trace summary">
|
||||
<div class="summary-cell">
|
||||
<span>Status</span>
|
||||
<strong id="summaryStatus">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Verdict</span>
|
||||
<strong id="summaryVerdict">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Groundedness</span>
|
||||
<strong id="summaryScore">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Steps</span>
|
||||
<strong id="summarySteps">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Tools</span>
|
||||
<strong id="summaryTools">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Duration</span>
|
||||
<strong id="summaryDuration">-</strong>
|
||||
</div>
|
||||
<div class="summary-cell">
|
||||
<span>Tokens</span>
|
||||
<strong id="summaryTokens">-</strong>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section class="trace-grid">
|
||||
<section class="trace-pane agent-pane" aria-labelledby="agentPaneTitle">
|
||||
<div class="pane-heading">
|
||||
<div>
|
||||
<p>Agent rail</p>
|
||||
<h2 id="agentPaneTitle">Persisted steps</h2>
|
||||
</div>
|
||||
<span class="pane-count" id="stepCount">0</span>
|
||||
</div>
|
||||
<div class="trace-rail" id="stepsRail"></div>
|
||||
</section>
|
||||
|
||||
<section class="trace-pane evidence-pane" aria-labelledby="evidencePaneTitle">
|
||||
<div class="pane-heading">
|
||||
<div>
|
||||
<p>Evidence ledger</p>
|
||||
<h2 id="evidencePaneTitle">Tool invocations</h2>
|
||||
</div>
|
||||
<span class="pane-count" id="toolCount">0</span>
|
||||
</div>
|
||||
<div class="tool-filters" id="toolFilters"></div>
|
||||
<div class="tool-ledger" id="toolLedger"></div>
|
||||
</section>
|
||||
|
||||
<section class="trace-pane inspector-pane" aria-labelledby="inspectorPaneTitle">
|
||||
<div class="pane-heading">
|
||||
<div>
|
||||
<p>Inspector</p>
|
||||
<h2 id="inspectorPaneTitle">Verifier and RAG</h2>
|
||||
</div>
|
||||
<span class="pane-count" id="inspectorMode">idle</span>
|
||||
</div>
|
||||
|
||||
<div class="inspector-stack">
|
||||
<section class="inspector-section" aria-labelledby="skillBoundaryTitle">
|
||||
<h3 id="skillBoundaryTitle">Skill boundary</h3>
|
||||
<div class="boundary-grid" id="skillBoundary"></div>
|
||||
</section>
|
||||
|
||||
<section class="inspector-section" aria-labelledby="verifierTitle">
|
||||
<h3 id="verifierTitle">Verifier facts</h3>
|
||||
<div id="verifierFacts"></div>
|
||||
</section>
|
||||
|
||||
<section class="inspector-section" aria-labelledby="toolInspectorTitle">
|
||||
<h3 id="toolInspectorTitle">Selected tool</h3>
|
||||
<div id="toolInspector"></div>
|
||||
</section>
|
||||
|
||||
<section class="inspector-section" aria-labelledby="answerTitle">
|
||||
<h3 id="answerTitle">Final answer</h3>
|
||||
<div class="answer-box" id="finalAnswer">-</div>
|
||||
</section>
|
||||
</div>
|
||||
</section>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script src="trace.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,574 @@
|
||||
class TraceWorkbench {
|
||||
constructor() {
|
||||
this.trace = null;
|
||||
this.selectedToolId = null;
|
||||
this.activeFilter = 'all';
|
||||
|
||||
this.sessionForm = document.getElementById('sessionForm');
|
||||
this.sessionIdInput = document.getElementById('sessionIdInput');
|
||||
this.loadButton = document.getElementById('loadButton');
|
||||
this.stateLine = document.getElementById('stateLine');
|
||||
this.stepsRail = document.getElementById('stepsRail');
|
||||
this.toolLedger = document.getElementById('toolLedger');
|
||||
this.toolFilters = document.getElementById('toolFilters');
|
||||
this.skillBoundary = document.getElementById('skillBoundary');
|
||||
this.verifierFacts = document.getElementById('verifierFacts');
|
||||
this.toolInspector = document.getElementById('toolInspector');
|
||||
this.finalAnswer = document.getElementById('finalAnswer');
|
||||
|
||||
this.bindEvents();
|
||||
this.bootstrapFromUrl();
|
||||
}
|
||||
|
||||
bindEvents() {
|
||||
this.sessionForm.addEventListener('submit', (event) => {
|
||||
event.preventDefault();
|
||||
this.loadTrace(this.sessionIdInput.value.trim());
|
||||
});
|
||||
}
|
||||
|
||||
bootstrapFromUrl() {
|
||||
const params = new URLSearchParams(window.location.search);
|
||||
const sessionId = params.get('sessionId');
|
||||
if (sessionId) {
|
||||
this.sessionIdInput.value = sessionId;
|
||||
this.loadTrace(sessionId);
|
||||
return;
|
||||
}
|
||||
this.renderEmpty();
|
||||
}
|
||||
|
||||
async loadTrace(sessionId) {
|
||||
if (!sessionId) {
|
||||
this.setState('Enter a session id before loading.', 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
this.setState(`Loading ${sessionId}...`, 'loading');
|
||||
this.loadButton.disabled = true;
|
||||
|
||||
try {
|
||||
const response = await fetch(`/api/diagnosis/${encodeURIComponent(sessionId)}/trace`);
|
||||
const data = await this.handleResponse(response);
|
||||
this.trace = data;
|
||||
this.selectedToolId = data.toolInvocations && data.toolInvocations.length
|
||||
? data.toolInvocations[0].id
|
||||
: null;
|
||||
this.activeFilter = 'all';
|
||||
this.updateUrl(sessionId);
|
||||
this.renderTrace();
|
||||
this.setState(`Loaded ${sessionId}.`, '');
|
||||
} catch (error) {
|
||||
this.trace = null;
|
||||
this.renderEmpty();
|
||||
this.setState(error.message, 'error');
|
||||
} finally {
|
||||
this.loadButton.disabled = false;
|
||||
}
|
||||
}
|
||||
|
||||
async handleResponse(response) {
|
||||
let payload = null;
|
||||
try {
|
||||
payload = await response.json();
|
||||
} catch (error) {
|
||||
throw new Error(`HTTP ${response.status}: response is not JSON`);
|
||||
}
|
||||
|
||||
if (!response.ok || payload.code !== 200) {
|
||||
throw new Error(payload.message || `HTTP ${response.status}: trace request failed`);
|
||||
}
|
||||
|
||||
return payload.data;
|
||||
}
|
||||
|
||||
updateUrl(sessionId) {
|
||||
const url = new URL(window.location.href);
|
||||
url.searchParams.set('sessionId', sessionId);
|
||||
window.history.replaceState({}, '', url.toString());
|
||||
}
|
||||
|
||||
renderTrace() {
|
||||
const session = this.trace.session || {};
|
||||
const verifier = this.getVerifier();
|
||||
const summary = this.trace.summary || {};
|
||||
const steps = this.trace.steps || [];
|
||||
const tools = this.trace.toolInvocations || [];
|
||||
|
||||
this.text('summaryStatus', session.status || '-');
|
||||
this.text('summaryVerdict', verifier.verdict || '-');
|
||||
this.text('summaryScore', this.formatScore(verifier.groundedness_score));
|
||||
this.text('summarySteps', `${summary.returnedStepCount ?? steps.length}/${summary.persistedStepCount ?? session.stepCount ?? '-'}`);
|
||||
this.text('summaryTools', `${summary.returnedToolCallCount ?? tools.length}/${summary.persistedToolCallCount ?? session.toolCallCount ?? '-'}`);
|
||||
this.text('summaryDuration', this.formatDuration(session.totalDurationMs));
|
||||
this.text('summaryTokens', this.formatNumber(session.totalTokenCount));
|
||||
this.text('stepCount', steps.length);
|
||||
this.text('toolCount', tools.length);
|
||||
this.text('inspectorMode', tools.length ? 'tool' : 'trace');
|
||||
|
||||
this.renderSteps(steps);
|
||||
this.renderToolFilters(tools);
|
||||
this.renderTools(tools);
|
||||
this.renderSkillBoundary();
|
||||
this.renderVerifier(verifier);
|
||||
this.renderToolInspector();
|
||||
this.finalAnswer.textContent = session.answer || 'No final answer persisted.';
|
||||
}
|
||||
|
||||
renderEmpty() {
|
||||
this.text('summaryStatus', '-');
|
||||
this.text('summaryVerdict', '-');
|
||||
this.text('summaryScore', '-');
|
||||
this.text('summarySteps', '-');
|
||||
this.text('summaryTools', '-');
|
||||
this.text('summaryDuration', '-');
|
||||
this.text('summaryTokens', '-');
|
||||
this.text('stepCount', '0');
|
||||
this.text('toolCount', '0');
|
||||
this.text('inspectorMode', 'idle');
|
||||
|
||||
this.stepsRail.innerHTML = '<div class="empty-panel">No trace loaded.</div>';
|
||||
this.toolFilters.innerHTML = '';
|
||||
this.toolLedger.innerHTML = '<div class="empty-panel">No tool invocations loaded.</div>';
|
||||
this.skillBoundary.innerHTML = this.renderBoundaryItems({
|
||||
selectedSkill: '-',
|
||||
plannerReadSkill: 0,
|
||||
executorReadSkill: 0,
|
||||
verifierReadSkill: 0
|
||||
});
|
||||
this.verifierFacts.innerHTML = '<div class="empty-panel">No verifier evaluation loaded.</div>';
|
||||
this.toolInspector.innerHTML = '<div class="empty-panel">Select a tool invocation after loading a trace.</div>';
|
||||
this.finalAnswer.textContent = '-';
|
||||
}
|
||||
|
||||
renderSteps(steps) {
|
||||
if (!steps.length) {
|
||||
this.stepsRail.innerHTML = '<div class="empty-panel">No persisted agent steps.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
this.stepsRail.innerHTML = steps.map((step, index) => {
|
||||
const agent = (step.agentName || 'agent').toLowerCase();
|
||||
const cls = this.agentClass(agent);
|
||||
return `
|
||||
<article class="step-entry ${cls}" data-index="${this.escape(String(index + 1).padStart(2, '0'))}">
|
||||
<div class="step-title">
|
||||
<h3>${this.escape(step.agentName || 'agent')}</h3>
|
||||
${this.renderBadge(step.hasToolCall ? 'tool' : 'model', step.hasToolCall ? 'warn' : 'ok')}
|
||||
</div>
|
||||
<div class="meta-line">
|
||||
<span class="meta-pill">step ${this.escape(step.stepIndex ?? index)}</span>
|
||||
<span class="meta-pill">${this.escape(this.formatDuration(step.durationMs))}</span>
|
||||
<span class="meta-pill">${this.escape(this.formatNumber(step.tokenCount))} tokens</span>
|
||||
<span class="meta-pill">${this.escape(this.formatDate(step.createdAt))}</span>
|
||||
</div>
|
||||
${this.renderTextDetail('Model input', step.modelInput)}
|
||||
${this.renderTextDetail('Model output', step.modelOutput)}
|
||||
${this.renderTextDetail('Thought', step.thought)}
|
||||
</article>
|
||||
`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
renderToolFilters(tools) {
|
||||
if (!tools.length) {
|
||||
this.toolFilters.innerHTML = '';
|
||||
return;
|
||||
}
|
||||
|
||||
const groups = this.groupByTool(tools);
|
||||
const filters = [
|
||||
{ key: 'all', label: `all ${tools.length}` },
|
||||
{ key: 'success', label: `success ${tools.filter(tool => tool.success !== false).length}` },
|
||||
{ key: 'failed', label: `failed ${tools.filter(tool => tool.success === false).length}` },
|
||||
...Object.entries(groups).map(([name, count]) => ({ key: `tool:${name}`, label: `${name} ${count}` }))
|
||||
];
|
||||
|
||||
this.toolFilters.innerHTML = filters.map(filter => `
|
||||
<button class="tool-filter ${filter.key === this.activeFilter ? 'active' : ''}" data-filter="${this.escape(filter.key)}">
|
||||
${this.escape(filter.label)}
|
||||
</button>
|
||||
`).join('');
|
||||
|
||||
this.toolFilters.querySelectorAll('button').forEach((button) => {
|
||||
button.addEventListener('click', () => {
|
||||
this.activeFilter = button.dataset.filter;
|
||||
this.renderToolFilters(this.trace.toolInvocations || []);
|
||||
this.renderTools(this.trace.toolInvocations || []);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
renderTools(tools) {
|
||||
const filtered = tools.filter(tool => this.matchesFilter(tool));
|
||||
if (!filtered.length) {
|
||||
this.toolLedger.innerHTML = '<div class="empty-panel">No tool invocations match this filter.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
this.toolLedger.innerHTML = filtered.map((tool, index) => {
|
||||
const failed = tool.success === false;
|
||||
const selected = String(tool.id) === String(this.selectedToolId);
|
||||
return `
|
||||
<button class="tool-row ${failed ? 'failed' : ''} ${selected ? 'selected' : ''}" data-tool-id="${this.escape(tool.id)}">
|
||||
<div class="tool-title">
|
||||
<h3>${this.escape(tool.toolName || 'tool')}</h3>
|
||||
${this.renderBadge(failed ? 'failed' : 'ok', failed ? 'bad' : 'ok')}
|
||||
</div>
|
||||
<div class="tool-meta">
|
||||
<span>#${index + 1}</span>
|
||||
<span>id ${this.escape(tool.id ?? '-')}</span>
|
||||
<span>step ${this.escape(tool.stepId ?? '-')}</span>
|
||||
<span>${this.escape(this.formatDuration(tool.durationMs))}</span>
|
||||
${tool.retrievalLayer ? `<span>${this.escape(tool.retrievalLayer)}</span>` : ''}
|
||||
${tool.relevanceLevel ? `<span>${this.escape(tool.relevanceLevel)}</span>` : ''}
|
||||
</div>
|
||||
</button>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
this.toolLedger.querySelectorAll('.tool-row').forEach((row) => {
|
||||
row.addEventListener('click', () => {
|
||||
this.selectedToolId = row.dataset.toolId;
|
||||
this.renderTools(this.trace.toolInvocations || []);
|
||||
this.renderToolInspector();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
renderSkillBoundary() {
|
||||
const steps = this.trace.steps || [];
|
||||
const selectedSkill = this.extractSelectedSkill(steps);
|
||||
const counts = {
|
||||
selectedSkill: selectedSkill || '-',
|
||||
plannerReadSkill: this.countReadSkill(steps, 'planner'),
|
||||
executorReadSkill: this.countReadSkill(steps, 'executor'),
|
||||
verifierReadSkill: this.countReadSkill(steps, 'verifier')
|
||||
};
|
||||
this.skillBoundary.innerHTML = this.renderBoundaryItems(counts);
|
||||
}
|
||||
|
||||
renderBoundaryItems(counts) {
|
||||
return `
|
||||
<div class="boundary-item">
|
||||
<span class="field-label">Selected skill</span>
|
||||
<strong>${this.escape(counts.selectedSkill)}</strong>
|
||||
</div>
|
||||
<div class="boundary-item">
|
||||
<span class="field-label">Planner mentions</span>
|
||||
<strong>${this.escape(counts.plannerReadSkill)}</strong>
|
||||
</div>
|
||||
<div class="boundary-item">
|
||||
<span class="field-label">Executor mentions</span>
|
||||
<strong>${this.escape(counts.executorReadSkill)}</strong>
|
||||
</div>
|
||||
<div class="boundary-item">
|
||||
<span class="field-label">Verifier mentions</span>
|
||||
<strong>${this.escape(counts.verifierReadSkill)}</strong>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
renderVerifier(verifier) {
|
||||
if (!verifier || !Object.keys(verifier).length) {
|
||||
this.verifierFacts.innerHTML = '<div class="empty-panel">No verifier evaluation in selfEvaluation.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
const facts = Array.isArray(verifier.facts_checked) ? verifier.facts_checked : [];
|
||||
const traceSummary = Array.isArray(verifier.tool_trace_summary) ? verifier.tool_trace_summary : [];
|
||||
const header = `
|
||||
<div class="meta-line">
|
||||
${this.renderBadge(verifier.verdict || 'verdict -', this.verdictClass(verifier.verdict))}
|
||||
<span class="meta-pill">score ${this.escape(this.formatScore(verifier.groundedness_score))}</span>
|
||||
<span class="meta-pill">facts ${facts.length}</span>
|
||||
<span class="meta-pill">trace refs ${traceSummary.length}</span>
|
||||
</div>
|
||||
`;
|
||||
|
||||
if (!facts.length) {
|
||||
this.verifierFacts.innerHTML = header + '<div class="empty-panel">Verifier returned no facts_checked rows.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
this.verifierFacts.innerHTML = header + facts.map((fact) => {
|
||||
const refs = Array.isArray(fact.evidence_refs) ? fact.evidence_refs : [];
|
||||
return `
|
||||
<article class="fact-row">
|
||||
<p>${this.escape(fact.fact || '-')}</p>
|
||||
<div class="meta-line">
|
||||
${this.renderBadge(fact.verification || '-', this.factClass(fact.verification))}
|
||||
<span class="meta-pill">${fact.is_critical ? 'critical' : 'supporting'}</span>
|
||||
<span class="meta-pill">refs ${refs.length}</span>
|
||||
</div>
|
||||
${fact.detail ? `<pre>${this.escape(fact.detail)}</pre>` : ''}
|
||||
${refs.length ? `<pre>${this.escape(this.prettyJson(refs))}</pre>` : ''}
|
||||
</article>
|
||||
`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
renderToolInspector() {
|
||||
const tools = this.trace ? (this.trace.toolInvocations || []) : [];
|
||||
const tool = tools.find(item => String(item.id) === String(this.selectedToolId));
|
||||
if (!tool) {
|
||||
this.toolInspector.innerHTML = '<div class="empty-panel">No tool invocation selected.</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
const retrieval = tool.retrievalDetails || {};
|
||||
const ragFields = [
|
||||
['Input params', tool.inputParams || this.parseJson(tool.inputParamsRaw)],
|
||||
['Output preview', tool.outputPreview || '-'],
|
||||
['Retrieval layer', tool.retrievalLayer || '-'],
|
||||
['L0 matches', tool.l0MatchCount ?? '-'],
|
||||
['L1 matches', tool.l1MatchCount ?? '-'],
|
||||
['Relevance', tool.relevanceLevel || '-'],
|
||||
['Dedup reason', tool.dedupReason || '-'],
|
||||
['Query transform', retrieval.query_transform],
|
||||
['Retrieval trace', retrieval.retrieval_trace],
|
||||
['Context pack', retrieval.context_pack_summary],
|
||||
['Rerank trace', retrieval.rerank_trace],
|
||||
['Evidence blocks', retrieval.evidence_blocks]
|
||||
];
|
||||
|
||||
this.toolInspector.innerHTML = `
|
||||
<div class="meta-line">
|
||||
${this.renderBadge(tool.success === false ? 'failed' : 'ok', tool.success === false ? 'bad' : 'ok')}
|
||||
<span class="meta-pill">${this.escape(tool.toolName || 'tool')}</span>
|
||||
<span class="meta-pill">${this.escape(this.formatDuration(tool.durationMs))}</span>
|
||||
<span class="meta-pill">output ${this.escape(this.formatNumber(tool.outputLength))}</span>
|
||||
${tool.truncated ? '<span class="meta-pill">truncated</span>' : ''}
|
||||
</div>
|
||||
${tool.errorMessage ? `<pre>${this.escape(tool.errorMessage)}</pre>` : ''}
|
||||
<div class="field-grid">
|
||||
${ragFields.map(([label, value]) => this.renderField(label, value)).join('')}
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
renderField(label, value) {
|
||||
const empty = value === undefined || value === null || value === '';
|
||||
return `
|
||||
<div class="field-row">
|
||||
<span class="field-label">${this.escape(label)}</span>
|
||||
<pre>${this.escape(empty ? '-' : this.stringifyValue(value))}</pre>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
renderTextDetail(label, text) {
|
||||
if (!text) {
|
||||
return '';
|
||||
}
|
||||
return `
|
||||
<details>
|
||||
<summary>${this.escape(label)}</summary>
|
||||
<pre>${this.escape(text)}</pre>
|
||||
</details>
|
||||
`;
|
||||
}
|
||||
|
||||
renderBadge(text, cls) {
|
||||
return `<span class="badge ${this.escape(cls || 'ok')}">${this.escape(text)}</span>`;
|
||||
}
|
||||
|
||||
getVerifier() {
|
||||
const evaluation = (this.trace && this.trace.session && this.trace.session.selfEvaluation) || {};
|
||||
if (evaluation.verifier_evaluation) {
|
||||
return evaluation.verifier_evaluation;
|
||||
}
|
||||
if (evaluation.verdict || evaluation.groundedness_score || evaluation.facts_checked) {
|
||||
return evaluation;
|
||||
}
|
||||
return {};
|
||||
}
|
||||
|
||||
extractSelectedSkill(steps) {
|
||||
const plannerText = steps
|
||||
.filter(step => (step.agentName || '').toLowerCase().includes('planner'))
|
||||
.map(step => `${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||
.join('\n');
|
||||
const allText = steps
|
||||
.map(step => `${step.modelInput || ''}\n${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||
.join('\n');
|
||||
const sourceText = `${plannerText}\n${allText}`;
|
||||
const direct = sourceText.match(/"selected_skill"\s*:\s*"([^"]+)"/);
|
||||
if (direct) {
|
||||
return direct[1];
|
||||
}
|
||||
const skillName = sourceText.match(/"skill_name"\s*:\s*"([^"]+)"/);
|
||||
if (skillName) {
|
||||
return skillName[1];
|
||||
}
|
||||
const loose = sourceText.match(/selected_skill\s*[:=]\s*([A-Za-z0-9_.-]+)/);
|
||||
return loose ? loose[1] : '';
|
||||
}
|
||||
|
||||
countReadSkill(steps, agentName) {
|
||||
const text = steps
|
||||
.filter(step => (step.agentName || '').toLowerCase().includes(agentName))
|
||||
.map(step => `${step.modelInput || ''}\n${step.modelOutput || ''}\n${step.thought || ''}`)
|
||||
.join('\n');
|
||||
const matches = text.match(/\bread_skill\b/g);
|
||||
return matches ? matches.length : 0;
|
||||
}
|
||||
|
||||
matchesFilter(tool) {
|
||||
if (this.activeFilter === 'all') {
|
||||
return true;
|
||||
}
|
||||
if (this.activeFilter === 'success') {
|
||||
return tool.success !== false;
|
||||
}
|
||||
if (this.activeFilter === 'failed') {
|
||||
return tool.success === false;
|
||||
}
|
||||
if (this.activeFilter.startsWith('tool:')) {
|
||||
return tool.toolName === this.activeFilter.slice(5);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
groupByTool(tools) {
|
||||
return tools.reduce((groups, tool) => {
|
||||
const name = tool.toolName || 'tool';
|
||||
groups[name] = (groups[name] || 0) + 1;
|
||||
return groups;
|
||||
}, {});
|
||||
}
|
||||
|
||||
agentClass(agent) {
|
||||
if (agent.includes('planner')) {
|
||||
return 'planner';
|
||||
}
|
||||
if (agent.includes('executor')) {
|
||||
return 'executor';
|
||||
}
|
||||
if (agent.includes('verifier')) {
|
||||
return 'verifier';
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
verdictClass(verdict) {
|
||||
if (!verdict) {
|
||||
return 'warn';
|
||||
}
|
||||
const value = String(verdict).toUpperCase();
|
||||
if (value.includes('PASS')) {
|
||||
return 'ok';
|
||||
}
|
||||
if (value.includes('REJECT')) {
|
||||
return 'bad';
|
||||
}
|
||||
return 'warn';
|
||||
}
|
||||
|
||||
factClass(verification) {
|
||||
if (!verification) {
|
||||
return 'warn';
|
||||
}
|
||||
const value = String(verification).toUpperCase();
|
||||
if (value.includes('SUPPORTED') || value.includes('PASS')) {
|
||||
return 'ok';
|
||||
}
|
||||
if (value.includes('UNSUPPORTED') || value.includes('CONTRADICT')) {
|
||||
return 'bad';
|
||||
}
|
||||
return 'warn';
|
||||
}
|
||||
|
||||
parseJson(raw) {
|
||||
if (!raw || typeof raw !== 'string') {
|
||||
return raw;
|
||||
}
|
||||
try {
|
||||
return JSON.parse(raw);
|
||||
} catch (error) {
|
||||
return raw;
|
||||
}
|
||||
}
|
||||
|
||||
stringifyValue(value) {
|
||||
if (typeof value === 'string') {
|
||||
return value;
|
||||
}
|
||||
return this.prettyJson(value);
|
||||
}
|
||||
|
||||
prettyJson(value) {
|
||||
try {
|
||||
return JSON.stringify(value, null, 2);
|
||||
} catch (error) {
|
||||
return String(value);
|
||||
}
|
||||
}
|
||||
|
||||
formatDuration(ms) {
|
||||
if (ms === undefined || ms === null || ms === '') {
|
||||
return '-';
|
||||
}
|
||||
if (ms < 1000) {
|
||||
return `${ms}ms`;
|
||||
}
|
||||
const seconds = ms / 1000;
|
||||
if (seconds < 60) {
|
||||
return `${seconds.toFixed(1)}s`;
|
||||
}
|
||||
const minutes = Math.floor(seconds / 60);
|
||||
const remaining = Math.round(seconds % 60);
|
||||
return `${minutes}m ${remaining}s`;
|
||||
}
|
||||
|
||||
formatNumber(value) {
|
||||
if (value === undefined || value === null || value === '') {
|
||||
return '-';
|
||||
}
|
||||
return Number(value).toLocaleString('en-US');
|
||||
}
|
||||
|
||||
formatScore(value) {
|
||||
if (value === undefined || value === null || value === '') {
|
||||
return '-';
|
||||
}
|
||||
const number = Number(value);
|
||||
return Number.isFinite(number) ? number.toFixed(2) : String(value);
|
||||
}
|
||||
|
||||
formatDate(value) {
|
||||
if (!value) {
|
||||
return '-';
|
||||
}
|
||||
const date = new Date(value);
|
||||
if (Number.isNaN(date.getTime())) {
|
||||
return String(value);
|
||||
}
|
||||
return date.toLocaleString('en-US', { hour12: false });
|
||||
}
|
||||
|
||||
text(id, value) {
|
||||
const node = document.getElementById(id);
|
||||
if (node) {
|
||||
node.textContent = value;
|
||||
}
|
||||
}
|
||||
|
||||
setState(message, statusClass) {
|
||||
this.stateLine.textContent = message;
|
||||
this.stateLine.className = `state-line ${statusClass || ''}`.trim();
|
||||
}
|
||||
|
||||
escape(value) {
|
||||
return String(value)
|
||||
.replace(/&/g, '&')
|
||||
.replace(/</g, '<')
|
||||
.replace(/>/g, '>')
|
||||
.replace(/"/g, '"')
|
||||
.replace(/'/g, ''');
|
||||
}
|
||||
}
|
||||
|
||||
document.addEventListener('DOMContentLoaded', () => {
|
||||
new TraceWorkbench();
|
||||
});
|
||||
Reference in New Issue
Block a user