feat: archive mvp demo trace acceptance

This commit is contained in:
zhuyongxin
2026-07-03 16:25:00 +08:00
parent 5b827fe90e
commit 6919092b83
24 changed files with 1343 additions and 2 deletions
+94
View File
@@ -0,0 +1,94 @@
# MVP Demo Runbook
This demo proves the MVP flow from user question to persisted diagnosis trace.
## Prerequisites
- MySQL, Redis, Milvus/Zilliz, and LLM/embedding configuration are available through the current project configuration.
- Security and secret cleanup are intentionally out of scope for this MVP slice.
- The `mvp-demo` profile enables mock Prometheus and CLS providers so log and metric tools can return repeatable evidence.
## Start
```powershell
mvn spring-boot:run "-Dspring-boot.run.profiles=mvp-demo"
```
The service listens on:
```text
http://localhost:9900
```
## 1. Run Chat Diagnosis
```powershell
$sessionId = "mvp-demo-payment-timeout-001"
$body = @{
Id = $sessionId
Question = "支付接口最近出现超时,请结合知识库、日志和指标判断可能原因,并给出修复建议。"
} | ConvertTo-Json
Invoke-RestMethod `
-Method Post `
-Uri "http://localhost:9900/api/chat" `
-ContentType "application/json" `
-Body $body
```
Expected result:
- `data.success` is `true`.
- `data.sessionId` equals `mvp-demo-payment-timeout-001`.
- `data.answer` contains a diagnosis answer.
## 2. Query Trace
```powershell
Invoke-RestMethod `
-Method Get `
-Uri "http://localhost:9900/api/diagnosis/$sessionId/trace"
```
Expected result:
- `code` is `200`.
- `data.session.sessionId` equals the chat session id.
- `data.steps` contains planner/executor/verifier records for complex questions.
- `data.toolInvocations` contains evidence tool calls such as `lookup_knowledge`, `query_logs`, or `query_metrics`.
- `data.session.selfEvaluation` contains verifier or rule evaluation when available.
## 3. Submit Feedback
```powershell
$feedback = @{
sessionId = $sessionId
feedback = "useful"
} | ConvertTo-Json
Invoke-RestMethod `
-Method Post `
-Uri "http://localhost:9900/api/feedback" `
-ContentType "application/json" `
-Body $feedback
```
Expected result:
- `success` is `true`.
- A later trace query shows `data.session.feedback` as `useful`.
## Demo Story
The important interview story is:
```text
one session id
-> user question
-> multi-agent execution
-> evidence tools
-> verifier/self-evaluation
-> final answer
-> feedback
-> trace API for replay and audit
```
+39
View File
@@ -0,0 +1,39 @@
# Payment Timeout Acceptance Case
## Goal
Validate that the MVP can diagnose a payment timeout incident and expose the complete trace for replay.
## Input
- Session id: `mvp-demo-payment-timeout-001`
- Question: `支付接口最近出现超时,请结合知识库、日志和指标判断可能原因,并给出修复建议。`
- Profile: `mvp-demo`
## Acceptance Criteria
1. Chat returns a successful answer with the same session id.
2. Trace API returns session metadata, final answer, ordered agent steps, and ordered tool invocations.
3. Trace contains enough evidence to explain which tools were used and whether verifier/self-evaluation was persisted.
4. Feedback can be submitted for the same session id.
5. A follow-up trace query shows the persisted feedback value.
## Trace Fields To Inspect
- `data.session.query`
- `data.session.answer`
- `data.session.selfEvaluation`
- `data.session.feedback`
- `data.steps[*].agentName`
- `data.steps[*].thought`
- `data.toolInvocations[*].toolName`
- `data.toolInvocations[*].inputParams`
- `data.toolInvocations[*].outputPreview`
- `data.toolInvocations[*].retrievalDetails`
- `data.summary`
## Known Limits
- This case is not a full offline test. It still requires valid infrastructure for chat, persistence, vector search, and model calls.
- Mock logs and metrics are enabled by the `mvp-demo` profile to make those evidence tools repeatable.
- Sensitive configuration cleanup is deferred by current MVP priority.