feat: archive mvp demo trace acceptance
This commit is contained in:
@@ -0,0 +1,39 @@
|
||||
# Payment Timeout Acceptance Case
|
||||
|
||||
## Goal
|
||||
|
||||
Validate that the MVP can diagnose a payment timeout incident and expose the complete trace for replay.
|
||||
|
||||
## Input
|
||||
|
||||
- Session id: `mvp-demo-payment-timeout-001`
|
||||
- Question: `支付接口最近出现超时,请结合知识库、日志和指标判断可能原因,并给出修复建议。`
|
||||
- Profile: `mvp-demo`
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. Chat returns a successful answer with the same session id.
|
||||
2. Trace API returns session metadata, final answer, ordered agent steps, and ordered tool invocations.
|
||||
3. Trace contains enough evidence to explain which tools were used and whether verifier/self-evaluation was persisted.
|
||||
4. Feedback can be submitted for the same session id.
|
||||
5. A follow-up trace query shows the persisted feedback value.
|
||||
|
||||
## Trace Fields To Inspect
|
||||
|
||||
- `data.session.query`
|
||||
- `data.session.answer`
|
||||
- `data.session.selfEvaluation`
|
||||
- `data.session.feedback`
|
||||
- `data.steps[*].agentName`
|
||||
- `data.steps[*].thought`
|
||||
- `data.toolInvocations[*].toolName`
|
||||
- `data.toolInvocations[*].inputParams`
|
||||
- `data.toolInvocations[*].outputPreview`
|
||||
- `data.toolInvocations[*].retrievalDetails`
|
||||
- `data.summary`
|
||||
|
||||
## Known Limits
|
||||
|
||||
- This case is not a full offline test. It still requires valid infrastructure for chat, persistence, vector search, and model calls.
|
||||
- Mock logs and metrics are enabled by the `mvp-demo` profile to make those evidence tools repeatable.
|
||||
- Sensitive configuration cleanup is deferred by current MVP priority.
|
||||
Reference in New Issue
Block a user