40 lines
1.4 KiB
Markdown
40 lines
1.4 KiB
Markdown
# Payment Timeout Acceptance Case
|
|
|
|
## Goal
|
|
|
|
Validate that the MVP can diagnose a payment timeout incident and expose the complete trace for replay.
|
|
|
|
## Input
|
|
|
|
- Session id: `mvp-demo-payment-timeout-001`
|
|
- Question: `支付接口最近出现超时，请结合知识库、日志和指标判断可能原因，并给出修复建议。`
|
|
- Profile: `mvp-demo`
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. Chat returns a successful answer with the same session id.
|
|
2. Trace API returns session metadata, final answer, ordered agent steps, and ordered tool invocations.
|
|
3. Trace contains enough evidence to explain which tools were used and whether verifier/self-evaluation was persisted.
|
|
4. Feedback can be submitted for the same session id.
|
|
5. A follow-up trace query shows the persisted feedback value.
|
|
|
|
## Trace Fields To Inspect
|
|
|
|
- `data.session.query`
|
|
- `data.session.answer`
|
|
- `data.session.selfEvaluation`
|
|
- `data.session.feedback`
|
|
- `data.steps[*].agentName`
|
|
- `data.steps[*].thought`
|
|
- `data.toolInvocations[*].toolName`
|
|
- `data.toolInvocations[*].inputParams`
|
|
- `data.toolInvocations[*].outputPreview`
|
|
- `data.toolInvocations[*].retrievalDetails`
|
|
- `data.summary`
|
|
|
|
## Known Limits
|
|
|
|
- This case is not a full offline test. It still requires valid infrastructure for chat, persistence, vector search, and model calls.
|
|
- Mock logs and metrics are enabled by the `mvp-demo` profile to make those evidence tools repeatable.
|
|
- Sensitive configuration cleanup is deferred by current MVP priority.
|