feat(demo): add interview quality audit
This commit is contained in:
@@ -37,7 +37,7 @@ http://localhost:9900
|
||||
推荐使用固定脚本:
|
||||
|
||||
```powershell
|
||||
powershell -ExecutionPolicy Bypass -File mvp/demo/scripts/run-payment-timeout-demo.ps1
|
||||
powershell -ExecutionPolicy Bypass -File mvp/demo/scripts/run-interview-demo-check.ps1
|
||||
```
|
||||
|
||||
脚本会写出:
|
||||
@@ -46,6 +46,7 @@ powershell -ExecutionPolicy Bypass -File mvp/demo/scripts/run-payment-timeout-de
|
||||
mvp/demo/output/chat-response.json
|
||||
mvp/demo/output/trace-response.json
|
||||
mvp/demo/output/feedback-response.json
|
||||
mvp/demo/output/interview-demo-summary.json
|
||||
```
|
||||
|
||||
现场话术:
|
||||
@@ -98,6 +99,8 @@ data.toolInvocations[*].outputPreview
|
||||
data.toolInvocations[*].retrievalLayer
|
||||
data.toolInvocations[*].relevanceLevel
|
||||
data.summary.hasVerifierEvaluation
|
||||
data.session.selfEvaluation.verifier_evaluation.prompt_audit.version
|
||||
data.session.selfEvaluation.verifier_evaluation.gatekeeper_result.rule_set_version
|
||||
```
|
||||
|
||||
现场话术:
|
||||
@@ -155,6 +158,10 @@ Verifier 不做新检索,只看工具 trace 汇总。
|
||||
如果 PASS,就输出原答案。
|
||||
如果 LOW_CONFID,可以补证据或加低置信提示。
|
||||
如果 REJECT,就降级输出,只保留已确认信息。
|
||||
|
||||
Prompt 和 Gatekeeper 的版本也会进入 trace。
|
||||
`prompt_audit.version` 用于说明本次 Chat 使用哪套 Prompt 契约,`gatekeeper_result.rule_set_version` 用于说明引用验真的规则版本。
|
||||
固定 fixture baseline 是回归判断来源,live demo 主要证明当前环境链路可跑通。
|
||||
```
|
||||
|
||||
## 7. 展示反馈闭环
|
||||
@@ -226,6 +233,7 @@ AIOps 有两个模式。
|
||||
mvp/demo/output/chat-response.json
|
||||
mvp/demo/output/trace-response.json
|
||||
mvp/demo/output/feedback-response.json
|
||||
mvp/demo/output/interview-demo-summary.json
|
||||
```
|
||||
|
||||
降级话术:
|
||||
|
||||
Reference in New Issue
Block a user