feat(demo): add interview quality audit

This commit is contained in:
zhuyongxin
2026-07-09 11:18:49 +08:00
parent a6c2d4459c
commit 9c9a0024d4
37 changed files with 1162 additions and 86 deletions
@@ -0,0 +1,32 @@
## 1. OpenSpec And devflow
- [x] 1.1 Create OpenSpec proposal/design/spec/tasks for `interview-demo-quality-audit`.
- [x] 1.2 Record context, question pool, interface impact, audit, and verification plan in devflow decisions.
- [x] 1.3 Pass OpenSpec validation and create `.committed`.
## 2. Prompt/Gatekeeper Version Audit
- [x] 2.1 Add compact Chat prompt audit metadata for planner, executor, verifier, and composer prompts.
- [x] 2.2 Persist `prompt_audit` under `verifier_evaluation` for Chat verifier/composer outcomes.
- [x] 2.3 Add focused tests proving prompt audit appears in persisted verifier evaluation.
- [x] 2.4 Extend eval checks for prompt audit and Gatekeeper rule metadata.
## 3. Eval Expansion
- [x] 3.1 Extend diagnosis eval case/result schema for prompt audit fields.
- [x] 3.2 Add fixture-backed cases for audit closure coverage.
- [x] 3.3 Regenerate baseline JSON and Markdown reports.
- [x] 3.4 Update eval docs/schema.
## 4. Interview Demo Stabilization
- [x] 4.1 Add `run-interview-demo-check.ps1` with service preflight, chat, trace, feedback, and summary output.
- [x] 4.2 Update demo README and 10-minute script to use the preflight path.
- [x] 4.3 Add interview Q&A documentation focused on Agent engineering tradeoffs.
## 5. Verification And Archive
- [x] 5.1 Run targeted tests for ChatService/prompt audit and diagnosis eval.
- [x] 5.2 Run relevant broader regression tests.
- [x] 5.3 Run E2E demo check with `mvp-demo` profile if dependencies are available; otherwise record the blocker.
- [x] 5.4 Archive the OpenSpec change, update devflow artifacts, and commit implementation + archive.
@@ -1,32 +0,0 @@
## 1. OpenSpec And devflow
- [x] 1.1 Create OpenSpec proposal/design/spec/tasks for `interview-demo-quality-audit`.
- [x] 1.2 Record context, question pool, interface impact, audit, and verification plan in devflow decisions.
- [x] 1.3 Pass OpenSpec validation and create `.committed`.
## 2. Prompt/Gatekeeper Version Audit
- [ ] 2.1 Add compact Chat prompt audit metadata for planner, executor, verifier, and composer prompts.
- [ ] 2.2 Persist `prompt_audit` under `verifier_evaluation` for Chat verifier/composer outcomes.
- [ ] 2.3 Add focused tests proving prompt audit appears in persisted verifier evaluation.
- [ ] 2.4 Extend eval checks for prompt audit and Gatekeeper rule metadata.
## 3. Eval Expansion
- [ ] 3.1 Extend diagnosis eval case/result schema for prompt audit fields.
- [ ] 3.2 Add fixture-backed cases for audit closure coverage.
- [ ] 3.3 Regenerate baseline JSON and Markdown reports.
- [ ] 3.4 Update eval docs/schema.
## 4. Interview Demo Stabilization
- [ ] 4.1 Add `run-interview-demo-check.ps1` with service preflight, chat, trace, feedback, and summary output.
- [ ] 4.2 Update demo README and 10-minute script to use the preflight path.
- [ ] 4.3 Add interview Q&A documentation focused on Agent engineering tradeoffs.
## 5. Verification And Archive
- [ ] 5.1 Run targeted tests for ChatService/prompt audit and diagnosis eval.
- [ ] 5.2 Run relevant broader regression tests.
- [ ] 5.3 Run E2E demo check with `mvp-demo` profile if dependencies are available; otherwise record the blocker.
- [ ] 5.4 Archive the OpenSpec change, update devflow artifacts, and commit implementation + archive.