osiman/SuperBizAgent-java
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(demo): add interview quality audit

Browse Source
This commit is contained in:
zhuyongxin
2026-07-09 11:18:49 +08:00
parent a6c2d4459c
commit 9c9a0024d4
37 changed files with 1162 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
devflow/projects/2026-07-09-interview-demo-quality-audit/evidence.md
+33
View File
@@ -23,3 +23,36 @@
The required `codebase-retrieval` and LSP tools were not exposed in this session. Impact analysis used `rg`, direct file reads, existing OpenSpec/devflow artifacts, and targeted tests instead.
## Implementation Evidence
- `src/main/java/com/superbiz/agent/service/ChatService.java`
- Adds `prompt_audit` under `verifier_evaluation` through the shared `persistVerifierEvaluation(...)` path.
- Uses compact metadata only: audit version, prompt names, prompt versions, and resource paths.
- `src/main/java/com/superbiz/agent/eval/DiagnosisTraceEvaluator.java`
- Adds deterministic checks for `requirePromptAudit`, `expectedPromptAuditVersion`, `expectedPromptVersions`, and `requireGatekeeperRules`.
- `src/main/java/com/superbiz/agent/eval/DiagnosisEvalReportWriter.java`
- Adds Prompt Audit and Gatekeeper rule count columns to Markdown reports.
- `mvp/eval/cases/diagnosis-cases.json`
- Expands fixed baseline to 12 fixture-backed cases.
- `mvp/eval/fixtures/prompt-gatekeeper-audit-closure-pass.json`
- Positive PASS fixture proving Prompt audit and Gatekeeper rule metadata closure.
- `mvp/eval/fixtures/audit-metadata-low-confid.json`
- LOW_CONFID fixture proving safe answer behavior while audit metadata remains present.
- `mvp/demo/scripts/run-interview-demo-check.ps1`
- Adds service readiness, Chat, Trace, feedback, and summary output for interview preflight.
## Verification Evidence
- OpenSpec:
- `openspec validate interview-demo-quality-audit --strict`: passed before archive.
- `openspec validate --specs --strict`: 10 specs passed after merging deltas into main specs.
- Unit/eval:
- `mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test`: passed.
- `mvn -q "-Dtest=ChatServiceSequentialAgentTest" test`: passed.
- `mvn -q "-Dtest=DiagnosisTraceEvaluatorTest,DiagnosisEvalBaselineDiffTest,ChatServiceSequentialAgentTest,ExecutorGatekeeperServiceTest,VerifierInputHookTest" test`: passed.
- Compile:
- `mvn -q -DskipTests compile`: passed.
- E2E:
- Started `mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo`.
- Ran `mvp/demo/scripts/run-interview-demo-check.ps1` against `http://localhost:9900`.
- Summary recorded `chatSuccess=true`, `verdict=LOW_CONFID`, `gatekeeperRuleSetVersion=gatekeeper-rules-v1`, and `promptAuditVersion=chat-prompts-v1`.