47 lines
2.2 KiB
Markdown
47 lines
2.2 KiB
Markdown
# Acceptance
|
|
|
|
## Static Verification
|
|
|
|
- `openspec validate interview-demo-quality-audit --strict`
|
|
- Result: passed.
|
|
- Coverage: OpenSpec proposal/design/spec/tasks consistency.
|
|
- PowerShell parser/runtime readiness check:
|
|
- Command: `powershell -NoProfile -ExecutionPolicy Bypass -File mvp/demo/scripts/run-interview-demo-check.ps1 -BaseUrl http://127.0.0.1:1 -OutputDir target/demo-check-syntax`
|
|
- Result: expected failure with actionable readiness message.
|
|
- Coverage: script parses under Windows PowerShell and fails before issuing diagnosis requests when service is unreachable.
|
|
|
|
## Script Verification
|
|
|
|
- `mvn -q "-Dtest=DiagnosisTraceEvaluatorTest" test`
|
|
- Result: passed.
|
|
- Coverage: 12/12 fixed eval fixtures, Prompt audit evaluator checks, Gatekeeper rule metadata checks, regenerated baseline reports.
|
|
- `mvn -q "-Dtest=ChatServiceSequentialAgentTest" test`
|
|
- Result: passed.
|
|
- Coverage: Chat verifier evaluation persists `prompt_audit`.
|
|
- `mvn -q "-Dtest=DiagnosisTraceEvaluatorTest,DiagnosisEvalBaselineDiffTest,ChatServiceSequentialAgentTest,ExecutorGatekeeperServiceTest,VerifierInputHookTest" test`
|
|
- Result: passed.
|
|
- Coverage: broader eval, baseline diff, Chat sequential flow, Gatekeeper, and Verifier input hook regression set.
|
|
- `mvn -q -DskipTests compile`
|
|
- Result: passed.
|
|
- Coverage: main source compilation.
|
|
|
|
## E2E Verification
|
|
|
|
- Started service with:
|
|
- `mvn spring-boot:run -Dspring-boot.run.profiles=mvp-demo`
|
|
- Ran:
|
|
- `powershell -NoProfile -ExecutionPolicy Bypass -File mvp/demo/scripts/run-interview-demo-check.ps1 -BaseUrl http://localhost:9900 -SessionId mvp-demo-interview-quality-audit-001`
|
|
- Result: passed.
|
|
- Summary:
|
|
- `chatSuccess=true`
|
|
- `verdict=LOW_CONFID`
|
|
- `gatekeeperStatus=fail`
|
|
- `gatekeeperRuleSetVersion=gatekeeper-rules-v1`
|
|
- `promptAuditVersion=chat-prompts-v1`
|
|
- tools included `lookup_knowledge`, `query_logs`, `query_metrics`, and `get_available_log_topics`
|
|
- Note: live E2E remains a compatibility check, not the deterministic PASS oracle. The fixed fixture baseline is the regression source of truth.
|
|
|
|
## Not Verified
|
|
|
|
- Browser UI inspection was not required for this change because the scope is backend trace/eval/demo script documentation, not frontend behavior.
|