feat(agent): harden verifier evidence references

2026-07-08 16:12:56 +08:00
parent a08672b31e
commit 7b8c75e571
29 changed files with 3426 additions and 94 deletions
@@ -0,0 +1,52 @@
+# Acceptance
+
+## Static Verification
+
+- `openspec validate verifier-evidence-reference-fidelity --strict`: passed.
+- `openspec validate --specs`: passed.
+
+## Script Verification
+
+- `mvn "-Dtest=ToolInvocationRecorderTest,ExecutorGatekeeperServiceTest,VerifierInputHookTest,QueryLogsToolsTest,ChatServiceSequentialAgentTest" test`
+  - Passed: 38 tests.
+- `mvn "-Dtest=ExecutorGatekeeperServiceTest" test`
+  - Passed: 9 tests.
+- `mvn test`
+  - Failed on unrelated environment-gated `MilvusConnectionTest.connect`: `MILVUS_TOKEN` environment variable was not set.
+  - Other executed tests in the run progressed until that single failure; focused tests for this change passed.
+
+## End-to-End Verification
+
+The Java service was restarted with `mvn spring-boot:run`; logs were written under `logs/`.
+
+| Case | Session | Result | Gatekeeper Audit |
+|---|---|---|---|
+| HikariCP positive | `iss007-hikari-positive-20260708-1553` | PASS; confirmed order-service HikariCP timeout and pool saturation logs | `pass / none`, checked_bindings=2 |
+| HikariCP negative | `iss007-hikari-negative-20260708-1555` | LOW_CONFID; no `generic-service`; no false positive for inventory-service | `fail / low_confid` |
+| HighMemoryUsage positive | `iss007-memory-positive-20260708-1558` | PASS; confirmed HighMemoryUsage 91%, did not confirm memory leak | `pass / none`, checked_bindings=1 |
+| SlowResponse positive | `iss007-slow-positive-20260708-1600` | PASS; confirmed SlowResponse and slow request logs, no DB pool root cause | `pass / none`, checked_bindings=7 |
+| Narrow HighCPUUsage | `iss007-narrow-highcpu-20260708-1602` | PASS; only covered payment-service HighCPUUsage | `pass / none`, checked_bindings=1 |
+
+## Database Audit
+
+`scripts/query_mysql.py` was used to verify:
+
+- `diagnosis_session.self_evaluation.verifier_evaluation.verdict`
+- `gatekeeper_result.status`
+- `gatekeeper_result.severity`
+- `gatekeeper_result.checked_bindings`
+- no-hit HikariCP query rows persist `evidence_status=no_evidence`
+
+## Remaining Risk
+
+- Negative no-hit claims still have incomplete precise references when Executor uses `$.logs` for empty arrays. Gatekeeper correctly downgrades to `LOW_CONFID`.
+- Prompt-only scope control is improved but not a hard contract. A future `scope_contract` may still be needed.
+- Full test suite requires `MILVUS_TOKEN` to pass `MilvusConnectionTest`.
+
+## OpenSpec Archive
+
+- `openspec archive verifier-evidence-reference-fidelity --yes`: succeeded.
+- Main specs updated:
+  - `openspec/specs/chat-verifier-agent/spec.md`
+  - `openspec/specs/evidence-trace-hardening/spec.md`
+- Non-blocking warning: proposal did not use OpenSpec's preferred `## Why` / `## What Changes` headers, but archive completed.