feat(agent): add verifier claim checks

2026-07-08 02:33:02 +08:00
parent c5e496e715
commit 1b31e78be5
19 changed files with 1247 additions and 115 deletions
@@ -0,0 +1,47 @@
+## Why
+
+Stage one moved Executor to `executor_evidence_v2`, and stage two added deterministic Gatekeeper checks before Verifier. The Verifier still mainly operates through the legacy `facts_checked` contract and the prompt still allows fallback extraction from `executor_final_answer`.
+
+That keeps two problems alive:
+
+- Verifier can still treat natural-language Executor output as a fact source.
+- Downstream code cannot distinguish claim-level verification results from legacy natural-language fact checks.
+
+This phase makes Verifier V2 claim-oriented: Verifier evaluates `executor_structured_output.claims` for whether each claim can be reasonably derived from evidence, emits `claim_checks`, and keeps `facts_checked` only as a compatibility projection.
+
+## What Changes
+
+- Update `chat-verifier-prompt.md` so the primary verification target is `executor_structured_output.claims`.
+- Add Verifier V2 output field `claim_checks`.
+- Preserve compatibility by mapping `claim_checks` into legacy `facts_checked`.
+- Parse and persist `claim_checks` in `ChatService`.
+- Ensure `gatekeeper_result.status=fail` cannot result in an effective `PASS`.
+- Ensure missing or malformed Executor structured output does not fall back to natural-language fact extraction for PASS.
+
+## Capabilities
+
+### New Capabilities
+
+None.
+
+### Modified Capabilities
+
+- `chat-verifier-agent`: Verifier output now includes claim-level checks and uses claim derivability as the primary groundedness contract.
+
+## Impact
+
+- Affected prompt: `src/main/resources/prompts/chat-verifier-prompt.md`.
+- Affected service: `ChatService.parseVerifierDecision(...)`, retry context generation, verifier persistence.
+- Affected audit: `diagnosis_session.self_evaluation.verifier_evaluation` gains `claim_checks` and keeps `facts_checked`.
+- Affected logging: verifier thought summaries may count `claim_checks`.
+- Affected tests: `ChatServiceSequentialAgentTest` and focused verifier parsing tests.
+- Database schema: no table or column change.
+
+## Non-Goals
+
+- No Composer in this phase.
+- No final-answer material filtering in this phase beyond existing LOW_CONFID/REJECT templates and temporary V2 renderer.
+- No Executor retry behavior change.
+- No Gatekeeper rule expansion.
+- No database schema migration.
+