48 lines
2.2 KiB
Markdown
48 lines
2.2 KiB
Markdown
## Why
|
|
|
|
Stage one moved Executor to `executor_evidence_v2`, and stage two added deterministic Gatekeeper checks before Verifier. The Verifier still mainly operates through the legacy `facts_checked` contract and the prompt still allows fallback extraction from `executor_final_answer`.
|
|
|
|
That keeps two problems alive:
|
|
|
|
- Verifier can still treat natural-language Executor output as a fact source.
|
|
- Downstream code cannot distinguish claim-level verification results from legacy natural-language fact checks.
|
|
|
|
This phase makes Verifier V2 claim-oriented: Verifier evaluates `executor_structured_output.claims` for whether each claim can be reasonably derived from evidence, emits `claim_checks`, and keeps `facts_checked` only as a compatibility projection.
|
|
|
|
## What Changes
|
|
|
|
- Update `chat-verifier-prompt.md` so the primary verification target is `executor_structured_output.claims`.
|
|
- Add Verifier V2 output field `claim_checks`.
|
|
- Preserve compatibility by mapping `claim_checks` into legacy `facts_checked`.
|
|
- Parse and persist `claim_checks` in `ChatService`.
|
|
- Ensure `gatekeeper_result.status=fail` cannot result in an effective `PASS`.
|
|
- Ensure missing or malformed Executor structured output does not fall back to natural-language fact extraction for PASS.
|
|
|
|
## Capabilities
|
|
|
|
### New Capabilities
|
|
|
|
None.
|
|
|
|
### Modified Capabilities
|
|
|
|
- `chat-verifier-agent`: Verifier output now includes claim-level checks and uses claim derivability as the primary groundedness contract.
|
|
|
|
## Impact
|
|
|
|
- Affected prompt: `src/main/resources/prompts/chat-verifier-prompt.md`.
|
|
- Affected service: `ChatService.parseVerifierDecision(...)`, retry context generation, verifier persistence.
|
|
- Affected audit: `diagnosis_session.self_evaluation.verifier_evaluation` gains `claim_checks` and keeps `facts_checked`.
|
|
- Affected logging: verifier thought summaries may count `claim_checks`.
|
|
- Affected tests: `ChatServiceSequentialAgentTest` and focused verifier parsing tests.
|
|
- Database schema: no table or column change.
|
|
|
|
## Non-Goals
|
|
|
|
- No Composer in this phase.
|
|
- No final-answer material filtering in this phase beyond existing LOW_CONFID/REJECT templates and temporary V2 renderer.
|
|
- No Executor retry behavior change.
|
|
- No Gatekeeper rule expansion.
|
|
- No database schema migration.
|
|
|