42 lines
1.7 KiB
Markdown
42 lines
1.7 KiB
Markdown
# Executor Verifier Claim Checks
|
|
|
|
## Background
|
|
|
|
Stage one moved Chat Executor to `executor_evidence_v2`, and stage two added deterministic Gatekeeper checks before Verifier. After those stages, Verifier still primarily used the legacy `facts_checked` contract and could still treat `executor_final_answer` as a fact source.
|
|
|
|
That left two risks:
|
|
|
|
- Verifier could still extract extra confirmed facts from natural-language Executor output.
|
|
- Downstream audit and retry consumers could not distinguish V2 claim-level verification from legacy fact checks.
|
|
|
|
## Goal
|
|
|
|
Make Verifier V2 claim-oriented:
|
|
|
|
- verify `executor_structured_output.claims` as the primary target;
|
|
- emit `claim_checks` as the authoritative V2 result;
|
|
- keep `facts_checked` only as a compatibility projection;
|
|
- enforce code-side guardrails so Gatekeeper failures or malformed structured output cannot remain effective `PASS`.
|
|
|
|
## Scope
|
|
|
|
- Updated `chat-verifier-prompt.md` to frame verification as claim derivability.
|
|
- Extended `ChatService` to parse, normalize, map, and persist `claim_checks`.
|
|
- Added effective verdict guardrails for Gatekeeper failure and malformed/missing Executor structured output.
|
|
- Updated verifier logging summaries to count `claim_checks`.
|
|
- Updated sequential workflow tests to cover claim mapping and downgrade behavior.
|
|
|
|
## Non-Goals
|
|
|
|
- No Composer integration in this phase.
|
|
- No final-answer material filtering beyond existing templates and temporary V2 renderer.
|
|
- No Executor retry behavior change.
|
|
- No Gatekeeper rule expansion.
|
|
- No database schema migration.
|
|
|
|
## OpenSpec
|
|
|
|
- Active change before archive: `openspec/changes/executor-verifier-claim-checks`
|
|
- Capability: `chat-verifier-agent`
|
|
- Scale: standard
|