Files
SuperBizAgent-java/openspec/changes/archive/2026-07-07-executor-verifier-claim-checks/proposal.md
T

2.2 KiB

Why

Stage one moved Executor to executor_evidence_v2, and stage two added deterministic Gatekeeper checks before Verifier. The Verifier still mainly operates through the legacy facts_checked contract and the prompt still allows fallback extraction from executor_final_answer.

That keeps two problems alive:

  • Verifier can still treat natural-language Executor output as a fact source.
  • Downstream code cannot distinguish claim-level verification results from legacy natural-language fact checks.

This phase makes Verifier V2 claim-oriented: Verifier evaluates executor_structured_output.claims for whether each claim can be reasonably derived from evidence, emits claim_checks, and keeps facts_checked only as a compatibility projection.

What Changes

  • Update chat-verifier-prompt.md so the primary verification target is executor_structured_output.claims.
  • Add Verifier V2 output field claim_checks.
  • Preserve compatibility by mapping claim_checks into legacy facts_checked.
  • Parse and persist claim_checks in ChatService.
  • Ensure gatekeeper_result.status=fail cannot result in an effective PASS.
  • Ensure missing or malformed Executor structured output does not fall back to natural-language fact extraction for PASS.

Capabilities

New Capabilities

None.

Modified Capabilities

  • chat-verifier-agent: Verifier output now includes claim-level checks and uses claim derivability as the primary groundedness contract.

Impact

  • Affected prompt: src/main/resources/prompts/chat-verifier-prompt.md.
  • Affected service: ChatService.parseVerifierDecision(...), retry context generation, verifier persistence.
  • Affected audit: diagnosis_session.self_evaluation.verifier_evaluation gains claim_checks and keeps facts_checked.
  • Affected logging: verifier thought summaries may count claim_checks.
  • Affected tests: ChatServiceSequentialAgentTest and focused verifier parsing tests.
  • Database schema: no table or column change.

Non-Goals

  • No Composer in this phase.
  • No final-answer material filtering in this phase beyond existing LOW_CONFID/REJECT templates and temporary V2 renderer.
  • No Executor retry behavior change.
  • No Gatekeeper rule expansion.
  • No database schema migration.