Files
SuperBizAgent-java/openspec/changes/archive/2026-07-07-executor-verifier-claim-checks/tasks.md
T

2.1 KiB

1. Prompt Contract

  • 1.1 Update chat-verifier-prompt.md so executor_structured_output.claims is the primary verification target.
  • 1.2 Remove the valid-structured-output path that scans executor_final_answer for extra confirmed facts.
  • 1.3 Add claim_checks and optional hypothesis_checks to the output contract.
  • 1.4 Keep facts_checked as compatibility output.
  • 1.5 State that missing/malformed structured output cannot produce PASS through natural-language fallback.

2. Parser And Compatibility Mapping

  • 2.1 Extend VerifierDecision to store claim_checks.
  • 2.2 Parse claim_checks from verifier output.
  • 2.3 Derive compatibility facts_checked from claim_checks when present.
  • 2.4 Preserve old facts_checked parsing when claim_checks is absent.
  • 2.5 Persist claim_checks in verifier evaluation.

3. Effective Verdict Guardrails

  • 3.1 Add code-side guard so gatekeeper_result.status=fail cannot result in effective PASS.
  • 3.2 Downgrade evidence.invocation_ref failures to REJECT.
  • 3.3 Downgrade other Gatekeeper failures to at least LOW_CONFID.
  • 3.4 Ensure missing/malformed Executor structured output cannot produce effective PASS.

4. Compatibility Consumers

  • 4.1 Ensure low-confidence rendering still uses compatibility facts_checked.
  • 4.2 Ensure buildRetryContext(...) still receives evidence gaps from compatibility facts_checked.
  • 4.3 Update verifier logging summary to account for claim_checks.
  • 4.4 Preserve existing traceability and Gatekeeper audit fields.

5. Tests And Verification

  • 5.1 Add or update tests for parsing claim_checks.
  • 5.2 Add tests for claim_checks to facts_checked compatibility mapping.
  • 5.3 Add tests for each verification mapping class: direct_observation, reasonable_inference, overstated, unsupported, external_unknown, contradicted.
  • 5.4 Add tests proving Gatekeeper fail cannot remain PASS.
  • 5.5 Add tests proving malformed/missing structured output cannot remain PASS.
  • 5.6 Run targeted tests.
  • 5.7 Validate this OpenSpec change.