2.1 KiB
2.1 KiB
1. Prompt Contract
- 1.1 Update
chat-verifier-prompt.mdsoexecutor_structured_output.claimsis the primary verification target. - 1.2 Remove the valid-structured-output path that scans
executor_final_answerfor extra confirmed facts. - 1.3 Add
claim_checksand optionalhypothesis_checksto the output contract. - 1.4 Keep
facts_checkedas compatibility output. - 1.5 State that missing/malformed structured output cannot produce PASS through natural-language fallback.
2. Parser And Compatibility Mapping
- 2.1 Extend
VerifierDecisionto storeclaim_checks. - 2.2 Parse
claim_checksfrom verifier output. - 2.3 Derive compatibility
facts_checkedfromclaim_checkswhen present. - 2.4 Preserve old
facts_checkedparsing whenclaim_checksis absent. - 2.5 Persist
claim_checksin verifier evaluation.
3. Effective Verdict Guardrails
- 3.1 Add code-side guard so
gatekeeper_result.status=failcannot result in effective PASS. - 3.2 Downgrade
evidence.invocation_reffailures to REJECT. - 3.3 Downgrade other Gatekeeper failures to at least LOW_CONFID.
- 3.4 Ensure missing/malformed Executor structured output cannot produce effective PASS.
4. Compatibility Consumers
- 4.1 Ensure low-confidence rendering still uses compatibility
facts_checked. - 4.2 Ensure
buildRetryContext(...)still receives evidence gaps from compatibilityfacts_checked. - 4.3 Update verifier logging summary to account for
claim_checks. - 4.4 Preserve existing traceability and Gatekeeper audit fields.
5. Tests And Verification
- 5.1 Add or update tests for parsing
claim_checks. - 5.2 Add tests for
claim_checkstofacts_checkedcompatibility mapping. - 5.3 Add tests for each verification mapping class:
direct_observation,reasonable_inference,overstated,unsupported,external_unknown,contradicted. - 5.4 Add tests proving Gatekeeper fail cannot remain PASS.
- 5.5 Add tests proving malformed/missing structured output cannot remain PASS.
- 5.6 Run targeted tests.
- 5.7 Validate this OpenSpec change.