Files
SuperBizAgent-java/openspec/changes/executor-evidence-output-contract/tasks.md
T

1.8 KiB

1. Executor Contract

  • 1.1 Update chat-executor-prompt.md with the evidence-attribution JSON contract.
  • 1.2 Require confirmed claims, hypotheses, recommended actions, missing information, and Chinese user_facing_answer.
  • 1.3 Add prompt-level constraints that runbook/skill/history guidance cannot become current incident facts without current-session evidence.

2. Runtime Parsing And Verifier Input

  • 2.1 Parse Executor final output into executor_structured_output when it is valid JSON.
  • 2.2 Add executor_output_parse_status for valid, missing, and malformed outputs.
  • 2.3 Pass executor_structured_output and parse status into the verifier payload while preserving existing executor_final_answer.
  • 2.4 Persist or expose the structured output through existing trace/evaluation snapshots without a schema change.

3. Verifier Prompt

  • 3.1 Update chat-verifier-prompt.md to verify structured claims before natural-language extraction.
  • 3.2 Add a verifier rule to scan user_facing_answer for extra confirmed-sounding facts not present in structured claims.
  • 3.3 Keep the existing natural-language fallback for malformed or absent structured output.

4. Tests And Evaluation

  • 4.1 Add focused tests for Executor output parsing and parse-status fallback.
  • 4.2 Add focused tests for verifier payload assembly with structured Executor output.
  • 4.3 Add prompt/fixture regression coverage for unsupported claims being placed outside confirmed claims.
  • 4.4 Add or update eval fixture checks that confirmed claims must have evidence bindings.

5. Verification

  • 5.1 Run targeted unit tests for the parsing and verifier-input changes.
  • 5.2 Run relevant Chat verifier/eval regression tests.
  • 5.3 Run OpenSpec validation for executor-evidence-output-contract.