feat(agent): add executor gatekeeper hook

2026-07-08 02:01:49 +08:00
parent 050cbc8fee
commit c5e496e715
23 changed files with 1343 additions and 30 deletions
@@ -0,0 +1,51 @@
+## MODIFIED Requirements
+
+### Requirement: Verifier SHALL consume explicit verification inputs
+The Verifier SHALL receive explicit verification inputs rather than inferring them only from raw conversation history.
+
+#### Scenario: gatekeeper result available to Verifier
+- **WHEN** the system prepares verifier inputs from Executor output
+- **THEN** the payload SHALL include `gatekeeper_result`
+- **AND** `gatekeeper_result.status` SHALL be one of `pass`, `warn`, or `fail`
+- **AND** `gatekeeper_result` SHALL include `failed_rules`, `warnings`, and `errors`
+
+### Requirement: Verifier SHALL be observable
+The Verifier's verdict SHALL be persisted for observability.
+
+#### Scenario: gatekeeper result written to self_evaluation
+- **WHEN** the Verifier evaluation is persisted
+- **THEN** `diagnosis_session.self_evaluation.verifier_evaluation` SHALL include `gatekeeper_result`
+- **AND** existing verifier fields such as `verdict`, `facts_checked`, `executor_output_parse_status`, and `tool_trace_summary` SHALL be preserved
+
+## ADDED Requirements
+
+### Requirement: Executor Gatekeeper SHALL validate deterministic structured-output failures
+The system SHALL run deterministic Gatekeeper checks after Executor output parsing and before Verifier model execution.
+
+#### Scenario: schema rule rejects removed fields
+- **WHEN** Executor structured output contains `diagnosis_summary` or `user_facing_answer`
+- **THEN** `gatekeeper_result.status` SHALL be `fail`
+- **AND** `gatekeeper_result.failed_rules` SHALL contain `schema.executor_v2`
+
+#### Scenario: schema rule rejects missing evidence bindings
+- **WHEN** a confirmed claim has no `evidence_bindings`
+- **THEN** `gatekeeper_result.status` SHALL be `fail`
+- **AND** `gatekeeper_result.failed_rules` SHALL contain `schema.executor_v2`
+
+#### Scenario: invocation rule rejects fabricated invocation ids
+- **WHEN** a claim evidence binding references a `source_invocation_ids` value that is not present in current-session `tool_invocation` rows
+- **THEN** `gatekeeper_result.status` SHALL be `fail`
+- **AND** `gatekeeper_result.failed_rules` SHALL contain `evidence.invocation_ref`
+
+#### Scenario: invocation rule rejects tool name mismatch
+- **WHEN** a claim evidence binding references an existing invocation id
+- **AND** the binding `tool_name` does not match the invocation's persisted `tool_name`
+- **THEN** `gatekeeper_result.status` SHALL be `fail`
+- **AND** `gatekeeper_result.failed_rules` SHALL contain `evidence.invocation_ref`
+
+#### Scenario: valid structured output passes initial gatekeeper rules
+- **WHEN** Executor emits `executor_evidence_v2`
+- **AND** each claim has evidence bindings pointing to current-session invocations with matching tool names
+- **THEN** `gatekeeper_result.status` SHALL be `pass`
+- **AND** `gatekeeper_result.failed_rules` SHALL be empty
+