# chat-verifier-agent Specification

## Purpose
TBD - created by archiving change chat-verifier-agent. Update Purpose after archive.
## Requirements
### Requirement: Verifier SHALL fact-check Executor answers
The system SHALL have a Verifier Agent that reads only Gatekeeper-projected structured Executor claims and verified claim-local evidence, then produces a structured verdict based on claim derivability.

#### Scenario: PASS verdict when all claims have evidence
- **WHEN** all critical claims in `verified_executor_output.claims` have direct observation or reasonable inference support in `verified_evidence`
- **AND** at least one critical claim has direct observation
- **AND** no critical claim is contradicted, unsupported, external unknown, or overstated
- **AND** verdict ceiling is PASS
- **THEN** the Verifier MAY output model verdict="PASS" with groundedness_score ≥ 0.5

#### Scenario: LOW_CONFID verdict with partial evidence
- **WHEN** no critical claim contradicts verified evidence
- **AND** some critical claims are `unsupported`, `external_unknown`, or `overstated`
- **THEN** the Verifier SHALL output model verdict="LOW_CONFID"

#### Scenario: LOW_CONFID verdict with only inference support
- **WHEN** no critical claim contradicts verified evidence
- **AND** all critical claims are only `reasonable_inference`
- **THEN** the Verifier SHALL output model verdict="LOW_CONFID"

#### Scenario: REJECT verdict when claims contradict evidence
- **WHEN** any critical claim in `verified_executor_output.claims` contradicts verified evidence
- **OR** the claim fabricates a key entity, error code, or conclusion that does not exist in verified evidence
- **THEN** the Verifier SHALL output model verdict="REJECT"

#### Scenario: Verified structured claims are the only verification target
- **WHEN** `verified_executor_output.claims` is present
- **THEN** Verifier SHALL verify each structured claim against matching `verified_evidence` through `claim_checks`
- **AND** each claim's evidence references SHALL match existing claim/invocation/tool/path identifiers when available
- **AND** a claim without matching verified evidence SHALL NOT be classified as `direct_observation`
- **AND** Verifier SHALL NOT receive or add confirmed facts from raw Executor text

#### Scenario: Executor output is invalid
- **WHEN** Executor does not return a legal structured contract
- **THEN** the Graph SHALL route directly to pre-verification Fallback
- **AND** Verifier SHALL NOT execute or fabricate a diagnostic verdict

### Requirement: Verifier SHALL output structured JSON
The Verifier SHALL output a JSON object with verdict, groundedness_score, claim_checks array, compatibility facts_checked array, and rationale.

#### Scenario: claim-level verifier output is accepted
- **WHEN** the Verifier checks Executor V2 structured output
- **THEN** the output SHALL contain `verdict`, `groundedness_score`, `critical_fact_count`, `claim_checks`, `facts_checked`, and `rationale`
- **AND** `claim_checks` SHALL be the primary V2 verification result
- **AND** `facts_checked` SHALL remain available for compatibility

#### Scenario: Output format validation
- **WHEN** the Verifier completes its analysis
- **THEN** the output SHALL contain "verdict", "groundedness_score", "claim_checks", "facts_checked", and "rationale" fields
- **AND** groundedness_score SHALL be a float between 0.0 and 1.0
- **AND** verdict SHALL be one of "PASS", "LOW_CONFID", or "REJECT"

#### Scenario: strict schema output
- **WHEN** the Verifier returns its result
- **THEN** it SHALL output exactly one JSON object
- **AND** it SHALL NOT output Markdown, code fences, or explanatory text outside the JSON object
- **AND** the JSON object SHALL include `critical_fact_count`
- **AND** each `claim_checks` item SHALL include `claim_id`, `verification`, `detail`, and `evidence_refs`
- **AND** each `facts_checked` item SHALL include `fact`, `is_critical`, `verification`, and `detail`

### Requirement: facts_checked SHALL use a fixed classification set
The system SHALL continue to expose compatibility `facts_checked` using its fixed verification classification set.

#### Scenario: claim checks are mapped to legacy facts
- **WHEN** Verifier output contains `claim_checks`
- **THEN** the shared Verifier protocol parser SHALL derive compatibility `facts_checked` when the model did not provide them
- **AND** `direct_observation` SHALL map to `direct_evidence`
- **AND** `reasonable_inference` and `overstated` SHALL map to `indirect_support`
- **AND** `unsupported` and `external_unknown` SHALL map to `no_evidence`
- **AND** `contradicted` SHALL map to `contradicted`

### Requirement: groundedness_score SHALL be derived from fact classifications
The groundedness score SHALL be computed from critical fact classifications instead of being freely chosen by the model.

#### Scenario: contradicted fact forces reject
- **WHEN** any critical fact is labeled `contradicted`
- **THEN** the Verifier SHALL output verdict="REJECT"
- **AND** groundedness_score SHALL be `0.0`

#### Scenario: score derived from supported facts
- **WHEN** no critical fact is contradicted
- **THEN** groundedness_score SHALL be computed from the mapped values of critical facts
- **AND** the implementation SHALL use the fixed mapping `direct_evidence=1.0`, `indirect_support=0.6`, `no_evidence=0.0`
- **AND** the result SHALL be clamped into `[0.0, 1.0]`

### Requirement: ChatService SHALL route based on Verifier verdict
The system SHALL use Diagnosis StateGraph conditional edges, rather than a ChatService outer loop, to route explicit Verifier execution status and effective verdict.

#### Scenario: PASS routes to Composer
- **WHEN** Verifier completes with effective verdict="PASS"
- **THEN** the Graph SHALL invoke Composer with filtered Verifier-allowed material
- **AND** the final user-facing answer SHALL NOT pass through raw Executor output
- **AND** the final user-facing answer SHALL NOT read Executor `user_facing_answer`

#### Scenario: LOW_CONFID does not qualify for evidence retry
- **WHEN** Verifier completes LOW_CONFID but ceiling is LOW_CONFID, no valid critical evidence gap exists, or evidence retry count is already one
- **THEN** the Graph SHALL route to Composer without another Planner cycle
- **AND** the final answer SHALL distinguish confirmed information, possible directions, and evidence gaps

#### Scenario: LOW_CONFID qualifies for evidence retry
- **WHEN** Verifier completes LOW_CONFID with ceiling PASS, at least one critical valid evidence gap, and evidence retry count zero
- **THEN** the Graph SHALL invoke one EVIDENCE_GAP_ONLY Planner cycle
- **AND** it SHALL NOT use groundedness threshold or a ChatService feature flag to decide the retry

#### Scenario: REJECT does not enter retry round
- **WHEN** Verifier completes with effective verdict="REJECT"
- **THEN** the Graph SHALL NOT start an evidence supplementation round
- **AND** it SHALL route to Composer-safe output

#### Scenario: REJECT produces bounded output
- **WHEN** effective verdict is REJECT
- **THEN** the system SHALL output a degraded result indicating current evidence cannot support a reliable conclusion
- **AND** it SHALL NOT pass through raw Executor answer
- **AND** it SHALL NOT include an unsupported root-cause conclusion

#### Scenario: Verifier execution fails
- **WHEN** Verifier exhausts technical retry or returns a non-retryable failure
- **THEN** the Graph SHALL route to pre-verification Fallback
- **AND** no execution status string SHALL be used as model or effective verdict

### Requirement: Verifier SHALL be observable
The Verifier execution, effective verdict, and downstream final-answer composition SHALL be persisted in the current Diagnosis Run self-evaluation container.

#### Scenario: claim checks written to self_evaluation
- **WHEN** a completed Verifier evaluation is persisted
- **THEN** `diagnosis_run.self_evaluation.verifier_evaluation` SHALL include `claim_checks`
- **AND** it SHALL continue to include compatibility `facts_checked`
- **AND** it SHALL include `verifier_status`, `model_verdict`, `effective_verdict`, `verdict`, `groundedness_score`, `rationale`, verified output/evidence, and Gatekeeper audit

#### Scenario: composer output written to self_evaluation
- **WHEN** final answer composition completes
- **THEN** `diagnosis_run.self_evaluation.verifier_evaluation` SHALL include compact `composer_output` when available
- **AND** handled Composer fallback SHALL remain observable through orchestration trace and status/reason fields
- **AND** existing claim/fact and Gatekeeper fields SHALL be preserved

#### Scenario: verdict written to self_evaluation
- **WHEN** Verifier completes
- **THEN** Graph result mapping SHALL write effective verdict under `diagnosis_run.self_evaluation.verifier_evaluation.verdict`
- **AND** existing `rule_evaluation` and `aiops_rule_evaluation` channels SHALL be preserved

#### Scenario: pre-verification fallback is persisted
- **WHEN** Graph reaches Fallback before Verifier completes
- **THEN** verifier evaluation SHALL include available status, Gatekeeper audit, failure reason, and Prompt audit
- **AND** it SHALL NOT fabricate `model_verdict` or `effective_verdict`

#### Scenario: gatekeeper result written to self_evaluation
- **WHEN** Graph result mapping persists available Gatekeeper state
- **THEN** `diagnosis_run.self_evaluation.verifier_evaluation` SHALL include `gatekeeper_result`
- **AND** the result SHALL retain status, severity, checked bindings, rules, failed rules, warnings, and errors when provided by Gatekeeper

#### Scenario: prompt audit written to verifier evaluation
- **WHEN** a complex Chat Graph result is persisted
- **THEN** the system SHALL include a `prompt_audit` object under `diagnosis_run.self_evaluation.verifier_evaluation`
- **AND** `prompt_audit.version` SHALL identify the Chat Prompt audit catalog version
- **AND** `prompt_audit.prompts` SHALL include Planner, Executor, Verifier, and Composer Prompt names and versions
- **AND** full Prompt text SHALL NOT be persisted

#### Scenario: prompt audit available on fallback paths
- **WHEN** Planner, Executor, Gatekeeper, Verifier, or Composer reaches a handled Fallback
- **THEN** the persisted verifier evaluation SHALL still include `prompt_audit`

#### Scenario: evaluation payload is inspected
- **WHEN** Graph verifier evaluation is persisted
- **THEN** it SHALL NOT contain raw Executor text or complete `tool_trace_summary`
- **AND** compatibility `executor_structured_output` SHALL contain at most the verified projection

### Requirement: self_evaluation SHALL be a container object
The `diagnosis_session.self_evaluation` field SHALL store multiple evaluation channels in one JSON object.

#### Scenario: rule evaluation stored separately
- **WHEN** the rule-based evidence scoring completes
- **THEN** the EvaluationService SHALL write the result under `rule_evaluation`
- **AND** existing `verifier_evaluation` data SHALL be preserved

#### Scenario: verifier evaluation stored separately
- **WHEN** the Verifier completes
- **THEN** the ChatService SHALL write the result under `verifier_evaluation`
- **AND** existing `rule_evaluation` data SHALL be preserved

#### Scenario: no whole-object overwrite after initialization
- **WHEN** either evaluation channel updates `self_evaluation`
- **THEN** the implementation SHALL use read-modify-write semantics
- **AND** it SHALL NOT replace the whole JSON object except when initializing from null

### Requirement: Verifier SHALL consume explicit verification inputs
The Verifier SHALL receive a Graph-built verified-only payload rather than inferring business inputs from conversation history, ThreadLocal state, raw Executor text, or complete tool history.

#### Scenario: explicit input blocks available to Verifier
- **WHEN** the Verifier Graph Node starts
- **THEN** the payload SHALL provide `diagnosis_context`, `verified_executor_output`, `verified_evidence`, `gatekeeper_audit`, and `verdict_ceiling`
- **AND** permitted structured `retry_context` SHALL be provided only after evidence retry preparation

#### Scenario: Verifier remains isolated from intermediate and raw material
- **WHEN** the Verifier input is serialized
- **THEN** it SHALL exclude Planner reasoning, Executor intermediate reasoning, raw Executor text, complete tool trace summary, Prompt text, and unrelated parent Graph State

#### Scenario: only passed bindings are available
- **WHEN** Gatekeeper returns mixed passed and failed checked bindings
- **THEN** `verified_executor_output` and `verified_evidence` SHALL contain only claims/material matching passed bindings
- **AND** the Verifier SHALL NOT receive failed or unreferenced tool material

#### Scenario: verified evidence preserves precise references
- **WHEN** the system prepares Verifier input
- **THEN** each verified evidence item SHALL preserve claim id, source invocation id, tool name, raw path, and matched text
- **AND** the item SHALL be traceable to current-run Gatekeeper validation

#### Scenario: gatekeeper audit and ceiling are available
- **WHEN** the system prepares Verifier input
- **THEN** the payload SHALL include raw Gatekeeper audit separately from normalized verdict ceiling
- **AND** a LOW_CONFID ceiling SHALL prevent effective PASS

#### Scenario: technical retry occurs
- **WHEN** the first Verifier attempt returns invalid output or a retryable invocation failure
- **THEN** the second attempt SHALL receive byte-identical serialized input
- **AND** Executor, Gatekeeper, and tools SHALL NOT rerun

### Requirement: Verifier facts SHALL be auditable
Verifier claims and facts SHALL be linkable to the verified binding projection used during verification.

#### Scenario: claim checks contain evidence refs
- **WHEN** the Verifier emits `claim_checks`
- **THEN** each check SHALL include an `evidence_refs` array
- **AND** any non-empty evidence ref SHALL correspond to existing verified evidence by claim id, source invocation id, tool name, or raw path
- **AND** it SHALL NOT reference a failed or unverified binding

#### Scenario: verifier evaluation persists traceability snapshot
- **WHEN** Graph result mapping persists verifier evaluation
- **THEN** it SHALL include `traceability_version`
- **AND** it SHALL include the bounded `verified_evidence` snapshot used by the Verifier
- **AND** it SHALL NOT persist a complete tool trace summary as Verifier input

### Requirement: Verifier inputs SHALL tolerate hardened no-evidence semantics
The verifier integration SHALL continue to work when evidence summaries distinguish failed calls, no-hit calls, and deduped retrievals more explicitly.

#### Scenario: Failed evidence remains a verifier-visible gap
- **WHEN** an evidence-bearing tool invocation fails
- **THEN** the verifier-facing trace summary SHALL preserve that failure as a gap
- **AND** the verifier flow SHALL continue without crashing

#### Scenario: Deduped retrievals do not count as fresh support
- **WHEN** the verifier-facing trace summary contains deduped `lookup_knowledge` entries
- **THEN** those entries SHALL be treated as no-new-evidence
- **AND** they SHALL NOT be interpreted as fresh direct support for the answer

### Requirement: Verifier evidence summaries SHALL preserve concrete supporting facts
The verifier-facing `tool_trace_summary` SHALL preserve compact concrete facts from persisted evidence-tool outputs so direct evidence is not misclassified as missing merely because raw output was truncated.

#### Scenario: Log evidence contains a concrete matching message
- **WHEN** a persisted `query_logs` invocation output contains a concrete log message matching a critical fact
- **THEN** the generated `tool_trace_summary` SHALL include that message or a bounded excerpt of it in `output_summary`
- **AND** Verifier SHALL be able to reference the invocation id as direct evidence

#### Scenario: Metrics evidence contains concrete alert fields
- **WHEN** a persisted `query_metrics` invocation output contains alert names, services, or metric values
- **THEN** the generated `tool_trace_summary` SHALL include the relevant alert names, services, and bounded metric values
- **AND** it SHALL NOT imply unsupported alerts that are absent from the tool output

#### Scenario: Summary remains bounded
- **WHEN** a tool output is large
- **THEN** the generated `tool_trace_summary` SHALL remain bounded
- **AND** it SHALL preserve concrete facts before generic boilerplate or low-value formatting

### Requirement: Verifier low-confidence output SHALL not present unsupported claims as confirmed
When Verifier returns `LOW_CONFID`, user-facing output SHALL clearly separate confirmed facts from evidence gaps and SHALL NOT leave unsupported Executor claims formatted as confirmed findings.

#### Scenario: LOW_CONFID with critical evidence gaps
- **WHEN** Verifier labels critical facts as `no_evidence`
- **THEN** the final user-facing response SHALL identify those gaps from verifier output
- **AND** unsupported raw Executor claims SHALL NOT be presented as confirmed conclusions

### Requirement: Executor SHALL output an evidence-attribution contract
The Chat Executor SHALL produce a machine-checkable final output that separates confirmed claims from hypotheses, recommendations, and missing information.

#### Scenario: Executor V2 final output contains only structured diagnostic fields
- **WHEN** Executor completes a Chat diagnosis step under the V2 contract
- **THEN** its final output SHALL contain `answer_version`, `claims`, `hypotheses`, `recommended_actions`, and `missing_info`
- **AND** `answer_version` SHALL equal `executor_evidence_v2`
- **AND** the output SHOULD be parseable as one JSON object without Markdown fences
- **AND** the output SHALL NOT contain `diagnosis_summary`
- **AND** the output SHALL NOT contain `user_facing_answer`

#### Scenario: Confirmed claims carry evidence bindings
- **WHEN** Executor emits an item under `claims`
- **THEN** the item SHALL include `claim_id`, `claim_type`, `claim_text`, `support_level`, and `evidence_bindings`
- **AND** `support_level` SHALL be one of `direct` or `indirect`
- **AND** `evidence_bindings` SHALL contain at least one evidence binding

#### Scenario: Evidence bindings support multiple tool types
- **WHEN** Executor binds evidence to a claim
- **THEN** each binding SHALL include `source_type`, `tool_name`, `source_invocation_ids`, and `evidence_excerpt`
- **AND** the binding MAY include `source_id`
- **AND** the binding SHALL be able to reference `lookup_knowledge`, `query_logs`, `query_metrics`, or other evidence-bearing tool traces
- **AND** the binding SHALL NOT rely only on a RAG-specific `chunk_id`

#### Scenario: Unsupported conclusions are not confirmed claims
- **WHEN** a possible root cause, detail, or remediation lacks current-session tool evidence
- **THEN** Executor SHALL place it under `hypotheses`, `recommended_actions`, or `missing_info`
- **AND** Executor SHALL NOT present it as a confirmed claim

#### Scenario: Runbook and skill guidance do not become incident facts
- **WHEN** Executor uses runbook, skill, or historical-case guidance
- **THEN** the guidance MAY influence `recommended_actions`
- **AND** the guidance SHALL NOT be emitted as a current incident fact unless current-session tool evidence supports it

### Requirement: User-facing Chat answers SHALL remain readable Chinese
The system SHALL preserve a readable Chinese answer for normal Chat users even when Executor emits a machine-checkable contract.

#### Scenario: V2 machine contract is not exposed as normal user answer
- **WHEN** Executor emits `executor_evidence_v2`
- **AND** Verifier returns `PASS`
- **THEN** normal user output SHALL be rendered as readable Chinese from the structured contract or a safe fallback template
- **AND** normal user output SHALL NOT be the raw Executor JSON object

#### Scenario: Machine contract remains available for trace inspection
- **WHEN** the Chat trace or verifier evaluation is inspected
- **THEN** the structured Executor contract MAY be shown for debugging or audit
- **AND** normal user output SHALL use the existing verifier-routed display path rather than exposing raw JSON by default

### Requirement: Structured Executor output SHALL degrade safely
The StateGraph runtime SHALL tolerate malformed or absent structured Executor output without crashing the Chat flow or invoking Verifier with untrusted material.

#### Scenario: Malformed Executor JSON is classified
- **WHEN** Executor returns malformed JSON or text outside the expected contract
- **THEN** Executor Node SHALL set INVALID_OUTPUT
- **AND** the Graph SHALL route directly to deterministic pre-verification Fallback
- **AND** Gatekeeper, Verifier, and model Composer SHALL NOT execute

#### Scenario: Structured parse failure remains observable
- **WHEN** Executor output parsing fails
- **THEN** orchestration events and verifier evaluation status/failure fields SHALL make the parse failure visible
- **AND** the failure SHALL NOT be treated as a successful evidence-attribution contract or diagnostic verdict

### Requirement: Executor Gatekeeper SHALL validate deterministic structured-output failures
The system SHALL run deterministic Gatekeeper checks as an explicit Graph Node after legal Executor output parsing and before Verifier model execution.

#### Scenario: schema rule rejects removed fields
- **WHEN** Executor structured output contains `diagnosis_summary` or `user_facing_answer`
- **THEN** `gatekeeper_result.status` SHALL be `fail`
- **AND** `gatekeeper_result.failed_rules` SHALL contain `schema.executor_v2`

#### Scenario: schema rule rejects missing evidence bindings
- **WHEN** a confirmed claim has no `evidence_bindings`
- **THEN** `gatekeeper_result.status` SHALL be `fail`
- **AND** `gatekeeper_result.failed_rules` SHALL contain `schema.executor_v2`

#### Scenario: invocation rule rejects fabricated invocation ids
- **WHEN** a claim evidence binding references an invocation id absent from current-run `tool_invocation` rows
- **THEN** `gatekeeper_result.status` SHALL be `fail`
- **AND** `gatekeeper_result.failed_rules` SHALL contain `evidence.invocation_ref`

#### Scenario: invocation rule rejects tool name mismatch
- **WHEN** a claim evidence binding references an existing current-run invocation id
- **AND** binding `tool_name` does not match persisted invocation `tool_name`
- **THEN** `gatekeeper_result.status` SHALL be `fail`
- **AND** `gatekeeper_result.failed_rules` SHALL contain `evidence.invocation_ref`

#### Scenario: valid structured output passes initial gatekeeper rules
- **WHEN** Executor emits legal `executor_evidence_v2`
- **AND** each claim has evidence bindings pointing to current-run invocations with matching tool names and paths
- **THEN** `gatekeeper_result.status` SHALL be `pass`
- **AND** `gatekeeper_result.failed_rules` SHALL be empty

#### Scenario: gatekeeper reject bypasses Verifier
- **WHEN** normalized Gatekeeper status is REJECT
- **THEN** the Graph SHALL route directly to pre-verification Fallback
- **AND** Verifier SHALL NOT execute

#### Scenario: gatekeeper low confidence is bounded
- **WHEN** normalized Gatekeeper status is LOW_CONFID with at least one passed binding
- **THEN** verified input SHALL contain only passed bindings
- **AND** effective verdict SHALL NOT exceed LOW_CONFID

### Requirement: Verifier claim checks SHALL use a fixed derivability classification set
The Verifier SHALL classify each structured claim using a fixed derivability classification set.

#### Scenario: claim check verification values are constrained
- **WHEN** Verifier emits `claim_checks`
- **THEN** each item SHALL use one of `direct_observation`, `reasonable_inference`, `overstated`, `unsupported`, `external_unknown`, or `contradicted`

#### Scenario: claim check evidence references remain auditable
- **WHEN** Verifier emits `claim_checks`
- **THEN** each claim check SHALL include `claim_id`, `verification`, `detail`, and `evidence_refs`
- **AND** every evidence ref SHALL preserve available `trace_ref`, `tool_name`, and `source_invocation_ids`

### Requirement: User-facing verifier outputs SHALL follow Composer-safe protocols
The system SHALL use Composer or fixed safe templates for PASS, LOW_CONFID, and REJECT user-facing responses.

#### Scenario: PASS uses Composer-safe output
- **WHEN** the final verdict is `PASS`
- **THEN** the user-facing response SHALL be generated from Verifier-allowed material through Composer or a safe fixed template
- **AND** it SHALL NOT use Executor `user_facing_answer`
- **AND** it SHALL NOT expose raw Executor JSON

#### Scenario: LOW_CONFID uses Composer-safe uncertainty output
- **WHEN** the final verdict is `LOW_CONFID`
- **THEN** the user-facing response SHALL include only confirmed facts, possible directions, evidence gaps, and next-step suggestions derived from Verifier-allowed material
- **AND** optional evidence gaps, if present, SHALL come only from verifier-identified gaps
- **AND** unsupported raw Executor claims SHALL NOT be presented as confirmed conclusions

#### Scenario: REJECT uses degraded template
- **WHEN** the final verdict is `REJECT`
- **THEN** the user-facing response SHALL use a degraded template or Composer-safe degraded output
- **AND** it SHALL include only confirmed facts, evidence gaps, and next-step suggestions
- **AND** it SHALL NOT include unverified raw answer content
- **AND** it SHALL NOT include a root-cause conclusion

### Requirement: Executor evidence bindings SHALL support precise evidence references
Executor V2 evidence bindings SHALL support precise evidence references that locate evidence inside a persisted tool invocation.

#### Scenario: Precise evidence binding contains invocation path and excerpt
- **WHEN** Executor binds evidence to a claim
- **THEN** the binding SHOULD include singular `source_invocation_id`
- **AND** the binding SHOULD include `raw_path`
- **AND** the binding SHALL include `tool_name` and `evidence_excerpt`
- **AND** the `raw_path` SHALL be interpreted relative to the referenced tool invocation's `retrieval_details.evidence_refs`

#### Scenario: Legacy plural invocation ids remain compatibility only
- **WHEN** Executor emits legacy `source_invocation_ids`
- **THEN** the system MAY read them for compatibility
- **AND** they SHALL NOT be sufficient for a precise Gatekeeper pass without `raw_path`

### Requirement: Gatekeeper SHALL validate evidence reference fidelity
Gatekeeper SHALL validate that Executor evidence bindings point to real current-session evidence references before Verifier uses them as primary evidence.

#### Scenario: Valid precise binding passes
- **WHEN** a binding's `source_invocation_id` exists in the current session
- **AND** the binding's `tool_name` matches the persisted invocation
- **AND** the binding's `raw_path` exists in `retrieval_details.evidence_refs`
- **AND** the binding's `evidence_excerpt` is supported by the matching evidence ref text
- **THEN** Gatekeeper SHALL return `status=pass`
- **AND** Gatekeeper SHALL return `severity=none`

#### Scenario: Missing raw path is low confidence
- **WHEN** a binding references an existing invocation
- **AND** the binding omits `raw_path`
- **THEN** Gatekeeper SHALL return `status=fail`
- **AND** Gatekeeper SHALL return `severity=low_confid`
- **AND** the effective verifier result SHALL NOT be `PASS`

#### Scenario: Old invocation without evidence refs is low confidence
- **WHEN** a binding references an existing invocation
- **AND** the invocation does not contain `retrieval_details.evidence_refs`
- **THEN** Gatekeeper SHALL return `status=fail`
- **AND** Gatekeeper SHALL return `severity=low_confid`
- **AND** the effective verifier result SHALL NOT be `PASS`

#### Scenario: Unknown raw path is rejected
- **WHEN** a binding references an existing invocation
- **AND** the binding's `raw_path` is absent from that invocation's `retrieval_details.evidence_refs`
- **THEN** Gatekeeper SHALL return `status=fail`
- **AND** Gatekeeper SHALL return `severity=reject`
- **AND** `failed_rules` SHALL include `evidence.raw_path`

#### Scenario: Mismatched excerpt is rejected
- **WHEN** a binding references an existing invocation and raw path
- **AND** the binding's `evidence_excerpt` is not supported by the matching system-side evidence ref text
- **THEN** Gatekeeper SHALL return `status=fail`
- **AND** Gatekeeper SHALL return `severity=reject`
- **AND** `failed_rules` SHALL include `evidence.excerpt_mismatch`

### Requirement: Verifier SHALL use verified claim-local evidence for derivability
Verifier SHALL judge structured claims only against Gatekeeper-verified claim-local evidence excerpts and their precise current-run references.

#### Scenario: Verified excerpt supports direct observation
- **WHEN** verdict ceiling is PASS
- **AND** a claim's verified evidence matched text directly contains the claim's concrete facts
- **THEN** Verifier MAY classify that claim as `direct_observation`

#### Scenario: Verified evidence is complete Verifier context
- **WHEN** verified claims and evidence are available
- **THEN** Verifier SHALL use them as its evidence context
- **AND** it SHALL NOT require or request a complete tool trace summary
- **AND** it SHALL NOT read raw Executor or unreferenced tool material

### Requirement: Gatekeeper severity SHALL constrain effective verdict
Runtime effective verdict calculation SHALL treat normalized Gatekeeper ceiling as a hard upper bound independent from Verifier model output.

#### Scenario: Reject severity bypasses Verifier
- **WHEN** `gatekeeper_result.severity=reject`
- **THEN** the Graph SHALL route to pre-verification Fallback without invoking Verifier
- **AND** it SHALL NOT fabricate an effective diagnostic verdict

#### Scenario: Low confidence severity prevents PASS
- **WHEN** `gatekeeper_result.severity=low_confid`
- **AND** the Verifier model returns `verdict=PASS`
- **THEN** deterministic effective-verdict calculation SHALL downgrade the result
- **AND** effective verdict SHALL be `LOW_CONFID`

#### Scenario: Gatekeeper audit includes severity
- **WHEN** Graph verifier evaluation is persisted
- **THEN** `diagnosis_run.self_evaluation.verifier_evaluation.gatekeeper_result` SHALL include available `status`, `severity`, `checked_bindings`, `failed_rules`, `warnings`, and `errors`

### Requirement: Verifier input hook SHALL only perform narrow compatibility backfill
The verifier input hook SHALL avoid converting broad tool summaries into precise evidence references.

#### Scenario: Unique invocation candidate may be backfilled
- **WHEN** an evidence binding omits `source_invocation_id`
- **AND** exactly one current-session invocation exists for the binding's `tool_name`
- **THEN** the hook MAY backfill `source_invocation_id`
- **AND** it SHALL add a Gatekeeper warning describing the auto-backfill

#### Scenario: Raw path is never backfilled
- **WHEN** an evidence binding omits `raw_path`
- **THEN** the hook SHALL NOT synthesize `raw_path`
- **AND** Gatekeeper SHALL treat the binding as not precise enough to pass

### Requirement: Executor prompt SHALL constrain narrow-scope over-expansion
The Executor prompt SHALL instruct Executor to keep narrow confirmation questions focused on observation-level claims.

#### Scenario: Narrow scope produces minimal observation claims
- **WHEN** the user asks to confirm one specific service, alert, or symptom
- **THEN** Executor SHOULD output the minimum necessary claims, normally one and at most two
- **AND** those claims SHALL be `observation` or `negative_observation` unless current-session evidence proves more
- **AND** Executor SHALL NOT emit unrelated root-cause, remediation, or excluded-topic claims as confirmed facts

### Requirement: Gatekeeper SHALL expose rule catalog metadata in audit output
Gatekeeper SHALL include the rule catalog version and enabled rule metadata in its validation result.

#### Scenario: Gatekeeper pass includes rule metadata
- **WHEN** Gatekeeper returns `status=pass`
- **THEN** the result SHALL include `rule_set_version`
- **AND** it SHALL include a rule metadata summary

#### Scenario: Gatekeeper fail includes rule metadata
- **WHEN** Gatekeeper returns `status=fail`
- **THEN** the result SHALL include `rule_set_version`
- **AND** it SHALL include a rule metadata summary

#### Scenario: Gatekeeper fallback pass includes rule metadata
- **WHEN** the Verifier input hook returns a fallback Gatekeeper pass because no Gatekeeper service is available
- **THEN** the result SHOULD still include the default rule set version and an empty or default rule metadata summary

### Requirement: Gatekeeper rule catalog SHALL remain deterministic
The Gatekeeper rule catalog SHALL configure metadata and simple parameters only; validation behavior SHALL remain deterministic Java code.

#### Scenario: Rule metadata is lightweight
- **WHEN** rule metadata is loaded
- **THEN** each enabled rule SHOULD expose an id, description, enabled flag, and default severity or relevant parameter
- **AND** rule metadata SHALL NOT execute dynamic scripts