# single-react-evidence-semantic-guards Specification ## Purpose TBD - created by archiving change single-react-evidence-semantic-guards. Update Purpose after archive. ## Requirements ### Requirement: Deterministic Draft and evidence validation For a Draft with a non-null Conclusion, the Harness SHALL deterministically reject it unless every Analysis has a unique non-blank Analysis ID, a supported kind, non-blank text, and at least one Tool Call ID, and every Conclusion, Action Plan item, and Recommendation has non-empty references to existing Analysis IDs. For a Draft with `conclusion=null`, Release SHALL NOT require normal conclusion structure or invoke EvidenceRepair; any supplied Tool references SHALL still resolve to current-Run READY canonical invocations and SHALL obey positive/negative evidence semantics. #### Scenario: Duplicate or missing Analysis ID in concluded Draft - **WHEN** a Draft with a Conclusion contains a blank or duplicate Analysis ID - **THEN** EvidenceGuard returns violations and SemanticGuard is not invoked #### Scenario: Broken report reference in concluded Draft - **WHEN** a Conclusion, Action Plan item, or Recommendation has an empty or unknown Analysis reference - **THEN** EvidenceGuard rejects the Draft before semantic review #### Scenario: No-conclusion Draft has valid negative observation - **WHEN** a `conclusion=null` Draft cites a current-Run READY `NO_EVIDENCE` call as `NEGATIVE_OBSERVATION` - **THEN** Release accepts the reference authenticity without running EvidenceRepair or SemanticGuard #### Scenario: No-conclusion Draft fabricates a Tool reference - **WHEN** a `conclusion=null` Draft cites a missing, cross-Run, incomplete or ERROR Tool call - **THEN** the reference is excluded and cannot be published as an observed fact ### Requirement: Current Run canonical invocation ownership EvidenceGuard SHALL resolve each referenced Tool Call through `runId + toolCallId` and SHALL accept only an invocation owned by the current Run with lifecycle `READY`, a non-empty `agent_result`, and evidence status `EVIDENCE_FOUND` or `NO_EVIDENCE`. #### Scenario: Fabricated or cross-Run Tool Call - **WHEN** a Draft references a missing Tool Call or the resolved record belongs to another Run - **THEN** EvidenceGuard rejects the reference and does not expose any record content to SemanticGuard #### Scenario: Failed or incomplete invocation - **WHEN** a referenced invocation is `PROJECTING`, `ERROR`, lacks `agent_result`, or has `evidence_status=ERROR` - **THEN** EvidenceGuard rejects the Draft ### Requirement: Analysis kind matches evidence semantics EvidenceGuard SHALL permit `NORMAL` Analysis only with `EVIDENCE_FOUND` calls and SHALL permit `NEGATIVE_OBSERVATION` Analysis only with `NO_EVIDENCE` calls. #### Scenario: Valid negative observation - **WHEN** a `NEGATIVE_OBSERVATION` references a READY `NO_EVIDENCE` projection with its query scope and zero-match data - **THEN** EvidenceGuard accepts the binding without interpreting it as proof of system health or root-cause exclusion #### Scenario: Positive claim uses no-evidence result - **WHEN** a `NORMAL` Analysis references a `NO_EVIDENCE` invocation - **THEN** EvidenceGuard rejects the binding ### Requirement: Verified evidence snapshot is minimal and deterministic The Harness SHALL strictly parse only supported Tool projections and SHALL construct evidence grouped by Analysis ID from referenced `agent_result` and required bounded request scope. The snapshot MUST NOT contain Tool Call IDs, Redis keys, raw responses, or unreferenced invocations. #### Scenario: Supported RAG, log, and MySQL projections - **WHEN** a Draft references valid RAG, log, or MySQL calls - **THEN** the snapshot contains the corresponding stable source, scope, timestamp, exact excerpt or bounded values grouped under the referencing Analysis #### Scenario: Projection contract mismatch - **WHEN** a projection has an unknown Tool name, invalid JSON, mismatched Tool Call ID, or evidence status inconsistent with its canonical record - **THEN** EvidenceGuard fails closed ### Requirement: Evidence repair is single-turn and semantics-preserving On the first EvidenceGuard failure, the Harness SHALL allow exactly one direct no-Tool model call to repair identifier and reference structure. It MUST NOT rerun the Diagnosis Agent or any Tool, and MUST reject a repair that changes user-visible report semantics. #### Scenario: Structural repair succeeds - **WHEN** the one repair attempt changes only IDs/references and the repaired Draft passes EvidenceGuard - **THEN** the repaired Draft proceeds to SemanticGuard #### Scenario: Repair changes report text - **WHEN** the repair changes Conclusion, Analysis, Action Plan, Recommendation, Limitation text, kind, order, or human-confirmation flag - **THEN** the Harness returns `EVIDENCE_VALIDATION_FAILED` #### Scenario: Second validation fails - **WHEN** the repaired Draft still fails EvidenceGuard - **THEN** the Harness returns `EVIDENCE_VALIDATION_FAILED` with empty verified sources and does not invoke SemanticGuard ### Requirement: Isolated single-turn SemanticGuard SemanticGuard SHALL reuse the system ChatModel through a fresh single-turn Prompt containing only the original Query, the complete user-visible Draft without Tool Call IDs, and the verified evidence snapshot. It MUST have no Tool, memory, ReAct loop, Redis access, raw response, or callback to the Diagnosis Agent. #### Scenario: Semantic input isolation - **WHEN** a verified Draft enters SemanticGuard - **THEN** the model sees the original Query, all report sections and verified evidence, but no Tool Call ID, Redis key, raw response, diagnosis history, or Tool definition #### Scenario: Binary review output - **WHEN** SemanticGuard completes normally - **THEN** it returns only `SUPPORTED` or `UNSUPPORTED` with a non-blank audit reason and cannot return a corrected report ### Requirement: Semantic model budgets timeout cancellation and retry The Harness SHALL enforce input/output byte limits, Run byte/model/token budgets, per-attempt timeout, total SemanticGuard timeout, Run cancellation, strict JSON parsing, and the configured two-attempt technical retry policy. It SHALL retry only timeout, transport, parse, or schema failures and SHALL use the exact same input for both attempts. #### Scenario: Technical failure then success - **WHEN** the first SemanticGuard attempt times out or returns invalid output and the second attempt returns a valid verdict - **THEN** exactly two model attempts are recorded and the second verdict controls release #### Scenario: Unsupported is not retried - **WHEN** SemanticGuard returns valid `UNSUPPORTED` - **THEN** the Harness records one attempt and immediately applies the unsupported fallback #### Scenario: Run cancellation during model call - **WHEN** the Run is cancelled while a guard model call is pending - **THEN** the Future is cancelled, no late model result is released, and cancellation is not converted into a normal Fallback ### Requirement: Fail-closed release policy The release use case SHALL publish the unchanged verified Draft only when a non-null Conclusion passes EvidenceGuard and SemanticGuard returns `SUPPORTED`. It SHALL publish fixed `SafeFallback` content for evidence failure, semantic unsupported, final semantic technical failure, a valid no-conclusion Draft, information saturation, or handled budget termination. No-conclusion and controlled-stop release SHALL be deterministic from verified references and ProgressSnapshot and MUST NOT invoke a repair or semantic model call. A fallback result MUST NOT include an unsupported Draft, full verified snapshot, internal stop counters, or SemanticGuard reason. #### Scenario: Supported report release - **WHEN** EvidenceGuard succeeds for a concluded Draft and SemanticGuard returns `SUPPORTED` - **THEN** release outcome is `SUCCESS` and the same verified Draft semantics are returned without summarization or partial editing #### Scenario: Unsupported report fallback - **WHEN** SemanticGuard returns `UNSUPPORTED` - **THEN** release outcome is `FALLBACK`, type is `SEMANTIC_UNSUPPORTED`, and verified sources are derived only from the snapshot #### Scenario: Semantic review remains unavailable - **WHEN** all permitted technical attempts fail - **THEN** release outcome is `FALLBACK`, type is `SEMANTIC_UNAVAILABLE`, and no Draft or internal failure reason is exposed #### Scenario: Evidence validation fallback sources - **WHEN** evidence repair fails or the second EvidenceGuard rejects a concluded Draft - **THEN** release outcome is `FALLBACK`, type is `EVIDENCE_VALIDATION_FAILED`, and `verified_sources` is empty #### Scenario: Missing context ends without Tool calls - **WHEN** a valid no-conclusion Draft has no Tool calls and identifies required missing context - **THEN** release outcome is `FALLBACK`, type is `MISSING_REQUIRED_CONTEXT`, and no guard model call occurs #### Scenario: Finite checks do not support a conclusion - **WHEN** a no-conclusion Draft or controlled stop has a non-empty verified ProgressSnapshot - **THEN** release outcome is `FALLBACK`, type is `INSUFFICIENT_EVIDENCE`, and observed facts describe only actual completed checks #### Scenario: Invalid Draft has publishable progress - **WHEN** the Agent's final Draft is rejected by strict parsing but its bounded ProgressSnapshot contains current-Run verified observed facts - **THEN** Release publishes `FALLBACK` with type `INSUFFICIENT_EVIDENCE` using only that snapshot and MUST NOT use any content from the invalid Draft ### Requirement: Stage-five public isolation The stage-five implementation SHALL remain internal and MUST NOT switch public Chat, AiOps, SSE, persistence, or legacy multi-Agent behavior. #### Scenario: Focused implementation scope - **WHEN** stage-five changes are inspected - **THEN** only internal guard/release code, prompts, tests, OpenSpec and devflow artifacts have changed