Files

4.1 KiB

MODIFIED Requirements

Requirement: Deterministic Draft and evidence validation

For a Draft with a non-null Conclusion, the Harness SHALL deterministically reject it unless every Analysis has a unique non-blank Analysis ID, a supported kind, non-blank text, and at least one Tool Call ID, and every Conclusion, Action Plan item, and Recommendation has non-empty references to existing Analysis IDs. For a Draft with conclusion=null, Release SHALL NOT require normal conclusion structure or invoke EvidenceRepair; any supplied Tool references SHALL still resolve to current-Run READY canonical invocations and SHALL obey positive/negative evidence semantics.

Scenario: Duplicate or missing Analysis ID in concluded Draft

  • WHEN a Draft with a Conclusion contains a blank or duplicate Analysis ID
  • THEN EvidenceGuard returns violations and SemanticGuard is not invoked

Scenario: Broken report reference in concluded Draft

  • WHEN a Conclusion, Action Plan item, or Recommendation has an empty or unknown Analysis reference
  • THEN EvidenceGuard rejects the Draft before semantic review

Scenario: No-conclusion Draft has valid negative observation

  • WHEN a conclusion=null Draft cites a current-Run READY NO_EVIDENCE call as NEGATIVE_OBSERVATION
  • THEN Release accepts the reference authenticity without running EvidenceRepair or SemanticGuard

Scenario: No-conclusion Draft fabricates a Tool reference

  • WHEN a conclusion=null Draft cites a missing, cross-Run, incomplete or ERROR Tool call
  • THEN the reference is excluded and cannot be published as an observed fact

Requirement: Fail-closed release policy

The release use case SHALL publish the unchanged verified Draft only when a non-null Conclusion passes EvidenceGuard and SemanticGuard returns SUPPORTED. It SHALL publish fixed SafeFallback content for evidence failure, semantic unsupported, final semantic technical failure, a valid no-conclusion Draft, information saturation, or handled budget termination. No-conclusion and controlled-stop release SHALL be deterministic from verified references and ProgressSnapshot and MUST NOT invoke a repair or semantic model call. A fallback result MUST NOT include an unsupported Draft, full verified snapshot, internal stop counters, or SemanticGuard reason.

Scenario: Supported report release

  • WHEN EvidenceGuard succeeds for a concluded Draft and SemanticGuard returns SUPPORTED
  • THEN release outcome is SUCCESS and the same verified Draft semantics are returned without summarization or partial editing

Scenario: Unsupported report fallback

  • WHEN SemanticGuard returns UNSUPPORTED
  • THEN release outcome is FALLBACK, type is SEMANTIC_UNSUPPORTED, and verified sources are derived only from the snapshot

Scenario: Semantic review remains unavailable

  • WHEN all permitted technical attempts fail
  • THEN release outcome is FALLBACK, type is SEMANTIC_UNAVAILABLE, and no Draft or internal failure reason is exposed

Scenario: Evidence validation fallback sources

  • WHEN evidence repair fails or the second EvidenceGuard rejects a concluded Draft
  • THEN release outcome is FALLBACK, type is EVIDENCE_VALIDATION_FAILED, and verified_sources is empty

Scenario: Missing context ends without Tool calls

  • WHEN a valid no-conclusion Draft has no Tool calls and identifies required missing context
  • THEN release outcome is FALLBACK, type is MISSING_REQUIRED_CONTEXT, and no guard model call occurs

Scenario: Finite checks do not support a conclusion

  • WHEN a no-conclusion Draft or controlled stop has a non-empty verified ProgressSnapshot
  • THEN release outcome is FALLBACK, type is INSUFFICIENT_EVIDENCE, and observed facts describe only actual completed checks

Scenario: Invalid Draft has publishable progress

  • WHEN the Agent's final Draft is rejected by strict parsing but its bounded ProgressSnapshot contains current-Run verified observed facts
  • THEN Release publishes FALLBACK with type INSUFFICIENT_EVIDENCE using only that snapshot and MUST NOT use any content from the invalid Draft