52 lines
4.1 KiB
Markdown
52 lines
4.1 KiB
Markdown
## MODIFIED Requirements
|
|
|
|
### Requirement: Deterministic Draft and evidence validation
|
|
For a Draft with a non-null Conclusion, the Harness SHALL deterministically reject it unless every Analysis has a unique non-blank Analysis ID, a supported kind, non-blank text, and at least one Tool Call ID, and every Conclusion, Action Plan item, and Recommendation has non-empty references to existing Analysis IDs. For a Draft with `conclusion=null`, Release SHALL NOT require normal conclusion structure or invoke EvidenceRepair; any supplied Tool references SHALL still resolve to current-Run READY canonical invocations and SHALL obey positive/negative evidence semantics.
|
|
|
|
#### Scenario: Duplicate or missing Analysis ID in concluded Draft
|
|
- **WHEN** a Draft with a Conclusion contains a blank or duplicate Analysis ID
|
|
- **THEN** EvidenceGuard returns violations and SemanticGuard is not invoked
|
|
|
|
#### Scenario: Broken report reference in concluded Draft
|
|
- **WHEN** a Conclusion, Action Plan item, or Recommendation has an empty or unknown Analysis reference
|
|
- **THEN** EvidenceGuard rejects the Draft before semantic review
|
|
|
|
#### Scenario: No-conclusion Draft has valid negative observation
|
|
- **WHEN** a `conclusion=null` Draft cites a current-Run READY `NO_EVIDENCE` call as `NEGATIVE_OBSERVATION`
|
|
- **THEN** Release accepts the reference authenticity without running EvidenceRepair or SemanticGuard
|
|
|
|
#### Scenario: No-conclusion Draft fabricates a Tool reference
|
|
- **WHEN** a `conclusion=null` Draft cites a missing, cross-Run, incomplete or ERROR Tool call
|
|
- **THEN** the reference is excluded and cannot be published as an observed fact
|
|
|
|
### Requirement: Fail-closed release policy
|
|
The release use case SHALL publish the unchanged verified Draft only when a non-null Conclusion passes EvidenceGuard and SemanticGuard returns `SUPPORTED`. It SHALL publish fixed `SafeFallback` content for evidence failure, semantic unsupported, final semantic technical failure, a valid no-conclusion Draft, information saturation, or handled budget termination. No-conclusion and controlled-stop release SHALL be deterministic from verified references and ProgressSnapshot and MUST NOT invoke a repair or semantic model call. A fallback result MUST NOT include an unsupported Draft, full verified snapshot, internal stop counters, or SemanticGuard reason.
|
|
|
|
#### Scenario: Supported report release
|
|
- **WHEN** EvidenceGuard succeeds for a concluded Draft and SemanticGuard returns `SUPPORTED`
|
|
- **THEN** release outcome is `SUCCESS` and the same verified Draft semantics are returned without summarization or partial editing
|
|
|
|
#### Scenario: Unsupported report fallback
|
|
- **WHEN** SemanticGuard returns `UNSUPPORTED`
|
|
- **THEN** release outcome is `FALLBACK`, type is `SEMANTIC_UNSUPPORTED`, and verified sources are derived only from the snapshot
|
|
|
|
#### Scenario: Semantic review remains unavailable
|
|
- **WHEN** all permitted technical attempts fail
|
|
- **THEN** release outcome is `FALLBACK`, type is `SEMANTIC_UNAVAILABLE`, and no Draft or internal failure reason is exposed
|
|
|
|
#### Scenario: Evidence validation fallback sources
|
|
- **WHEN** evidence repair fails or the second EvidenceGuard rejects a concluded Draft
|
|
- **THEN** release outcome is `FALLBACK`, type is `EVIDENCE_VALIDATION_FAILED`, and `verified_sources` is empty
|
|
|
|
#### Scenario: Missing context ends without Tool calls
|
|
- **WHEN** a valid no-conclusion Draft has no Tool calls and identifies required missing context
|
|
- **THEN** release outcome is `FALLBACK`, type is `MISSING_REQUIRED_CONTEXT`, and no guard model call occurs
|
|
|
|
#### Scenario: Finite checks do not support a conclusion
|
|
- **WHEN** a no-conclusion Draft or controlled stop has a non-empty verified ProgressSnapshot
|
|
- **THEN** release outcome is `FALLBACK`, type is `INSUFFICIENT_EVIDENCE`, and observed facts describe only actual completed checks
|
|
|
|
#### Scenario: Invalid Draft has publishable progress
|
|
- **WHEN** the Agent's final Draft is rejected by strict parsing but its bounded ProgressSnapshot contains current-Run verified observed facts
|
|
- **THEN** Release publishes `FALLBACK` with type `INSUFFICIENT_EVIDENCE` using only that snapshot and MUST NOT use any content from the invalid Draft
|