26 lines
1.6 KiB
Markdown
26 lines
1.6 KiB
Markdown
## ADDED Requirements
|
|
|
|
### Requirement: Evidence summaries SHALL preserve strongest concrete support across merged invocations
|
|
When multiple invocations are merged into one verifier-facing summary entry, the summary SHALL preserve concrete support from the strongest successful invocation.
|
|
|
|
#### Scenario: Merged log calls include one direct hit
|
|
- **WHEN** repeated `query_logs` calls for the same topic are merged
|
|
- **AND** at least one call contains a concrete direct evidence hit
|
|
- **THEN** the merged summary SHALL include a bounded direct-hit excerpt
|
|
- **AND** it SHALL preserve all contributing `source_invocation_ids`
|
|
|
|
#### Scenario: Truncated preview still contains direct evidence
|
|
- **WHEN** a persisted output preview is marked truncated
|
|
- **AND** the preview contains a concrete direct evidence hit
|
|
- **THEN** the summary SHALL treat the row as successful evidence
|
|
- **AND** it SHALL NOT downgrade the row to no-evidence only because the raw output was truncated
|
|
|
|
### Requirement: Live trace counts SHALL distinguish model steps from evidence tool invocations
|
|
The persisted trace SHALL make it possible to audit model step counts separately from evidence tool invocation counts.
|
|
|
|
#### Scenario: Diagnosis session exposes aggregate counts
|
|
- **WHEN** a diagnosis session completes
|
|
- **THEN** `diagnosis_session.step_count` SHALL count persisted Agent model steps
|
|
- **AND** `diagnosis_session.tool_call_count` SHALL count persisted evidence-tool invocation rows
|
|
- **AND** helper workflow calls that are not evidence rows SHALL be auditable from agent steps or logs without inflating `tool_invocation`
|