84 lines
5.1 KiB
Markdown
84 lines
5.1 KiB
Markdown
# chat-composer-agent Specification
|
|
|
|
## Purpose
|
|
TBD - created by archiving change executor-composer-final-answer. Update Purpose after archive.
|
|
## Requirements
|
|
### Requirement: Composer SHALL generate final user-facing Chat answers
|
|
The system SHALL invoke the Composer Graph Node after Verifier routing to generate the final user-facing Chat answer from Verifier-allowed material.
|
|
|
|
#### Scenario: Composer receives only filtered material
|
|
- **WHEN** the Composer Graph Node invokes its configured Agent
|
|
- **THEN** the Composer input SHALL contain `original_query`, `verdict`, `allowed_claims`, `allowed_hypotheses`, `missing_info`, `recommended_actions`, and `rationale`
|
|
- **AND** the Composer input SHALL NOT contain raw tool output
|
|
- **AND** the Composer input SHALL NOT contain the full unscreened Executor output
|
|
- **AND** the Composer input SHALL NOT contain Executor `user_facing_answer`
|
|
|
|
#### Scenario: Composer outputs strict JSON
|
|
- **WHEN** Composer completes
|
|
- **THEN** it SHALL output exactly one JSON object
|
|
- **AND** the JSON object SHALL include `answer_summary`, `recommended_actions`, and `user_facing_answer`
|
|
- **AND** it SHALL NOT output Markdown, code fences, or explanatory text outside the JSON object
|
|
|
|
#### Scenario: Composer does not introduce new facts
|
|
- **WHEN** Composer produces `answer_summary`, `recommended_actions`, or `user_facing_answer`
|
|
- **THEN** every service name, entity, timestamp, error code, metric value, root cause, and recommendation reason SHALL be derived from the Composer input
|
|
- **AND** Composer SHALL NOT add facts from model knowledge, raw tool history, or Executor raw text
|
|
|
|
### Requirement: Composer input SHALL honor Verifier claim checks
|
|
The Composer Graph Node SHALL construct Composer input by filtering verified Executor structured output through Verifier `claim_checks`.
|
|
|
|
#### Scenario: Passing claims become allowed claims
|
|
- **WHEN** a claim check verification is `direct_observation`
|
|
- **THEN** the Composer input builder SHALL include the matching verified Executor claim in `allowed_claims`
|
|
|
|
#### Scenario: Reasonable inferences remain bounded
|
|
- **WHEN** a claim check verification is `reasonable_inference`
|
|
- **THEN** the Composer input builder MAY include the matching verified Executor claim in `allowed_claims`
|
|
- **AND** the final answer SHALL NOT describe it as the sole confirmed root cause unless the allowed claim itself is a root-cause claim and the final verdict is `PASS`
|
|
|
|
#### Scenario: Overstated claims are not confirmed findings
|
|
- **WHEN** a claim check verification is `overstated`
|
|
- **THEN** the Composer input builder SHALL NOT include the matching claim as a confirmed item in `allowed_claims`
|
|
- **AND** it MAY include it as `allowed_hypotheses` or represent it in `missing_info`
|
|
|
|
#### Scenario: Unsupported or external claims are withheld
|
|
- **WHEN** a claim check verification is `unsupported`, `external_unknown`, or `contradicted`
|
|
- **THEN** the Composer input builder SHALL NOT include the matching claim in `allowed_claims`
|
|
- **AND** the final user-facing answer SHALL NOT present that claim as confirmed
|
|
|
|
### Requirement: Composer SHALL respect verdict-specific wording
|
|
Composer SHALL phrase final answers according to the effective Verifier verdict.
|
|
|
|
#### Scenario: PASS answer uses confirmed material
|
|
- **WHEN** the effective verdict is `PASS`
|
|
- **THEN** the final answer MAY state confirmed findings from `allowed_claims`
|
|
- **AND** it SHALL only state root cause confirmed when an allowed root-cause claim is present
|
|
|
|
#### Scenario: LOW_CONFID answer separates findings and gaps
|
|
- **WHEN** the effective verdict is `LOW_CONFID`
|
|
- **THEN** the final answer SHALL distinguish confirmed information from possible directions
|
|
- **AND** it SHALL mention evidence gaps from `missing_info`
|
|
- **AND** it SHALL NOT turn `allowed_hypotheses` into confirmed findings
|
|
|
|
#### Scenario: REJECT answer avoids root-cause conclusions
|
|
- **WHEN** the effective verdict is `REJECT`
|
|
- **THEN** Composer input SHALL have `allowed_hypotheses=[]`
|
|
- **AND** the final answer SHALL state that current evidence cannot support a reliable conclusion
|
|
- **AND** the final answer SHALL NOT include a root-cause conclusion
|
|
|
|
### Requirement: Composer failures SHALL degrade safely
|
|
The system SHALL tolerate malformed or failed Composer execution without leaking raw JSON or unverified Executor material.
|
|
|
|
#### Scenario: malformed Composer output falls back safely
|
|
- **WHEN** Composer exhausts its fixed-input technical retry or returns a non-retryable failure
|
|
- **THEN** the deterministic Fallback Node SHALL produce a final answer using only filtered material
|
|
- **AND** the final answer SHALL NOT expose raw Composer output
|
|
- **AND** the final answer SHALL NOT expose raw Executor output
|
|
- **AND** the final answer SHALL NOT use Executor `user_facing_answer`
|
|
|
|
#### Scenario: Composer audit is persisted
|
|
- **WHEN** Graph result mapping persists verifier evaluation
|
|
- **THEN** `diagnosis_run.self_evaluation.verifier_evaluation.composer_output` SHALL record parsed Composer audit when available
|
|
- **AND** handled Composer fallback SHALL be observable through orchestration trace and available status/reason fields
|
|
- **AND** the audit SHALL remain compact and SHALL NOT store full raw tool output
|