Files
T

4.8 KiB

ADDED Requirements

Requirement: Composer SHALL generate final user-facing Chat answers

The system SHALL invoke a Composer expression layer after Verifier to generate the final user-facing Chat answer from Verifier-allowed material.

Scenario: Composer receives only filtered material

  • WHEN ChatService invokes Composer
  • THEN the Composer input SHALL contain original_query, verdict, allowed_claims, allowed_hypotheses, missing_info, recommended_actions, and rationale
  • AND the Composer input SHALL NOT contain raw tool output
  • AND the Composer input SHALL NOT contain the full unscreened Executor output
  • AND the Composer input SHALL NOT contain Executor user_facing_answer

Scenario: Composer outputs strict JSON

  • WHEN Composer completes
  • THEN it SHALL output exactly one JSON object
  • AND the JSON object SHALL include answer_summary, recommended_actions, and user_facing_answer
  • AND it SHALL NOT output Markdown, code fences, or explanatory text outside the JSON object

Scenario: Composer does not introduce new facts

  • WHEN Composer produces answer_summary, recommended_actions, or user_facing_answer
  • THEN every service name, entity, timestamp, error code, metric value, root cause, and recommendation reason SHALL be derived from the Composer input
  • AND Composer SHALL NOT add facts from model knowledge, raw tool history, or Executor raw text

Requirement: Composer input SHALL honor Verifier claim checks

ChatService SHALL construct Composer input by filtering Executor structured output through Verifier claim_checks.

Scenario: Passing claims become allowed claims

  • WHEN a claim check verification is direct_observation
  • THEN ChatService SHALL include the matching Executor claim in allowed_claims

Scenario: Reasonable inferences remain bounded

  • WHEN a claim check verification is reasonable_inference
  • THEN ChatService MAY include the matching Executor claim in allowed_claims
  • AND the final answer SHALL NOT describe it as the sole confirmed root cause unless the allowed claim itself is a root-cause claim and the final verdict is PASS

Scenario: Overstated claims are not confirmed findings

  • WHEN a claim check verification is overstated
  • THEN ChatService SHALL NOT include the matching Executor claim as a confirmed item in allowed_claims
  • AND ChatService MAY include it as allowed_hypotheses or represent it in missing_info

Scenario: Unsupported or external claims are withheld

  • WHEN a claim check verification is unsupported, external_unknown, or contradicted
  • THEN ChatService SHALL NOT include the matching Executor claim in allowed_claims
  • AND the final user-facing answer SHALL NOT present that claim as confirmed

Requirement: Composer SHALL respect verdict-specific wording

Composer SHALL phrase final answers according to the effective Verifier verdict.

Scenario: PASS answer uses confirmed material

  • WHEN the effective verdict is PASS
  • THEN the final answer MAY state confirmed findings from allowed_claims
  • AND it SHALL only state root cause confirmed when an allowed root-cause claim is present

Scenario: LOW_CONFID answer separates findings and gaps

  • WHEN the effective verdict is LOW_CONFID
  • THEN the final answer SHALL distinguish confirmed information from possible directions
  • AND it SHALL mention evidence gaps from missing_info
  • AND it SHALL NOT turn allowed_hypotheses into confirmed findings

Scenario: REJECT answer avoids root-cause conclusions

  • WHEN the effective verdict is REJECT
  • THEN Composer input SHALL have allowed_hypotheses=[]
  • AND the final answer SHALL state that current evidence cannot support a reliable conclusion
  • AND the final answer SHALL NOT include a root-cause conclusion

Requirement: Composer failures SHALL degrade safely

The system SHALL tolerate malformed Composer output without leaking raw JSON or unverified Executor material.

Scenario: malformed Composer output falls back safely

  • WHEN Composer returns malformed JSON or omits required fields
  • THEN ChatService SHALL produce a final answer using a fixed safe fallback template based only on filtered material
  • AND the final answer SHALL NOT expose raw Composer output
  • AND the final answer SHALL NOT expose raw Executor output
  • AND the final answer SHALL NOT use Executor user_facing_answer

Scenario: Composer audit is persisted

  • WHEN ChatService persists verifier evaluation
  • THEN diagnosis_session.self_evaluation.verifier_evaluation.composer_output SHALL record whether Composer output was valid or fallback was used
  • AND the audit SHALL include the parsed Composer fields when valid
  • AND the audit SHALL remain compact and SHALL NOT store full raw tool output