SuperBizAgent-java/openspec/specs/diagnosis-playbook-skills/spec.md

diagnosis-playbook-skills Specification

Purpose

Provide versionable diagnosis playbook skills for high-frequency MVP troubleshooting flows, loaded through progressive disclosure so agents can follow scenario-specific evidence workflows without bloating every prompt.

Requirements

Requirement: Skill catalog SHALL expose diagnosis playbooks compactly

The system SHALL provide a compact skill catalog containing each playbook skill name and description.

Scenario: Planner or Executor receives available skill metadata

• GIVEN classpath skill folders exist under skills/
• WHEN Chat or AIOps Planner/Executor agents are built
• THEN their system prompts SHALL include a compact diagnosis skill catalog
• AND the catalog SHALL include skill names and descriptions only, not full skill bodies

Requirement: Executor SHALL read full playbook instructions on demand

The system SHALL expose a read_skill tool to Executor agents for loading a full SKILL.md body by skill name.

Scenario: Executor reads an existing skill

• GIVEN a skill named diagnose-mysql-connection-pool
• WHEN the Executor calls read_skill with that name
• THEN the tool SHALL return the full skill instructions
• AND the result SHALL include the skill name

Scenario: Executor requests an unknown skill

• WHEN the Executor calls read_skill with an unknown name
• THEN the tool SHALL return a bounded error message
• AND the message SHALL list valid skill names

Requirement: Playbook skills SHALL preserve evidence and verifier boundaries

The system SHALL keep skills as workflow guidance and keep factual evidence collection in existing evidence tools.

Scenario: Executor uses a playbook

• WHEN a diagnosis playbook applies to a user issue
• THEN the Executor SHALL use the playbook to decide evidence order and stop conditions
• AND factual claims SHALL still be supported by lookup_knowledge, query_logs, query_metrics, or alert tools
• AND Chat Verifier SHALL continue to validate only existing tool_trace_summary

Requirement: Initial playbook set SHALL cover fixed MVP diagnosis cases

The system SHALL provide playbooks for the existing fixed diagnosis evaluation scenarios.

Scenario: Fixed diagnosis case has a matching playbook

• WHEN the case is payment timeout, MySQL pool exhaustion, Redis timeout, slow response, or JVM memory risk
• THEN a matching diagnosis skill SHALL exist
• AND the skill SHALL state required evidence tools and low-confidence behavior

Requirement: Executor SHALL avoid duplicate playbook reads in one diagnosis path

The system SHALL prevent repeated reads of the same selected diagnosis playbook during a single Executor diagnosis path unless a new retry round explicitly requests a fresh playbook read.

Scenario: Executor has already read the selected skill

• WHEN the Executor has already called read_skill for the selected skill in the current diagnosis path
• THEN the Executor SHALL continue using the already loaded playbook instructions
• AND it SHALL NOT call read_skill again for the same skill in that path

Scenario: Retry round may read selected skill again

• WHEN ChatService starts a distinct retry round after Verifier requests evidence supplementation
• THEN the Executor MAY call read_skill again for the selected skill
• AND the repeated read SHALL be attributable to the new round rather than the same Executor path

Requirement: Live traces SHALL expose skill selection boundaries

The system SHALL make the Planner and Executor skill boundary auditable from persisted agent steps and logs.

Scenario: Planner selects without reading skill body

• WHEN a Planner step selects a diagnosis playbook
• THEN the persisted Planner output SHALL include the selected skill name
• AND the Planner step SHALL NOT include a read_skill tool call

Scenario: Executor reads selected skill

• WHEN the Executor starts a playbook-backed diagnosis
• THEN the persisted Executor step SHALL show read_skill for the selected skill before evidence-tool execution