80 lines
4.1 KiB
Markdown
80 lines
4.1 KiB
Markdown
# diagnosis-playbook-skills Specification
|
|
|
|
## Purpose
|
|
|
|
Provide versionable diagnosis playbook skills for high-frequency MVP troubleshooting flows, loaded through progressive disclosure so agents can follow scenario-specific evidence workflows without bloating every prompt.
|
|
## Requirements
|
|
### Requirement: Skill catalog SHALL expose diagnosis playbooks compactly
|
|
|
|
The system SHALL provide a compact skill catalog containing each playbook skill name and description.
|
|
|
|
#### Scenario: Planner or Executor receives available skill metadata
|
|
|
|
- **GIVEN** classpath skill folders exist under `skills/`
|
|
- **WHEN** Chat or AIOps Planner/Executor agents are built
|
|
- **THEN** their system prompts SHALL include a compact diagnosis skill catalog
|
|
- **AND** the catalog SHALL include skill names and descriptions only, not full skill bodies
|
|
|
|
### Requirement: Executor SHALL read full playbook instructions on demand
|
|
|
|
The system SHALL expose a `read_skill` tool to Executor agents for loading a full `SKILL.md` body by skill name.
|
|
|
|
#### Scenario: Executor reads an existing skill
|
|
|
|
- **GIVEN** a skill named `diagnose-mysql-connection-pool`
|
|
- **WHEN** the Executor calls `read_skill` with that name
|
|
- **THEN** the tool SHALL return the full skill instructions
|
|
- **AND** the result SHALL include the skill name
|
|
|
|
#### Scenario: Executor requests an unknown skill
|
|
|
|
- **WHEN** the Executor calls `read_skill` with an unknown name
|
|
- **THEN** the tool SHALL return a bounded error message
|
|
- **AND** the message SHALL list valid skill names
|
|
|
|
### Requirement: Playbook skills SHALL preserve evidence and verifier boundaries
|
|
|
|
The system SHALL keep skills as workflow guidance and keep factual evidence collection in existing evidence tools.
|
|
|
|
#### Scenario: Executor uses a playbook
|
|
|
|
- **WHEN** a diagnosis playbook applies to a user issue
|
|
- **THEN** the Executor SHALL use the playbook to decide evidence order and stop conditions
|
|
- **AND** factual claims SHALL still be supported by `lookup_knowledge`, `query_logs`, `query_metrics`, or alert tools
|
|
- **AND** Chat Verifier SHALL continue to validate only existing `tool_trace_summary`
|
|
|
|
### Requirement: Initial playbook set SHALL cover fixed MVP diagnosis cases
|
|
|
|
The system SHALL provide playbooks for the existing fixed diagnosis evaluation scenarios.
|
|
|
|
#### Scenario: Fixed diagnosis case has a matching playbook
|
|
|
|
- **WHEN** the case is payment timeout, MySQL pool exhaustion, Redis timeout, slow response, or JVM memory risk
|
|
- **THEN** a matching diagnosis skill SHALL exist
|
|
- **AND** the skill SHALL state required evidence tools and low-confidence behavior
|
|
|
|
### Requirement: Executor SHALL avoid duplicate playbook reads in one diagnosis path
|
|
The system SHALL prevent repeated reads of the same selected diagnosis playbook during a single Executor diagnosis path unless a new retry round explicitly requests a fresh playbook read.
|
|
|
|
#### Scenario: Executor has already read the selected skill
|
|
- **WHEN** the Executor has already called `read_skill` for the selected skill in the current diagnosis path
|
|
- **THEN** the Executor SHALL continue using the already loaded playbook instructions
|
|
- **AND** it SHALL NOT call `read_skill` again for the same skill in that path
|
|
|
|
#### Scenario: Retry round may read selected skill again
|
|
- **WHEN** ChatService starts a distinct retry round after Verifier requests evidence supplementation
|
|
- **THEN** the Executor MAY call `read_skill` again for the selected skill
|
|
- **AND** the repeated read SHALL be attributable to the new round rather than the same Executor path
|
|
|
|
### Requirement: Live traces SHALL expose skill selection boundaries
|
|
The system SHALL make the Planner and Executor skill boundary auditable from persisted agent steps and logs.
|
|
|
|
#### Scenario: Planner selects without reading skill body
|
|
- **WHEN** a Planner step selects a diagnosis playbook
|
|
- **THEN** the persisted Planner output SHALL include the selected skill name
|
|
- **AND** the Planner step SHALL NOT include a `read_skill` tool call
|
|
|
|
#### Scenario: Executor reads selected skill
|
|
- **WHEN** the Executor starts a playbook-backed diagnosis
|
|
- **THEN** the persisted Executor step SHALL show `read_skill` for the selected skill before evidence-tool execution
|