reader/skills/llm-summary-review/SKILL.md

name, description

name	description
llm-summary-review	Validate and refine LLM-generated summary JSON for extracted articles in this repository. Use when the user wants to review, validate, repair, or iterate on `outputs/result.json` or similar summary outputs produced from extracted article JSON.

LLM Summary Review

Use this skill when working on the repository's summary loop after article extraction is done.

What This Skill Does

• Verifies that an LLM summary result matches the expected JSON contract
• Reuses the repository validator instead of re-checking fields manually
• Repairs invalid outputs by telling the LLM exactly what to fix
• Keeps the workflow aligned with the extraction JSON produced by this project

Inputs

Typical files:

• Extracted article JSON: outputs/*.extracted.json
• LLM summary result JSON: outputs/result.json
• Prompt template: outputs/llm-summary-prompt.txt

Workflow

1. Validate the current summary result with the repository validator:

python -m summary_mcp.validate_llm_result outputs/result.json --extracted outputs/read-flow-2026.extracted.json

2. If validation passes:

   • Report that the result is structurally valid
   • Briefly note any warnings
   • Do not rewrite the result unless the user asks

3. If validation fails:

   • Read the validator errors carefully
   • Ask the LLM to regenerate or repair only the failing parts
   • Re-run the validator until it passes or a retry limit is hit

Repair Prompt Pattern

When asking an LLM to repair a bad result, provide:

• The original extracted article JSON
• The current invalid summary JSON
• The validator error list
• A strict instruction to preserve valid fields and fix only the failing ones

Use this repair template:

请修复下面这份不符合要求的摘要 JSON。

要求：
- 只输出合法 JSON
- 保留已经正确的字段
- 只修复 validator 报出的错误
- 不要补充解释

validator errors:
{{errors}}

原始提取结果:
{{extracted_json}}

当前摘要结果:
{{result_json}}

Validation Rules

The validator currently enforces:

• Required fields exist
• Field types are correct
• category is one of: 资讯 方法论 工具实践 观点评论
• summary length is within bounds
• highlights, keywords, and topics counts are within bounds
• keywords and topics do not overlap
• title and url match the extracted article when an extracted JSON file is provided

Repository Implementation

Relevant code:

• Validator model: src/summary_mcp/models/llm_result.py
• Validator logic: src/summary_mcp/validators/llm_result.py
• CLI entry: src/summary_mcp/validate_llm_result.py

Prefer using the existing validator rather than recreating checks in free-form reasoning.

When To Stop

Stop when one of these is true:

• The validator returns valid: true
• The user asks to inspect the remaining failures manually
• Repeated retries fail and the user should decide how to proceed