first commit

skills/llm-summary-review/SKILL.md

@@ -0,0 +1,101 @@
+---
+name: llm-summary-review
+description: Validate and refine LLM-generated summary JSON for extracted articles in this repository. Use when the user wants to review, validate, repair, or iterate on `outputs/result.json` or similar summary outputs produced from extracted article JSON.
+---
+
+# LLM Summary Review
+
+Use this skill when working on the repository's summary loop after article extraction is done.
+
+## What This Skill Does
+
+- Verifies that an LLM summary result matches the expected JSON contract
+- Reuses the repository validator instead of re-checking fields manually
+- Repairs invalid outputs by telling the LLM exactly what to fix
+- Keeps the workflow aligned with the extraction JSON produced by this project
+
+## Inputs
+
+Typical files:
+
+- Extracted article JSON: `outputs/*.extracted.json`
+- LLM summary result JSON: `outputs/result.json`
+- Prompt template: `outputs/llm-summary-prompt.txt`
+
+## Workflow
+
+1. Validate the current summary result with the repository validator:
+
+```bash
+python -m summary_mcp.validate_llm_result outputs/result.json --extracted outputs/read-flow-2026.extracted.json
+```
+
+2. If validation passes:
+   - Report that the result is structurally valid
+   - Briefly note any warnings
+   - Do not rewrite the result unless the user asks
+
+3. If validation fails:
+   - Read the validator errors carefully
+   - Ask the LLM to regenerate or repair only the failing parts
+   - Re-run the validator until it passes or a retry limit is hit
+
+## Repair Prompt Pattern
+
+When asking an LLM to repair a bad result, provide:
+
+- The original extracted article JSON
+- The current invalid summary JSON
+- The validator error list
+- A strict instruction to preserve valid fields and fix only the failing ones
+
+Use this repair template:
+
+```text
+请修复下面这份不符合要求的摘要 JSON。
+
+要求：
+- 只输出合法 JSON
+- 保留已经正确的字段
+- 只修复 validator 报出的错误
+- 不要补充解释
+
+validator errors:
+{{errors}}
+
+原始提取结果:
+{{extracted_json}}
+
+当前摘要结果:
+{{result_json}}
+```
+
+## Validation Rules
+
+The validator currently enforces:
+
+- Required fields exist
+- Field types are correct
+- `category` is one of: `资讯` `方法论` `工具实践` `观点评论`
+- `summary` length is within bounds
+- `highlights`, `keywords`, and `topics` counts are within bounds
+- `keywords` and `topics` do not overlap
+- `title` and `url` match the extracted article when an extracted JSON file is provided
+
+## Repository Implementation
+
+Relevant code:
+
+- Validator model: `src/summary_mcp/models/llm_result.py`
+- Validator logic: `src/summary_mcp/validators/llm_result.py`
+- CLI entry: `src/summary_mcp/validate_llm_result.py`
+
+Prefer using the existing validator rather than recreating checks in free-form reasoning.
+
+## When To Stop
+
+Stop when one of these is true:
+
+- The validator returns `valid: true`
+- The user asks to inspect the remaining failures manually
+- Repeated retries fail and the user should decide how to proceed

skills/llm-summary-review/agents/openai.yaml

@@ -0,0 +1,3 @@
+display_name: LLM Summary Review
+short_description: Validate and repair summary JSON outputs for this repository.
+default_prompt: Validate an LLM summary JSON against the repository rules, explain any failures, and repair the output if needed.

skills/llm-summary-review/scripts/validate_result.py

@@ -0,0 +1,37 @@
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from pathlib import Path
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser(description="Validate an LLM summary JSON using the repository validator.")
+    parser.add_argument("result", type=Path, help="Path to the summary result JSON file")
+    parser.add_argument("--extracted", type=Path, default=None, help="Optional extracted article JSON file")
+    args = parser.parse_args()
+
+    repo_root = Path(__file__).resolve().parents[3]
+    sys.path.insert(0, str(repo_root / "src"))
+
+    from summary_mcp.validators.llm_result import validate_llm_result
+
+    report = validate_llm_result(args.result, args.extracted)
+    print(
+        json.dumps(
+            {
+                "valid": report.valid,
+                "errors": report.errors,
+                "warnings": report.warnings,
+                "normalized_result": report.normalized_result,
+            },
+            ensure_ascii=False,
+            indent=2,
+        )
+    )
+    raise SystemExit(0 if report.valid else 1)
+
+
+if __name__ == "__main__":
+    main()