osiman/reader
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

keyword cleanup: v2 engine, alias rule layer, LLM semantic suggestions

Browse Source 
- build_review_bundle.py: 新增 _compute_percentile/_compute_growth,
  候选池从固定阈值改为百分位排名 + 增速因子 (v2 policy)
- term_cleanup_policy.json: 升级 v2 schema
- generate_term_cleanup_suggestions.py: 新增 _prepare_alias_suggestions,
  规则层输出 alias (大小写/单复数/分词变体)
- generate_term_cleanup_semantic_suggestions.py: 新增 LLM 语义建议脚本
  (DeepSeek API, 产出 semantic alias/stopword/promote)
- SKILL.md: 更新为 5 Phase 工作流程
- 首轮清洗 apply: interest 54, aliases 17组, stopwords 17个
- docs/design/keyword-cleanup-flow-overview.md: 流程文档
- plans/: 引擎设计方案
This commit is contained in:
root
2026-05-14 17:17:49 +08:00
parent 4399c9ca90
commit 590d050218
13 changed files with 1592 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files
configs/term_change_log.json
+437
View File
@@ -99,6 +99,443 @@
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-04-08.json",
"suggestion_date": "2026-04-08",
"based_on_days": 7
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Anthropic",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=13, days_seen=10, recent_count=13.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Harness Engineering",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=12, days_seen=11, recent_count=12.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Skill",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=11, days_seen=9, recent_count=11.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "上下文工程",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=6, days_seen=6, recent_count=6.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "多Agent协作",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=6, days_seen=6, recent_count=6.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Claude",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=6, days_seen=5, recent_count=6.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "上下文管理",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=6, days_seen=5, recent_count=6.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "渐进式披露",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=5, days_seen=5, recent_count=5.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Skills",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=5, days_seen=4, recent_count=5.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "SKILL.md",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=4, days_seen=4, recent_count=4.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "CLAUDE.md",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=4, days_seen=4, recent_count=4.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "上下文压缩",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=4, days_seen=4, recent_count=4.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "AI Coding Agent",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=3, days_seen=3, recent_count=3.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Hermes Agent",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=3, days_seen=3, recent_count=3.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Vibe Coding",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=5, days_seen=4, recent_count=5.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Context Engineering",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=3, days_seen=3, recent_count=3.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "Cursor",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=3, days_seen=3, recent_count=3.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T08:20:35.134979Z",
"action": "add_interest_keyword",
"term": "大模型",
"reason": "Meets the configured interest-keyword review threshold and is not yet covered by interest keywords or stopwords. Evidence: total_count=4, days_seen=3, recent_count=4.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_interest_keyword",
"term": "TypeScript",
"reason": "Core language for AI agent development (e.g., Claude Code, Cursor) and backend engineering, complements existing Python/Java/Go keywords.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_interest_keyword",
"term": "代码审查",
"reason": "Chinese term for 'code review', a key practice in backend engineering and AI agent development workflows.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "Channels",
"reason": "Too generic; could refer to communication channels, YouTube channels, or software channels, not specific to user's focus areas.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "Memory",
"reason": "Extremely broad term; could refer to computer memory, human memory, or memory in various contexts, not discriminative enough.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "Prompt",
"reason": "Already covered by 'Prompt Engineering' as a more specific term; 'Prompt' alone is too broad and matches many unrelated articles.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "AGI",
"reason": "Too broad and speculative; not directly actionable for the user's practical engineering focus areas.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "AI日报",
"reason": "Generic news term; not a technical concept or tool, would add noise to the keyword index.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "AIHOT",
"reason": "Unclear meaning, likely a brand or aggregator, not a specific technical term.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "All In Code",
"reason": "Too vague; could refer to a podcast, a philosophy, or a project, not a specific technical concept.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "auto-twitter-campaign",
"reason": "Too specific to a single project/tool, not a general interest keyword for the user's focus areas.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "ChangeSet",
"reason": "Generic term used in version control and databases; too broad to be a useful filter.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "Lumina",
"reason": "Unclear reference; could be a product, framework, or brand, not clearly aligned with user's focus.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "OpenViking",
"reason": "Unclear reference; not a known tool or concept in the user's stated focus areas.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "Seedance 2.0",
"reason": "Unclear reference; likely a product or version, not a general technical term.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_stopword",
"term": "质量门禁",
"reason": "Chinese term for 'quality gate', too generic in software engineering; not specific to user's focus areas.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Agent Skill",
"reason": "Singular variant",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "Agent Skills",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Binlog",
"reason": "Case variant (auto-ranked)",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "binlog",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Coding Agent",
"reason": "Abbreviated form of 'AI Coding Agent', referring to the same concept.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "AI Coding Agent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Subagent",
"reason": "Case variant",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "SubAgent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Subagents",
"reason": "Plural variant",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "SubAgent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "vibe coding",
"reason": "Case variant",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "Vibe Coding",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Agent架构",
"reason": "Chinese translation of 'Agent architecture', a core concept in AI Agent engineering.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "AI Agent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Agent专业化",
"reason": "Chinese term for 'Agent specialization', directly related to Agent engineering.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "AI Agent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Agent Teams",
"reason": "English equivalent of 'Multi-Agent collaboration', same concept.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "多Agent协作",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "Agentic Engineering",
"reason": "Broader term for engineering with AI agents, closely related to Agent engineering focus.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "AI Agent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "CLI工具",
"reason": "Chinese translation of 'CLI tool', same concept.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "CLI",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "AI编程",
"reason": "Chinese term for 'AI programming', closely related to AI Coding Agent.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "AI Coding Agent",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "记忆管理",
"reason": "Chinese term for 'memory management', closely related to context management in LLM applications.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "上下文管理",
"suggestion_date": "2026-05-14",
"based_on_days": 365
},
{
"applied_at": "2026-05-14T09:12:50.749877Z",
"action": "add_alias",
"term": "会话管理",
"reason": "Chinese term for 'session management', related to context management in LLM applications.",
"suggestions_path": "outputs/term_index/review/term-cleanup-suggestions-2026-05-14.json",
"value": "上下文管理",
"suggestion_date": "2026-05-14",
"based_on_days": 365
}
]
}