feat(knowledge): Executor 行动记忆 + 归一化质量等级解决 ISS-002 重复检索
- RetrievedDocTracker 升级为域级+文档级双层记录(Map<sessionId, Map<domain, Set<filePath>>>) - LookupKnowledgeTool 新增 Min-Max 归一化层(BGE-M3 L2 距离→[0,1] similarity) - 三等级 relevanceLevel:PRECISE / HIGHLY_RELEVANT / REFERENCE + completenessHint 兜底信号 - LookupResult 新增 relevanceLevel、completenessHint、retrievedDomainsThisSession - Executor prompt 重写:4 条检索约束 + 合法出口不查全不追责,重复检索才惩罚 - 入库可观测性:V010 迁移 + retrieval_details JSON 扩展 - 归档 executor-action-memory-relevance change
This commit is contained in:
@@ -55,3 +55,4 @@ uploads/
|
|||||||
/volumes
|
/volumes
|
||||||
/server.pid
|
/server.pid
|
||||||
.claude/settings.local.json
|
.claude/settings.local.json
|
||||||
|
.opencode/plugins/emdash-notifications.js
|
||||||
|
|||||||
@@ -11,3 +11,4 @@
|
|||||||
| 2026-06-26 | session-storage | 会话存储/可观测 | diagnosis_session, agent_step, tool_invocation, token追踪, 多Agent路由 | openspec/changes/session-storage | archived |
|
| 2026-06-26 | session-storage | 会话存储/可观测 | diagnosis_session, agent_step, tool_invocation, token追踪, 多Agent路由 | openspec/changes/session-storage | archived |
|
||||||
| 2026-06-29 | confidence-feedback | 质量评估/反馈机制 | evidence_score, selfEvaluation, feedback, useful, not_useful, case_library, BAD_CASE, tool_invocation规则引擎, 反馈按钮, sessionId回传 | openspec/changes/confidence-feedback | archived |
|
| 2026-06-29 | confidence-feedback | 质量评估/反馈机制 | evidence_score, selfEvaluation, feedback, useful, not_useful, case_library, BAD_CASE, tool_invocation规则引擎, 反馈按钮, sessionId回传 | openspec/changes/confidence-feedback | archived |
|
||||||
| 2026-06-30 | session-dedup-knowledge-map | 去重/知识图谱 | RetrievedDocTracker, KnowledgeDomainService, knowledge_domain, covers, whenToRetrieve, Planner注入, ISS-001 | openspec/changes/session-dedup-knowledge-map | archived |
|
| 2026-06-30 | session-dedup-knowledge-map | 去重/知识图谱 | RetrievedDocTracker, KnowledgeDomainService, knowledge_domain, covers, whenToRetrieve, Planner注入, ISS-001 | openspec/changes/session-dedup-knowledge-map | archived |
|
||||||
|
| 2026-07-01 | executor-action-memory-relevance | 检索质量/行动记忆 | relevanceLevel, completenessHint, Min-Max归一化, RetrievedDocTracker域级记录, Executor检索约束, ISS-002 | openspec/changes/archive/2026-07-01-executor-action-memory-relevance | archived |
|
||||||
|
|||||||
@@ -0,0 +1,70 @@
|
|||||||
|
# Acceptance: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## 分档
|
||||||
|
|
||||||
|
standard
|
||||||
|
|
||||||
|
## 任务完成状态
|
||||||
|
|
||||||
|
| 任务 | 状态 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| T1: RetrievedDocTracker 域级升级 | ✅ 完成 | 双层 Map 结构,域级+文档级记录 |
|
||||||
|
| T2: LookupResult 新增字段 | ✅ 完成 | relevanceLevel / completenessHint / retrievedDomainsThisSession |
|
||||||
|
| T3: 归一化计算逻辑 | ✅ 完成 | Min-Max 归一化 + 三等级判定 |
|
||||||
|
| T4: LookupKnowledgeTool 集成 | ✅ 完成 | 归一化层 + 行动记忆注入 + 域拦截 |
|
||||||
|
| T5: Executor Prompt 重写 | ✅ 完成 | 4 条检索约束,无 knowledge map |
|
||||||
|
| T6: 入库可观测性 | ✅ 完成 | V010 + Entity + JSON 扩展 |
|
||||||
|
| T7: BGE-M3 归一化验证测试 | ✅ 完成 | 范数=1.00000002,测试通过 |
|
||||||
|
|
||||||
|
## 静态验证
|
||||||
|
|
||||||
|
- [x] **语法/编译检查**: 所有 Java 文件编译通过
|
||||||
|
- [x] **Impact Analysis**: LookupKnowledgeTool、RetrievedDocTracker 变更范围经 `gitnexus_impact` 检查,均为 L2 内部接口影响
|
||||||
|
- [x] **Cross-artifact 对齐检查**: brief → proposal → design → specs → tasks 闭环,无 gap
|
||||||
|
- [x] **Prompt 约束检查**: chat-executor-prompt.md 不包含 knowledge map,包含 4 条检索约束
|
||||||
|
|
||||||
|
## 脚本验证
|
||||||
|
|
||||||
|
- [x] **V010 Flyway 迁移**: 迁移成功,`relevance_level` 和 `dedup_reason` 列已添加
|
||||||
|
```sql
|
||||||
|
ALTER TABLE tool_invocation
|
||||||
|
ADD COLUMN relevance_level VARCHAR(20),
|
||||||
|
ADD COLUMN dedup_reason VARCHAR(32);
|
||||||
|
```
|
||||||
|
- [x] **FullPipelineSmokeTest**: BGE-M3 归一化测试通过(范数=1.00000002)
|
||||||
|
- [x] **数据库数据校验**:
|
||||||
|
- `relevance_level` 列已写入 HIGHLY_RELEVANT / REFERENCE
|
||||||
|
- `dedup_reason` 列已写入 doc_retrieved / null
|
||||||
|
- `retrieval_details` JSON 包含 l1_top_similarity、completeness_hint、retrieved_domains、dedup_reason
|
||||||
|
|
||||||
|
## 浏览器/人工验证
|
||||||
|
|
||||||
|
- [x] **应用启动验证**: Spring Boot 应用正常启动,端口 9900
|
||||||
|
- [x] **Chat API 调用验证**: 通过 curl 测试 chat 接口,lookup_knowledge 调用链完整
|
||||||
|
```
|
||||||
|
curl -X POST "http://localhost:9900/api/chat/send" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"sessionId": "b66d799e", "question": "..."}'
|
||||||
|
```
|
||||||
|
- [x] **日志验证**: 应用日志可观察到 relevanceLevel、retrievedDomainsThisSession 输出
|
||||||
|
- [x] **归一化数学验证**: l1_top_score=0.383 → l1_top_similarity=0.8085(`1 - 0.383/2.0 = 0.8085`)✅
|
||||||
|
- [x] **域追踪验证**: `[infrastructure]` → `[infrastructure, api]` 域列表正常扩展
|
||||||
|
|
||||||
|
## 未验证
|
||||||
|
|
||||||
|
| 场景 | 原因 | 风险 | 补验建议 |
|
||||||
|
|------|------|------|---------|
|
||||||
|
| PRECISE 等级(L0 唯一精确匹配) | 测试会话无精确匹配场景 | 低 — L0 matchCount=1 的判断逻辑与 HIGHLY_RELEVANT 共用,实现确定性强 | 构造一条 L0 精确匹配的知识库文档后测试 |
|
||||||
|
| domain_retrieved 域级去重 | 需要同一域全部文档已检索再查该域才触发 | 低 — isDomainRetrieved 逻辑简单,与 isDocRetrieved 等价 | Phase 2 启用域级硬限流时测试 |
|
||||||
|
| DEDUPED 等级 | 当前 code path 去重时仍写 REFERENCE,DEDUPED 未被使用 | 低 — 设计预留,当前未启用 | Phase 2 若启用 DEDUPED 等级时验证 |
|
||||||
|
| Phase 2 域级硬限流 | 非本次范围 | 中 — 当前仅有软约束(prompt),LLM 仍可能在 REFERENCE 下继续检索 | 实测观察,如果 lookup 调用仍偏高,启动 Phase 2 |
|
||||||
|
|
||||||
|
## 剩余风险
|
||||||
|
|
||||||
|
1. **Prompt 软约束局限性**:实测 10 次调用中 9 次为 REFERENCE,说明 LLM 仍倾向于继续检索。如果 prompt 约束效果不足,需启用 Phase 2 域级硬限流。
|
||||||
|
2. **L1 Metadata 解析兼容性**:L1 domain 兜底路径解析 metadata JSON,如果知识库文档 frontmatter 格式不一致可能解析失败,已有 try-catch 兜底。
|
||||||
|
|
||||||
|
## 归档状态
|
||||||
|
|
||||||
|
- [ ] OpenSpec change 尚未归档
|
||||||
|
- [ ] devflow/index.md 状态为 `implemented`,待改为 `archived`
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
# Brief: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
ISS-002:Executor 在单次会话中调用 `lookup_knowledge` 20+ 次,大部分是同域换变体的冗余调用。前序 change `session-dedup-knowledge-map` 解决了文档级重复召回(ISS-001),但未解决 Executor 重复调用问题。
|
||||||
|
|
||||||
|
## 目标
|
||||||
|
|
||||||
|
- Executor 获得行动记忆(知道自己本次会话已检索了哪些域)
|
||||||
|
- 检索结果提供归一化质量等级(PRECISE/HIGHLY_RELEVANT/REFERENCE)+ 兜底信号
|
||||||
|
- Executor prompt 提供明确的检索约束和"放弃检索"的合法出口
|
||||||
|
- 原始分数入库保留可观测性,但不暴露给 LLM
|
||||||
|
|
||||||
|
## 范围
|
||||||
|
|
||||||
|
- `RetrievedDocTracker`:域级 + 文档级双层记录
|
||||||
|
- `LookupKnowledgeTool`:归一化层 + 行动记忆注入
|
||||||
|
- `LookupResult`:新增 relevanceLevel / completenessHint / retrievedDomainsThisSession
|
||||||
|
- `chat-executor-prompt.md`:检索约束重写
|
||||||
|
- `ToolInvocation` + V010:入库可观测性
|
||||||
|
|
||||||
|
## 非目标
|
||||||
|
|
||||||
|
- 不给 Executor 注入 knowledge map(保持 Agent 边界)
|
||||||
|
- 不修改 Planner prompt 或 Planner 逻辑
|
||||||
|
- 不修改 PrimaryResult / SupplementResult 的字段(不暴露原始分数)
|
||||||
|
- Phase 2 域级硬限制暂不实施
|
||||||
|
|
||||||
|
## 分档
|
||||||
|
|
||||||
|
standard
|
||||||
|
|
||||||
|
## 关联 OpenSpec change
|
||||||
|
|
||||||
|
openspec/changes/executor-action-memory-relevance
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# Decisions: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## 过程日志
|
||||||
|
|
||||||
|
### Clarify 阶段
|
||||||
|
|
||||||
|
**入口摘要**:ISS-002 Executor 无约束重复调用 lookup_knowledge(单会话 20+ 次),需要行动记忆 + 归一化质量等级 + prompt 约束来解决。
|
||||||
|
|
||||||
|
**slug**: `executor-action-memory-relevance`
|
||||||
|
|
||||||
|
**规模分档**: `standard`(涉及 7 个文件,跨 DTO/工具层/持久化/Prompt,有设计决策需澄清)
|
||||||
|
|
||||||
|
### Context 阶段
|
||||||
|
|
||||||
|
**devflow/index.md 使用状态**: 已命中。前序 change `session-dedup-knowledge-map`(archived)提供了 RetrievedDocTracker、KnowledgeDomainService、ISS-002 文档。
|
||||||
|
|
||||||
|
**相关 ADR**: 无直接 ADR,但 `session-dedup-knowledge-map` 的 decisions.md 和 evidence.md 记录了文档级去重和 knowledge map 注入的决策。
|
||||||
|
|
||||||
|
**不能违反的历史决策**:
|
||||||
|
1. RetrievedDocTracker 的文档级去重必须保留
|
||||||
|
2. knowledge map 只注入 Planner,不注入 Executor(本次讨论确认)
|
||||||
|
3. L0/L1 原始分数不暴露给 LLM,只在归一化层内部使用(本次讨论确认)
|
||||||
|
|
||||||
|
**需进入 OpenSpec 的上下文点**:
|
||||||
|
1. L1 score 是 L2 距离(值域 [0,+∞)),不是归一化分数——阈值设计需基于实际分布
|
||||||
|
2. L0 的 category 可从 KnowledgeEntry.getCategory() 直接获取;L1 需解析 metadata JSON
|
||||||
|
3. ReactAgent 是自主决策工具调用的 Agent,Prompt 约束是软约束
|
||||||
|
|
||||||
|
### Grill 阶段 — Question Pool
|
||||||
|
|
||||||
|
**维度:术语**
|
||||||
|
1. [evidence-driven] `relevanceLevel` 三个等级(PRECISE/HIGHLY_RELEVANT/REFERENCE)的边界是否清晰,是否存在 LLM 误解的可能? → **已查证**:三个等级语义明确,PRECISE=唯一匹配、HIGHLY_RELEVANT=高分命中、REFERENCE=低置信度参考。LLM 理解风险低。
|
||||||
|
|
||||||
|
**维度:边界**
|
||||||
|
2. [evidence-driven] L1 score 是 L2 距离(值域 [0,+∞)),当前代码无阈值判断。归一化阈值如何设计? → **已查证**:L2 距离典型范围取决于 BGE-M3 1024 维 embedding 的尺度,需从 `tool_invocation.retrieval_details` 中查询实际 `l1_scores` 分布才能定阈值。当前先以常量定义,标记为"需实测校准"。
|
||||||
|
3. [evidence-driven] L1 结果的 category 提取需要解析 metadata JSON 字符串,当前 `SearchResult.metadata` 是 `toString()` 的结果。归一化层是否需要 L1 的 domain? → **已查证**:L1 的 domain 主要用于 RetrievedDocTracker 的域级记录。如果 L0 已命中且包含 category,可直接用 L0 的 category;如果仅 L1 命中,需解析 metadata 提取 category。当前知识库中 L0 大概率先命中,L1 domain 提取作为兜底路径。
|
||||||
|
4. [user-interview] 归一化阈值(L1 score 分界线)在实测数据不足时,是否接受先用保守初始值 + 后续调优的策略? → **用户待确认**
|
||||||
|
|
||||||
|
**维度:验收**
|
||||||
|
5. [evidence-driven] 现有 `tool_invocation` 表 `retrieval_details` JSON 中 `l1_scores` 存的是 L2 距离原始值,新增的 `relevance_level` 和 `completeness_hint` 入库后是否需要回填历史数据? → **已查证**:不需要回填历史数据,新列 nullable 即可,历史记录 relevance_level=null。
|
||||||
|
|
||||||
|
### Grill 结论
|
||||||
|
|
||||||
|
**evidence-driven 汇报**:
|
||||||
|
- E1: relevanceLevel 三等级语义清晰,LLM 误解风险低
|
||||||
|
- E2: L1 score 是 L2 距离,值域不固定,阈值需实测校准
|
||||||
|
- E3: L0 category 直接可用,L1 category 需解析 metadata(兜底路径)
|
||||||
|
- E4: 历史数据不回填,新列 nullable
|
||||||
|
|
||||||
|
**user-interview 已确认**:
|
||||||
|
- Q4: 归一化阈值先用保守初始值 + 后续调优 → **用户已确认**,并建议用 Min-Max 归一化到 [0,1]
|
||||||
|
|
||||||
|
### Specify 阶段补充
|
||||||
|
|
||||||
|
**BGE-M3 L2 归一化实测验证**:
|
||||||
|
- FullPipelineSmokeTest.embeddingBgeM3Works() 新增 L2 范数断言
|
||||||
|
- 结果:范数=1.00000002,误差 < 0.01,测试通过
|
||||||
|
- 结论:BGE-M3 输出为 L2 归一化单位向量,L2 距离数学硬上界 = 2.0
|
||||||
|
- Min-Max 归一化公式:`similarity = 1 - min(l2Score, 2.0) / 2.0`
|
||||||
|
|
||||||
|
**Cross-artifact 对齐检查**:
|
||||||
|
|
||||||
|
| 对齐项 | 状态 |
|
||||||
|
|--------|------|
|
||||||
|
| brief 目标/范围/非目标 → proposal 覆盖 | 已对齐 |
|
||||||
|
| proposal 范围/约束 → design 覆盖 | 已对齐 |
|
||||||
|
| design 归一化/行动记忆/接口影响 → specs 覆盖 | 已对齐 |
|
||||||
|
| specs 可观察行为 → tasks 覆盖 | 已对齐 |
|
||||||
|
|
||||||
|
**接口影响分级**:
|
||||||
|
- RetrievedDocTracker 数据结构升级 → L2(内部接口,消费者只有 LookupKnowledgeTool)
|
||||||
|
- LookupResult 新增 3 字段 → L2(工具返回值,无跨模块调用方)
|
||||||
|
- tool_invocation 新增 2 列 → L2(Flyway nullable,不影响现有查询)
|
||||||
|
- chat-executor-prompt.md 更新 → L1(Prompt 文本变更)
|
||||||
|
|
||||||
|
### Audit 阶段
|
||||||
|
|
||||||
|
**架构风险评估**(5 句以内):
|
||||||
|
1. 归一化层嵌入 LookupKnowledgeTool 内部(静态方法),无跨模块耦合风险。
|
||||||
|
2. RetrievedDocTracker 升级为双层结构,数据量级不变(文档数 × session 数),内存无风险。
|
||||||
|
3. L1 metadata 解析 category 是兜底路径,如果 JSON 格式不一致可能解析失败——已有 try-catch 兜底。
|
||||||
|
4. 归一化阈值 yml 配置化,运行时调优不需要改代码和重启——运维友好。
|
||||||
|
5. Prompt 约束仍依赖 LLM 遵守——如果 Phase 1 效果不足,Phase 2 域级硬限制的 isDomainRetrieved 已就绪,无需额外改造。
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
# Evidence: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## Evidence-driven 结论
|
||||||
|
|
||||||
|
### E1: relevanceLevel 三等级语义清晰度
|
||||||
|
|
||||||
|
- **来源**: Grill 阶段 Question Pool #1
|
||||||
|
- **查证结果**: 三个等级语义明确,边界清晰:
|
||||||
|
- PRECISE:L0 唯一精确匹配,LLM 应直接使用
|
||||||
|
- HIGHLY_RELEVANT:归一化 similarity ≥ 0.75,高度相关
|
||||||
|
- REFERENCE:归一化 similarity ≥ 0.5,相关参考
|
||||||
|
- **结论**: LLM 误解风险低,语义边界足够清晰
|
||||||
|
|
||||||
|
### E2: L1 Score 值域与归一化阈值
|
||||||
|
|
||||||
|
- **来源**: Grill 阶段 Question Pool #2
|
||||||
|
- **查证结果**:
|
||||||
|
- L1 score 是 L2 距离,值域 [0, +∞)
|
||||||
|
- BGE-M3 输出为 L2 归一化单位向量(实测范数=1.00000002),L2 距离数学硬上界 = 2.0
|
||||||
|
- Min-Max 归一化公式:`similarity = 1 - min(l2Score, 2.0) / 2.0`
|
||||||
|
- **结论**: 使用 `maxL2Distance=2.0` 作为归一化上界,阈值 yml 可配置
|
||||||
|
|
||||||
|
### E3: L1 Domain 提取兜底路径
|
||||||
|
|
||||||
|
- **来源**: Grill 阶段 Question Pool #3
|
||||||
|
- **查证结果**:
|
||||||
|
- L0 的 domain 可从 `KnowledgeEntry.getCategory()` 直接获取
|
||||||
|
- L1 结果的 domain 需解析 `SearchResult.metadata` JSON 字符串
|
||||||
|
- 当前知识库设计下 L0 大概率先命中,L1 domain 提取作为兜底
|
||||||
|
- **结论**: 先尝试 L0 category,失败时解析 L1 metadata JSON(try-catch 兜底)
|
||||||
|
|
||||||
|
### E4: 历史数据不回填
|
||||||
|
|
||||||
|
- **来源**: Grill 阶段 Question Pool #5
|
||||||
|
- **查证结果**: 新列 `relevance_level` 和 `dedup_reason` 均为 nullable,不影响现有查询
|
||||||
|
- **结论**: 历史记录保持 null,不需要回填迁移
|
||||||
|
|
||||||
|
### E5: BGE-M3 L2 归一化实测验证
|
||||||
|
|
||||||
|
- **来源**: Specify 阶段 + FullPipelineSmokeTest
|
||||||
|
- **查证结果**:
|
||||||
|
- embeddingBgeM3Works() 测试新增 L2 范数断言
|
||||||
|
- 实测范数 = 1.00000002,误差 < 0.01
|
||||||
|
- 测试通过,BGE-M3 输出确认为 L2 归一化单位向量
|
||||||
|
- **结论**: L2 距离上界 = 2.0 的数学依据成立
|
||||||
|
|
||||||
|
### E6: V010 迁移验证
|
||||||
|
|
||||||
|
- **来源**: Apply 阶段运行时验证
|
||||||
|
- **查证结果**:
|
||||||
|
- Flyway V010 迁移成功执行
|
||||||
|
- `relevance_level` VARCHAR(20) 列可空,已正确写入
|
||||||
|
- `dedup_reason` VARCHAR(32) 列可空,已正确写入
|
||||||
|
- `retrieval_details` JSON 扩展字段(l1_top_similarity、relevance_level、completeness_hint、retrieved_domains、dedup_reason)全部写入
|
||||||
|
- **结论**: 入库可观测性符合设计
|
||||||
|
|
||||||
|
### E7: 数据库数据校验
|
||||||
|
|
||||||
|
- **来源**: Apply 阶段运行时验证
|
||||||
|
- **查证结果**:
|
||||||
|
- session `b66d799e` 共 10 条 lookup_knowledge 调用
|
||||||
|
- id=138: L2=0.383 → similarity=0.8085 → HIGHLY_RELEVANT(符合预期)
|
||||||
|
- id=139-147: 主要为 REFERENCE,doc_retrieved 去重正常触发
|
||||||
|
- retrieved_domains 域追踪:`[infrastructure]` → `[infrastructure, api]` 正常扩展
|
||||||
|
- **结论**: 归一化、行动记忆、去重机制数据层面全部验证通过
|
||||||
@@ -13,6 +13,7 @@
|
|||||||
- [知识库检索使用指南](architecture/knowledge-retrieval-usage.md) - 文档编写和使用说明 ⭐新增
|
- [知识库检索使用指南](architecture/knowledge-retrieval-usage.md) - 文档编写和使用说明 ⭐新增
|
||||||
- [会话管理](architecture/session-management.md) - Redis + MySQL 会话管理
|
- [会话管理](architecture/session-management.md) - Redis + MySQL 会话管理
|
||||||
- [实施规划](architecture/implementation-plan.md) - 分阶段实施计划
|
- [实施规划](architecture/implementation-plan.md) - 分阶段实施计划
|
||||||
|
- [证据评分与用户反馈](architecture/confidence-feedback.md) - evidence_score 规则引擎 + feedback API ⭐新增
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
Archive-ready for executor-action-memory-relevance
|
||||||
|
|
||||||
|
Created: 2026-07-01
|
||||||
|
Tasks complete: 7/7
|
||||||
|
Verification: static + script + manual passed
|
||||||
|
Unverified: PRECISE scenario, domain_retrieved scenario (low risk)
|
||||||
@@ -0,0 +1,189 @@
|
|||||||
|
# Design: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## 架构设计
|
||||||
|
|
||||||
|
### 整体数据流
|
||||||
|
|
||||||
|
```
|
||||||
|
用户问题
|
||||||
|
→ Supervisor → Planner(规划查哪些域)
|
||||||
|
→ Supervisor → Executor(自主调用 lookup_knowledge)
|
||||||
|
↓
|
||||||
|
LookupKnowledgeTool
|
||||||
|
├─ L0 精确匹配 → l0Matches (含 category)
|
||||||
|
├─ L1 语义检索 → l1Results (含 L2 score)
|
||||||
|
├─ 归一化层 → computeRelevanceLevel(l0Count, l1TopScore)
|
||||||
|
│ L2 距离 → similarity = 1 - min(score, 2.0) / 2.0
|
||||||
|
│ L0 唯一匹配 → PRECISE
|
||||||
|
│ L0 命中 + L1 similarity ≥ 0.75 → HIGHLY_RELEVANT
|
||||||
|
│ 仅 L1 similarity ≥ 0.75 → HIGHLY_RELEVANT
|
||||||
|
│ L0 多匹配 + L1 similarity [0.5, 0.75) → REFERENCE
|
||||||
|
│ 仅 L1 similarity [0.5, 0.75) → REFERENCE
|
||||||
|
├─ 域级行动记忆 → RetrievedDocTracker.markRetrieved(sessionId, domain, filePath)
|
||||||
|
│ getRetrievedDomains(sessionId) → retrievedDomainsThisSession
|
||||||
|
├─ 文档级去重 → 保留现有逻辑
|
||||||
|
└─ 组装 LookupResult(含 relevanceLevel, completenessHint, retrievedDomainsThisSession)
|
||||||
|
↓
|
||||||
|
LLM 看到:
|
||||||
|
relevanceLevel: PRECISE
|
||||||
|
completenessHint: "知识库中不存在比上述结果更精准的文档"
|
||||||
|
retrievedDomainsThisSession: ["infrastructure", "api"]
|
||||||
|
```
|
||||||
|
|
||||||
|
### Agent 边界(保持清晰)
|
||||||
|
|
||||||
|
| Agent | 知道什么 | 不知道什么 |
|
||||||
|
|-------|---------|-----------|
|
||||||
|
| Planner | 全域知识边界(knowledge map) | 执行细节、检索结果 |
|
||||||
|
| Executor | 自己的行动记忆(已检索域列表) | 全域知识边界(不注入 knowledge map) |
|
||||||
|
|
||||||
|
行动记忆通过**工具返回值**传递,不通过 prompt 注入。
|
||||||
|
|
||||||
|
### 数据结构设计
|
||||||
|
|
||||||
|
#### 1. RetrievedDocTracker 升级
|
||||||
|
|
||||||
|
```java
|
||||||
|
// 现有:sessionId → Set<filePath>(文档级)
|
||||||
|
ConcurrentHashMap<String, Set<String>> retrieved
|
||||||
|
|
||||||
|
// 新增:sessionId → { domain → Set<filePath> }(域级 + 文档级)
|
||||||
|
ConcurrentHashMap<String, Map<String, Set<String>>> sessionRetrievals
|
||||||
|
```
|
||||||
|
|
||||||
|
方法列表:
|
||||||
|
- `markRetrieved(sessionId, domain, filePath)` — 一次记录两层
|
||||||
|
- `isDocRetrieved(sessionId, filePath)` → boolean — 文档级去重(替代现有 isAlreadyRetrieved)
|
||||||
|
- `isDomainRetrieved(sessionId, domain)` → boolean — 域级检查(Phase 2 硬限制用)
|
||||||
|
- `getRetrievedDomains(sessionId)` → List<String> — 行动记忆(返回给 LLM)
|
||||||
|
- `clearSession(sessionId)` — 清理(不变)
|
||||||
|
|
||||||
|
#### 2. LookupResult 扩展
|
||||||
|
|
||||||
|
```java
|
||||||
|
@Data @Builder
|
||||||
|
public class LookupResult {
|
||||||
|
boolean found;
|
||||||
|
PrimaryResult primary; // 不变,不暴露原始分数
|
||||||
|
SupplementResult supplement; // 不变,不暴露原始分数
|
||||||
|
// ---- 新增 ----
|
||||||
|
String relevanceLevel; // PRECISE / HIGHLY_RELEVANT / REFERENCE
|
||||||
|
String completenessHint; // 兜底信号
|
||||||
|
List<String> retrievedDomainsThisSession; // 行动记忆
|
||||||
|
String message; // 不变
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**PrimaryResult 和 SupplementResult 不加任何分数字段**。原始分数在归一化层内部消化。
|
||||||
|
|
||||||
|
#### 3. 归一化计算
|
||||||
|
|
||||||
|
`RelevanceNormalizer`(LookupKnowledgeTool 内部静态方法):
|
||||||
|
|
||||||
|
```
|
||||||
|
输入:l0MatchCount, l1TopScore (L2 距离)
|
||||||
|
输出:RelevanceAssessment { relevanceLevel, completenessHint }
|
||||||
|
|
||||||
|
归一化公式(BGE-M3 输出 L2 归一化单位向量,已实测验证):
|
||||||
|
similarity = 1 - min(l2Score, maxL2Distance) / maxL2Distance
|
||||||
|
maxL2Distance 默认 2.0,yml 可覆盖
|
||||||
|
|
||||||
|
判定逻辑:
|
||||||
|
if l0MatchCount == 1 → PRECISE
|
||||||
|
if l0MatchCount > 1 && l1Similarity >= highlyRelevantThreshold → HIGHLY_RELEVANT
|
||||||
|
if l0MatchCount == 0 && l1Similarity >= highlyRelevantThreshold → HIGHLY_RELEVANT
|
||||||
|
if l0MatchCount > 1 && l1Similarity >= referenceThreshold → REFERENCE
|
||||||
|
if l0MatchCount == 0 && l1Similarity >= referenceThreshold → REFERENCE
|
||||||
|
else → 无结果
|
||||||
|
|
||||||
|
completenessHint 映射:
|
||||||
|
PRECISE → "知识库中不存在比上述结果更精准的文档"
|
||||||
|
HIGHLY_RELEVANT → "当前结果已高度相关,继续检索不太可能找到更精准的文档"
|
||||||
|
REFERENCE → "当前结果为相关参考,如需更精准信息请明确缺少的具体维度"
|
||||||
|
```
|
||||||
|
|
||||||
|
配置项(application.yml):
|
||||||
|
```yaml
|
||||||
|
retrieval:
|
||||||
|
normalization:
|
||||||
|
max-l2-distance: 2.0 # L2 距离上界(单位向量 = 2.0)
|
||||||
|
highly-relevant-threshold: 0.75 # similarity ≥ 0.75 → HIGHLY_RELEVANT
|
||||||
|
reference-threshold: 0.5 # similarity ≥ 0.5 → REFERENCE
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 4. 入库记录扩展
|
||||||
|
|
||||||
|
`tool_invocation` 表新增列:
|
||||||
|
|
||||||
|
| 列名 | 类型 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| `relevance_level` | VARCHAR(20) | PRECISE / HIGHLY_RELEVANT / REFERENCE / DEDUPED |
|
||||||
|
| `dedup_reason` | VARCHAR(32) | doc_retrieved / domain_retrieved / null |
|
||||||
|
|
||||||
|
`retrieval_details` JSON 扩展:
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"l0_match_count": 2,
|
||||||
|
"l0_titles": ["MySQL连接池配置", "HikariCP参数调优"],
|
||||||
|
"l1_top_score": 0.52,
|
||||||
|
"l1_top_similarity": 0.74,
|
||||||
|
"l1_match_count": 3,
|
||||||
|
"l1_scores": [0.52, 0.68, 0.91],
|
||||||
|
"relevance_level": "HIGHLY_RELEVANT",
|
||||||
|
"completeness_hint": "当前结果已高度相关...",
|
||||||
|
"retrieved_domains": ["infrastructure"],
|
||||||
|
"dedup_reason": null
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
原始 L2 score 和归一化后的 similarity 都入库,保留可观测性。
|
||||||
|
|
||||||
|
### Executor Prompt 设计
|
||||||
|
|
||||||
|
不加 knowledge map,只加基于行动记忆的行为规则:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
## 检索约束
|
||||||
|
|
||||||
|
### 1. 判断重复:基于已检索上下文
|
||||||
|
每次 lookup_knowledge 返回值中包含 retrievedDomainsThisSession,
|
||||||
|
表示本次会话已检索过的知识域。如果当前问题与已检索域语义重叠,
|
||||||
|
**禁止再次调用 lookup_knowledge**。
|
||||||
|
|
||||||
|
### 2. 重复了该怎么办
|
||||||
|
如果当前想检索的内容与【已检索上下文】语义相似:
|
||||||
|
- 禁止换关键词重新检索
|
||||||
|
- 直接基于已有事实回答
|
||||||
|
- 如果信息不足,先明确指出缺少什么具体维度
|
||||||
|
(如:"缺少 HikariCP 具体配置参数"、"缺少连接池耗尽的日志样例"),
|
||||||
|
再针对该维度进行一次定向补充检索——而非盲目换词重查
|
||||||
|
|
||||||
|
### 3. 合法出口:允许信息不全时给出结论
|
||||||
|
如果你认为已有信息足以回答核心问题,即使细节不全,
|
||||||
|
也请直接给出结论并说明局限性(如:"基于已有信息,连接池配置建议如下,
|
||||||
|
但具体参数值需结合实际负载调整")。
|
||||||
|
**不查全不会被追责,重复检索才会被惩罚。**
|
||||||
|
|
||||||
|
### 4. 利用质量信号判断
|
||||||
|
- relevanceLevel=PRECISE → 信息精准,直接使用,不再检索
|
||||||
|
- relevanceLevel=HIGHLY_RELEVANT + 域已在 retrievedDomainsThisSession → 禁止再次调用
|
||||||
|
- relevanceLevel=REFERENCE → 先指出缺什么维度,再定向补充一次
|
||||||
|
- completenessHint 是知识库给你的天花板信号,信任它
|
||||||
|
```
|
||||||
|
|
||||||
|
### 关键决策
|
||||||
|
|
||||||
|
1. **L0/L1 原始分数不暴露给 LLM** — 在归一化层内部消化,避免 LLM 混淆尺度
|
||||||
|
2. **BGE-M3 L2 归一化已实测验证** — 范数 1.00000002,maxL2Distance=2.0 是数学硬上界
|
||||||
|
3. **行动记忆通过工具返回值传递** — 不通过 prompt 注入,不修改 ReactAgent prompt 构建方式
|
||||||
|
4. **不给 Executor knowledge map** — 保持 Agent 边界:Planner 知道全域,Executor 只知道自己做了什么
|
||||||
|
5. **Phase 2 域级硬限制暂不实施** — 先观察 prompt 约束 + 归一化信号的效果
|
||||||
|
|
||||||
|
### 接口影响分级
|
||||||
|
|
||||||
|
| 变更 | 级别 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| RetrievedDocTracker 数据结构升级 | L2 内部接口 | 消费者只有 LookupKnowledgeTool,在同一实现范围内 |
|
||||||
|
| LookupResult 新增 3 个字段 | L2 内部接口 | 消费者是 LLM(工具返回值),无跨模块调用方 |
|
||||||
|
| tool_invocation 表新增 2 列 | L2 内部接口 | Flyway 迁移,nullable,不影响现有查询 |
|
||||||
|
| chat-executor-prompt.md 更新 | L1 内部实现 | Prompt 文本变更,不改变接口 |
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
# Proposal: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## 问题
|
||||||
|
|
||||||
|
ISS-002:Executor 在单次会话中调用 `lookup_knowledge` 20+ 次,大部分是同域换变体的冗余调用。
|
||||||
|
|
||||||
|
根因:
|
||||||
|
1. **行动记忆缺失**:Executor 不知道自己已经检索过哪些域,反复用不同关键词查同一个域
|
||||||
|
2. **质量信号缺失**:检索结果没有归一化质量等级,LLM 无法判断"结果够不够"
|
||||||
|
3. **Prompt 约束缺失**:现有 executor prompt 要求"所有需要外部信息的地方都必须调用工具",没有"放弃检索"的合法出口
|
||||||
|
|
||||||
|
## 建议方案
|
||||||
|
|
||||||
|
### 1. 行动记忆(通过工具返回值传递)
|
||||||
|
|
||||||
|
`RetrievedDocTracker` 数据结构升级:`Map<sessionId, Map<domain, Set<filePath>>>`。
|
||||||
|
|
||||||
|
每次 `lookup_knowledge` 返回值附带 `retrievedDomainsThisSession`,让 Executor 知道自己本次会话已检索过哪些域。
|
||||||
|
|
||||||
|
**不给 Executor knowledge map**——保持 Agent 边界清晰:Planner 知道全域(规划查哪个域),Executor 只知道自己做了什么(执行检索 + 基于结果推理)。
|
||||||
|
|
||||||
|
### 2. 归一化质量等级(封装 L0/L1 分数差异)
|
||||||
|
|
||||||
|
在 `LookupKnowledgeTool` 内部新增归一化层,将 L0 匹配数和 L1 score 统一为三个等级:
|
||||||
|
|
||||||
|
| 等级 | 含义 | LLM 应做什么 |
|
||||||
|
|------|------|-------------|
|
||||||
|
| `PRECISE` | 精准命中 | 直接使用,不再检索 |
|
||||||
|
| `HIGHLY_RELEVANT` | 高度相关 | 综合推理,大概率不需要继续查 |
|
||||||
|
| `REFERENCE` | 相关参考 | 可参考,如需更精准请明确缺什么维度 |
|
||||||
|
|
||||||
|
归一化逻辑:
|
||||||
|
- L0 唯一匹配 → PRECISE
|
||||||
|
- L0 命中 + L1 高分 → HIGHLY_RELEVANT
|
||||||
|
- L0 多匹配 + L1 中分 → HIGHLY_RELEVANT
|
||||||
|
- L0 多匹配 + 无 L1 → REFERENCE
|
||||||
|
- 仅 L1 命中 → 按 score 分 HIGHLY_RELEVANT / REFERENCE
|
||||||
|
|
||||||
|
**L0/L1 原始分数不返回给 LLM**,只在归一化层内部使用。原始分数入库(`tool_invocation.retrieval_details`)保留可观测性。
|
||||||
|
|
||||||
|
### 3. 兜底信号(completenessHint)
|
||||||
|
|
||||||
|
每次返回附带 `completenessHint`,给 LLM "天花板"信号:
|
||||||
|
|
||||||
|
| relevanceLevel | completenessHint |
|
||||||
|
|----------------|-----------------|
|
||||||
|
| PRECISE | "知识库中不存在比上述结果更精准的文档" |
|
||||||
|
| HIGHLY_RELEVANT | "当前结果已高度相关,继续检索不太可能找到更精准的文档" |
|
||||||
|
| REFERENCE | "当前结果为相关参考,如需更精准信息请明确缺少的具体维度" |
|
||||||
|
|
||||||
|
### 4. Executor prompt 重写检索约束
|
||||||
|
|
||||||
|
- 基于 `retrievedDomainsThisSession` 判断重复(不是"不要重复",而是"重复了该怎么办")
|
||||||
|
- 给 LLM 合法出口:"不查全不会被追责,重复检索才会被惩罚"
|
||||||
|
- 利用 `relevanceLevel` + `completenessHint` 判断质量
|
||||||
|
|
||||||
|
### 5. 入库可观测性
|
||||||
|
|
||||||
|
`tool_invocation` 表新增 `relevance_level` 和 `dedup_reason` 列。
|
||||||
|
`retrieval_details` JSON 扩展:加入归一化等级、兜底信号、已检索域、去重原因、L1 top score。
|
||||||
|
|
||||||
|
## 范围
|
||||||
|
|
||||||
|
- `LookupKnowledgeTool`:归一化层 + 行动记忆注入 + 域级拦截
|
||||||
|
- `RetrievedDocTracker`:数据结构升级(域级记录)
|
||||||
|
- `LookupResult`:新增 `relevanceLevel`、`completenessHint`、`retrievedDomainsThisSession`
|
||||||
|
- `chat-executor-prompt.md`:检索约束重写
|
||||||
|
- `ToolInvocation` 实体 + V010 迁移:新增列
|
||||||
|
- `LookupKnowledgeTool.saveToolInvocation()`:扩展入库字段
|
||||||
|
|
||||||
|
## 非目标
|
||||||
|
|
||||||
|
- 不给 Executor 注入 knowledge map(保持 Agent 边界)
|
||||||
|
- 不修改 Planner prompt 或 Planner 逻辑
|
||||||
|
- 不修改 `PrimaryResult`/`SupplementResult` 的字段(不暴露原始分数给 LLM)
|
||||||
|
- Phase 2 域级硬限制暂不实施,先观察 prompt 约束效果
|
||||||
|
|
||||||
|
## 风险
|
||||||
|
|
||||||
|
1. L1 score 阈值(0.3/0.7)需要根据实际 embedding 分布调优,当前为初始值
|
||||||
|
2. 归一化等级可能让 LLM 过早停止检索——需实测观察 REFERENCE 场景下的行为
|
||||||
|
3. Prompt 约束仍依赖 LLM 遵守——如果效果不足,需启用 Phase 2 域级硬限制
|
||||||
|
|
||||||
|
## 来自 devflow 的上下文约束
|
||||||
|
|
||||||
|
- 前序 change `session-dedup-knowledge-map`:已实现文档级去重(RetrievedDocTracker + filePath)和 Planner knowledge map 注入
|
||||||
|
- ISS-001:文档级重复召回已修复
|
||||||
|
- glossary:ReactAgent 是自主决策工具调用的 Agent,不受外部流程控制
|
||||||
|
- JPA ddl-auto 使用 validate 模式,表结构修改必须通过 Flyway 迁移
|
||||||
+110
@@ -0,0 +1,110 @@
|
|||||||
|
# Functional Spec: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## FS-1: L2 距离归一化
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
LookupKnowledgeTool 内部将 L1 的 L2 距离归一化为 [0,1] 区间的 similarity 值,基于 BGE-M3 输出为 L2 归一化单位向量(已实测验证,范数=1.00000002)。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- 归一化公式:`similarity = 1 - min(l2Score, maxL2Distance) / maxL2Distance`
|
||||||
|
- `maxL2Distance` 默认 2.0,可通过 `retrieval.normalization.max-l2-distance` 覆盖
|
||||||
|
- 归一化阈值可通过 `retrieval.normalization.highly-relevant-threshold` 和 `retrieval.normalization.reference-threshold` 配置
|
||||||
|
- 归一化计算在 LookupKnowledgeTool 内部完成,不暴露原始分数给 LLM
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] L2 score=0 → similarity=1.0
|
||||||
|
- [ ] L2 score=1.0 → similarity=0.5
|
||||||
|
- [ ] L2 score=2.0 → similarity=0.0
|
||||||
|
- [ ] L2 score=3.0(超出上界)→ similarity=0.0(min 函数截断)
|
||||||
|
- [ ] 配置项可通过 yml 覆盖默认值
|
||||||
|
|
||||||
|
## FS-2: 归一化质量等级判定
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
基于 L0 匹配数和归一化后的 L1 similarity,输出三等级 relevanceLevel + completenessHint。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- L0 唯一匹配 → PRECISE + "知识库中不存在比上述结果更精准的文档"
|
||||||
|
- L0 命中 + L1 similarity ≥ 0.75 → HIGHLY_RELEVANT + "当前结果已高度相关,继续检索不太可能找到更精准的文档"
|
||||||
|
- 仅 L1 similarity ≥ 0.75 → HIGHLY_RELEVANT + 对应 hint
|
||||||
|
- L0 多匹配 + L1 similarity [0.5, 0.75) → REFERENCE + "当前结果为相关参考,如需更精准信息请明确缺少的具体维度"
|
||||||
|
- 仅 L1 similarity [0.5, 0.75) → REFERENCE + 对应 hint
|
||||||
|
- L1 similarity < 0.5 → 不视为有效结果
|
||||||
|
- 无 L0 且无 L1 → found=false
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] L0 matchCount=1 → relevanceLevel=PRECISE
|
||||||
|
- [ ] L0 matchCount=2, L1 similarity=0.8 → relevanceLevel=HIGHLY_RELEVANT
|
||||||
|
- [ ] L0 matchCount=0, L1 similarity=0.8 → relevanceLevel=HIGHLY_RELEVANT
|
||||||
|
- [ ] L0 matchCount=3, L1 similarity=0.6 → relevanceLevel=REFERENCE
|
||||||
|
- [ ] L0 matchCount=0, L1 similarity=0.4 → found=false 或 supplement 被过滤
|
||||||
|
- [ ] 每个 relevanceLevel 对应正确的 completenessHint
|
||||||
|
|
||||||
|
## FS-3: 域级行动记忆
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
RetrievedDocTracker 升级为域级 + 文档级双层记录,支持查询当前会话已检索的域列表。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- `markRetrieved(sessionId, domain, filePath)` 一次记录两层
|
||||||
|
- `isDocRetrieved(sessionId, filePath)` 返回文档级去重结果
|
||||||
|
- `isDomainRetrieved(sessionId, domain)` 返回域级检查结果
|
||||||
|
- `getRetrievedDomains(sessionId)` 返回已检索域列表
|
||||||
|
- `clearSession(sessionId)` 清理所有记录
|
||||||
|
- 现有 `isAlreadyRetrieved(sessionId, filePath)` 语义不变(内部委托给 isDocRetrieved)
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] markRetrieved("s1", "infrastructure", "a.md") 后,isDocRetrieved("s1", "a.md")=true
|
||||||
|
- [ ] markRetrieved("s1", "infrastructure", "a.md") 后,isDomainRetrieved("s1", "infrastructure")=true
|
||||||
|
- [ ] markRetrieved("s1", "infrastructure", "a.md") 后,getRetrievedDomains("s1")=["infrastructure"]
|
||||||
|
- [ ] markRetrieved("s1", "api", "b.md") 后,getRetrievedDomains("s1")=["infrastructure","api"]
|
||||||
|
- [ ] clearSession("s1") 后,所有方法返回空/false
|
||||||
|
- [ ] 线程安全:ConcurrentHashMap + ConcurrentHashMap 内层
|
||||||
|
|
||||||
|
## FS-4: LookupResult 返回值扩展
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
LookupResult 新增 relevanceLevel、completenessHint、retrievedDomainsThisSession 三个字段,让 LLM 获得行动记忆和质量信号。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- 每次 lookup_knowledge 返回值包含这三个新字段
|
||||||
|
- PrimaryResult 和 SupplementResult 不变,不暴露原始分数
|
||||||
|
- 去重拦截时,返回值仍包含 retrievedDomainsThisSession(让 LLM 知道已检索了哪些域)
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] 正常检索返回时,LookupResult 包含 relevanceLevel + completenessHint + retrievedDomainsThisSession
|
||||||
|
- [ ] 文档级去重拦截时,LookupResult.message 包含去重提示,retrievedDomainsThisSession 不为 null
|
||||||
|
- [ ] PrimaryResult 和 SupplementResult 无新增分数字段
|
||||||
|
|
||||||
|
## FS-5: Executor Prompt 检索约束
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
重写 chat-executor-prompt.md 的检索规则,从"必须调用工具"改为"基于行动记忆和质量信号判断是否需要检索"。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- Prompt 不包含 knowledge map
|
||||||
|
- Prompt 包含 4 条检索约束(判断重复、重复了该怎么办、合法出口、利用质量信号)
|
||||||
|
- 原有规则"所有需要外部信息的地方,都必须调用对应的工具"被替换
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] Executor prompt 不包含 knowledge map 内容
|
||||||
|
- [ ] Executor prompt 包含"禁止换关键词重新检索"约束
|
||||||
|
- [ ] Executor prompt 包含"不查全不会被追责"合法出口
|
||||||
|
- [ ] Executor prompt 包含 relevanceLevel 行为指导
|
||||||
|
|
||||||
|
## FS-6: 入库可观测性
|
||||||
|
|
||||||
|
### 需求
|
||||||
|
tool_invocation 表新增 relevance_level 和 dedup_reason 列,retrieval_details JSON 扩展。
|
||||||
|
|
||||||
|
### 可观察行为
|
||||||
|
- 每次 lookup_knowledge 调用后,tool_invocation 记录包含 relevance_level 和 dedup_reason
|
||||||
|
- retrieval_details JSON 包含 l1_top_similarity(归一化后值)、relevance_level、completeness_hint、retrieved_domains、dedup_reason
|
||||||
|
- 历史数据新列为 null,不影响现有查询
|
||||||
|
|
||||||
|
### 验收标准
|
||||||
|
- [ ] V010 迁移脚本成功执行
|
||||||
|
- [ ] 新增 relevance_level 列 VARCHAR(20) nullable
|
||||||
|
- [ ] 新增 dedup_reason 列 VARCHAR(32) nullable
|
||||||
|
- [ ] saveToolInvocation() 写入新字段
|
||||||
|
- [ ] SQL 可查询归一化等级分布:`SELECT relevance_level, COUNT(*) FROM tool_invocation WHERE tool_name='lookup_knowledge' GROUP BY relevance_level`
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
# Tasks: executor-action-memory-relevance
|
||||||
|
|
||||||
|
## T1: RetrievedDocTracker 域级升级
|
||||||
|
|
||||||
|
**文件**: `src/main/java/com/superbiz/agent/tool/RetrievedDocTracker.java`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- 数据结构从 `ConcurrentHashMap<sessionId, Set<filePath>>` 升级为 `ConcurrentHashMap<sessionId, Map<domain, Set<filePath>>>`
|
||||||
|
- 新增 `markRetrieved(sessionId, domain, filePath)`
|
||||||
|
- 新增 `isDocRetrieved(sessionId, filePath)` — 从内层 Map 的 values 中查找 filePath
|
||||||
|
- 新增 `isDomainRetrieved(sessionId, domain)` — 检查 domain key 存在
|
||||||
|
- 新增 `getRetrievedDomains(sessionId)` → `List<String>`
|
||||||
|
- `isAlreadyRetrieved(sessionId, filePath)` 保留(委托给 isDocRetrieved,向后兼容)
|
||||||
|
- `clearSession(sessionId)` 清理外层 key
|
||||||
|
|
||||||
|
**验收**: FS-3 所有验收标准通过
|
||||||
|
|
||||||
|
## T2: LookupResult 新增字段
|
||||||
|
|
||||||
|
**文件**: `src/main/java/com/superbiz/agent/dto/LookupResult.java`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- 新增 `String relevanceLevel`
|
||||||
|
- 新增 `String completenessHint`
|
||||||
|
- 新增 `List<String> retrievedDomainsThisSession`
|
||||||
|
|
||||||
|
**验收**: 编译通过,字段存在且类型正确
|
||||||
|
|
||||||
|
## T3: 归一化计算逻辑
|
||||||
|
|
||||||
|
**文件**: `src/main/java/com/superbiz/agent/tool/LookupKnowledgeTool.java`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- 新增配置类或字段读取 `retrieval.normalization.max-l2-distance`(默认 2.0)、`highly-relevant-threshold`(默认 0.75)、`reference-threshold`(默认 0.5)
|
||||||
|
- 新增私有方法 `computeRelevance(int l0MatchCount, float l1TopScore)` → 返回包含 `relevanceLevel` + `completenessHint` 的 record/内部类
|
||||||
|
- L2 距离归一化:`similarity = 1 - min(l1TopScore, maxL2Distance) / maxL2Distance`
|
||||||
|
- 判定逻辑按 design.md 中的优先级实现
|
||||||
|
|
||||||
|
**验收**: FS-1 + FS-2 所有验收标准通过
|
||||||
|
|
||||||
|
## T4: LookupKnowledgeTool 集成归一化 + 行动记忆
|
||||||
|
|
||||||
|
**文件**: `src/main/java/com/superbiz/agent/tool/LookupKnowledgeTool.java`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- `lookupKnowledge()` 方法中,在 Step 4(组装结果)后、Step 5(去重过滤)前,调用 `computeRelevance()` 计算 relevanceLevel 和 completenessHint
|
||||||
|
- 从 l0Matches 提取 domain(`l0Matches.get(0).getCategory()`),L1 结果尝试从 metadata JSON 解析 category(兜底)
|
||||||
|
- markRetrieved 调用从 `markRetrieved(sessionId, docKey)` 改为 `markRetrieved(sessionId, domain, docKey)`
|
||||||
|
- 去重拦截时(文档级),LookupResult 也附带 retrievedDomainsThisSession
|
||||||
|
- LookupResult.builder() 中设置三个新字段
|
||||||
|
|
||||||
|
**验收**: FS-4 所有验收标准通过;日志中可看到 relevanceLevel 和 completenessHint 输出
|
||||||
|
|
||||||
|
## T5: Executor Prompt 重写
|
||||||
|
|
||||||
|
**文件**: `src/main/resources/prompts/chat-executor-prompt.md`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- 将"所有需要外部信息的地方,都必须调用对应的工具"替换为"需要外部信息时调用工具,但须遵守下方的检索约束"
|
||||||
|
- 新增"## 检索约束"区块,包含 4 条规则(判断重复、重复了该怎么办、合法出口、利用质量信号)
|
||||||
|
- 不注入 knowledge map
|
||||||
|
|
||||||
|
**验收**: FS-5 所有验收标准通过
|
||||||
|
|
||||||
|
## T6: 入库可观测性
|
||||||
|
|
||||||
|
**文件**:
|
||||||
|
- `src/main/resources/db/migration/V010__add_relevance_level_to_tool_invocation.sql`
|
||||||
|
- `src/main/java/com/superbiz/agent/domain/entity/ToolInvocation.java`
|
||||||
|
- `src/main/java/com/superbiz/agent/tool/LookupKnowledgeTool.java`(saveToolInvocation 方法)
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- V010: ALTER TABLE tool_invocation ADD relevance_level VARCHAR(20), ADD dedup_reason VARCHAR(32)
|
||||||
|
- ToolInvocation 实体新增 `relevanceLevel` 和 `dedupReason` 字段
|
||||||
|
- saveToolInvocation() 中:
|
||||||
|
- 设置 `inv.setRelevanceLevel(...)` 和 `inv.setDedupReason(...)`
|
||||||
|
- retrieval_details JSON 扩展:新增 l1_top_similarity、relevance_level、completeness_hint、retrieved_domains、dedup_reason 字段
|
||||||
|
- 去重拦截时,dedupReason 设为 "doc_retrieved";域级拦截时设为 "domain_retrieved"
|
||||||
|
|
||||||
|
**验收**: FS-6 所有验收标准通过
|
||||||
|
|
||||||
|
## T7: BGE-M3 归一化验证测试
|
||||||
|
|
||||||
|
**文件**: `src/test/java/com/superbiz/agent/service/FullPipelineSmokeTest.java`
|
||||||
|
|
||||||
|
**改动**:
|
||||||
|
- 已完成:embeddingBgeM3Works() 中新增 L2 范数断言(范数=1.00000002,测试已通过)
|
||||||
|
|
||||||
|
**验收**: 测试通过,范数断言 |norm - 1.0| < 0.01
|
||||||
@@ -61,6 +61,12 @@ public class ToolInvocation {
|
|||||||
@Column(name = "is_truncated")
|
@Column(name = "is_truncated")
|
||||||
private Boolean isTruncated;
|
private Boolean isTruncated;
|
||||||
|
|
||||||
|
@Column(name = "relevance_level", length = 20)
|
||||||
|
private String relevanceLevel;
|
||||||
|
|
||||||
|
@Column(name = "dedup_reason", length = 32)
|
||||||
|
private String dedupReason;
|
||||||
|
|
||||||
@JdbcTypeCode(SqlTypes.JSON)
|
@JdbcTypeCode(SqlTypes.JSON)
|
||||||
@Column(name = "retrieval_details", columnDefinition = "JSON")
|
@Column(name = "retrieval_details", columnDefinition = "JSON")
|
||||||
private String retrievalDetails;
|
private String retrievalDetails;
|
||||||
|
|||||||
@@ -27,6 +27,21 @@ public class LookupResult {
|
|||||||
*/
|
*/
|
||||||
private SupplementResult supplement;
|
private SupplementResult supplement;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 归一化质量等级:PRECISE / HIGHLY_RELEVANT / REFERENCE
|
||||||
|
*/
|
||||||
|
private String relevanceLevel;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 兜底信号:告诉 LLM 知识库的"天花板"
|
||||||
|
*/
|
||||||
|
private String completenessHint;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 本次会话已检索过的域列表(行动记忆)
|
||||||
|
*/
|
||||||
|
private List<String> retrievedDomainsThisSession;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 系统消息(如去重提示)
|
* 系统消息(如去重提示)
|
||||||
*/
|
*/
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
package com.superbiz.agent.tool;
|
package com.superbiz.agent.tool;
|
||||||
|
|
||||||
|
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||||
import com.superbiz.agent.domain.entity.ToolInvocation;
|
import com.superbiz.agent.domain.entity.ToolInvocation;
|
||||||
import com.superbiz.agent.dto.*;
|
import com.superbiz.agent.dto.*;
|
||||||
import com.superbiz.agent.repository.ToolInvocationRepository;
|
import com.superbiz.agent.repository.ToolInvocationRepository;
|
||||||
@@ -9,6 +10,7 @@ import com.superbiz.agent.util.SessionContextHolder;
|
|||||||
import lombok.extern.slf4j.Slf4j;
|
import lombok.extern.slf4j.Slf4j;
|
||||||
import org.springframework.ai.tool.annotation.Tool;
|
import org.springframework.ai.tool.annotation.Tool;
|
||||||
import org.springframework.beans.factory.annotation.Autowired;
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
import org.springframework.beans.factory.annotation.Value;
|
||||||
import org.springframework.stereotype.Component;
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
@@ -17,11 +19,29 @@ import java.util.stream.Collectors;
|
|||||||
/**
|
/**
|
||||||
* 知识库查询工具
|
* 知识库查询工具
|
||||||
* 提供给 Agent 的混合检索工具(L0 + L1)
|
* 提供给 Agent 的混合检索工具(L0 + L1)
|
||||||
|
* 内置归一化层:将 L0 匹配数 + L1 L2 距离归一化为统一质量等级
|
||||||
*/
|
*/
|
||||||
@Slf4j
|
@Slf4j
|
||||||
@Component
|
@Component
|
||||||
public class LookupKnowledgeTool {
|
public class LookupKnowledgeTool {
|
||||||
|
|
||||||
|
private static final String LEVEL_PRECISE = "PRECISE";
|
||||||
|
private static final String LEVEL_HIGHLY_RELEVANT = "HIGHLY_RELEVANT";
|
||||||
|
private static final String LEVEL_REFERENCE = "REFERENCE";
|
||||||
|
|
||||||
|
private static final String HINT_PRECISE = "知识库中不存在比上述结果更精准的文档";
|
||||||
|
private static final String HINT_HIGHLY_RELEVANT = "当前结果已高度相关,继续检索不太可能找到更精准的文档";
|
||||||
|
private static final String HINT_REFERENCE = "当前结果为相关参考,如需更精准信息请明确缺少的具体维度";
|
||||||
|
|
||||||
|
@Value("${retrieval.normalization.max-l2-distance:2.0}")
|
||||||
|
private double maxL2Distance;
|
||||||
|
|
||||||
|
@Value("${retrieval.normalization.highly-relevant-threshold:0.75}")
|
||||||
|
private double highlyRelevantThreshold;
|
||||||
|
|
||||||
|
@Value("${retrieval.normalization.reference-threshold:0.5}")
|
||||||
|
private double referenceThreshold;
|
||||||
|
|
||||||
@Autowired
|
@Autowired
|
||||||
private KnowledgeIndexService knowledgeIndexService;
|
private KnowledgeIndexService knowledgeIndexService;
|
||||||
|
|
||||||
@@ -34,6 +54,9 @@ public class LookupKnowledgeTool {
|
|||||||
@Autowired
|
@Autowired
|
||||||
private RetrievedDocTracker retrievedDocTracker;
|
private RetrievedDocTracker retrievedDocTracker;
|
||||||
|
|
||||||
|
@Autowired
|
||||||
|
private ObjectMapper objectMapper;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 查询知识库文档
|
* 查询知识库文档
|
||||||
*
|
*
|
||||||
@@ -50,7 +73,6 @@ public class LookupKnowledgeTool {
|
|||||||
"4) 配置说明 - 查询系统配置、中间件参数,例如 'HikariCP'、'Redis 集群配置'。" +
|
"4) 配置说明 - 查询系统配置、中间件参数,例如 'HikariCP'、'Redis 集群配置'。" +
|
||||||
"参数 query: 查询关键词或描述")
|
"参数 query: 查询关键词或描述")
|
||||||
public LookupResult lookupKnowledge(String query) {
|
public LookupResult lookupKnowledge(String query) {
|
||||||
// 生成请求ID用于追踪
|
|
||||||
String requestId = java.util.UUID.randomUUID().toString().substring(0, 8);
|
String requestId = java.util.UUID.randomUUID().toString().substring(0, 8);
|
||||||
long startTime = System.currentTimeMillis();
|
long startTime = System.currentTimeMillis();
|
||||||
|
|
||||||
@@ -69,7 +91,7 @@ public class LookupKnowledgeTool {
|
|||||||
log.info("[L0 精确匹配] 找到文档:");
|
log.info("[L0 精确匹配] 找到文档:");
|
||||||
for (int i = 0; i < Math.min(3, l0Matches.size()); i++) {
|
for (int i = 0; i < Math.min(3, l0Matches.size()); i++) {
|
||||||
KnowledgeEntry entry = l0Matches.get(i);
|
KnowledgeEntry entry = l0Matches.get(i);
|
||||||
log.info(" - [{}] 标题: {}, 路径: {}", i+1, entry.getTitle(), entry.getFilePath());
|
log.info(" - [{}] 标题: {}, 路径: {}, 域: {}", i+1, entry.getTitle(), entry.getFilePath(), entry.getCategory());
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -91,90 +113,201 @@ public class LookupKnowledgeTool {
|
|||||||
log.info("[L1 语义检索] 找到文档:");
|
log.info("[L1 语义检索] 找到文档:");
|
||||||
for (int i = 0; i < Math.min(3, l1Results.size()); i++) {
|
for (int i = 0; i < Math.min(3, l1Results.size()); i++) {
|
||||||
VectorSearchService.SearchResult result = l1Results.get(i);
|
VectorSearchService.SearchResult result = l1Results.get(i);
|
||||||
log.info(" - [{}] 文档ID: {}, 相似度得分: {}", i+1, result.getId(), result.getScore());
|
log.info(" - [{}] 文档ID: {}, L2距离: {}", i+1, result.getId(), String.format("%.4f", result.getScore()));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
log.info("[L1 语义检索] L0唯一匹配,跳过L1检索");
|
log.info("[L1 语义检索] L0唯一匹配,跳过L1检索");
|
||||||
}
|
}
|
||||||
|
|
||||||
// Step 4: 组装结果
|
// Step 4: 归一化质量等级判定
|
||||||
LookupResult result = buildResult(l0Matches, l1Results, highConfidence);
|
float l1TopScore = (l1Results != null && !l1Results.isEmpty()) ? l1Results.get(0).getScore() : Float.MAX_VALUE;
|
||||||
|
RelevanceAssessment assessment = computeRelevance(l0Matches.size(), l1TopScore);
|
||||||
|
log.info("[归一化] relevanceLevel={}, completenessHint={}", assessment.level, assessment.hint);
|
||||||
|
if (l1TopScore != Float.MAX_VALUE) {
|
||||||
|
double similarity = normalizeL2(l1TopScore);
|
||||||
|
log.info("[归一化] L2距离={}, similarity={}", String.format("%.4f", l1TopScore), String.format("%.4f", similarity));
|
||||||
|
}
|
||||||
|
|
||||||
// Step 5: session 级去重过滤
|
// Step 5: 组装结果
|
||||||
|
LookupResult result = buildResult(l0Matches, l1Results, highConfidence);
|
||||||
|
result.setRelevanceLevel(assessment.level);
|
||||||
|
result.setCompletenessHint(assessment.hint);
|
||||||
|
|
||||||
|
// Step 6: session 级去重过滤 + 域级行动记忆
|
||||||
String sessionId = SessionContextHolder.getSessionId();
|
String sessionId = SessionContextHolder.getSessionId();
|
||||||
|
String domain = extractDomain(l0Matches, l1Results);
|
||||||
|
|
||||||
if (sessionId != null && result.isFound()) {
|
if (sessionId != null && result.isFound()) {
|
||||||
String docKey = extractDocKey(result);
|
String docKey = extractDocKey(result);
|
||||||
if (docKey != null && retrievedDocTracker.isAlreadyRetrieved(sessionId, docKey)) {
|
if (docKey != null && retrievedDocTracker.isAlreadyRetrieved(sessionId, docKey)) {
|
||||||
log.info("[去重] 文档已在本会话中检索过,跳过: {}", docKey);
|
log.info("[去重] 文档已在本会话中检索过,跳过: {}", docKey);
|
||||||
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result);
|
List<String> retrievedDomains = retrievedDocTracker.getRetrievedDomains(sessionId);
|
||||||
|
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result, domain, "doc_retrieved");
|
||||||
return LookupResult.builder()
|
return LookupResult.builder()
|
||||||
.found(false)
|
.found(false)
|
||||||
.message("文档已在本会话中检索过,无需重复召回: " + docKey)
|
.message("文档已在本会话中检索过,无需重复召回: " + docKey)
|
||||||
|
.relevanceLevel(assessment.level)
|
||||||
|
.completenessHint(assessment.hint)
|
||||||
|
.retrievedDomainsThisSession(retrievedDomains)
|
||||||
.build();
|
.build();
|
||||||
}
|
}
|
||||||
if (docKey != null) {
|
if (docKey != null) {
|
||||||
retrievedDocTracker.markRetrieved(sessionId, docKey);
|
retrievedDocTracker.markRetrieved(sessionId, domain, docKey);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 记录结构化结果摘要(替代原始 MD 内容预览)
|
// 附加行动记忆
|
||||||
|
if (sessionId != null) {
|
||||||
|
result.setRetrievedDomainsThisSession(retrievedDocTracker.getRetrievedDomains(sessionId));
|
||||||
|
}
|
||||||
|
|
||||||
|
// 记录结构化结果摘要
|
||||||
long totalTime = System.currentTimeMillis() - startTime;
|
long totalTime = System.currentTimeMillis() - startTime;
|
||||||
log.info("----------------------------------------");
|
log.info("----------------------------------------");
|
||||||
log.info("<<< [工具返回] lookup_knowledge");
|
log.info("<<< [工具返回] lookup_knowledge");
|
||||||
log.info("<<< 结果: found={}, 耗时: {}ms (L0={}ms, L1={}ms)",
|
log.info("<<< 结果: found={}, relevanceLevel={}, 耗时: {}ms",
|
||||||
result.isFound(), totalTime, l0Time,
|
result.isFound(), result.getRelevanceLevel(), totalTime);
|
||||||
l1Results != null ? System.currentTimeMillis() - startTime - l0Time : 0);
|
log.info("<<< 行动记忆: retrievedDomainsThisSession={}", result.getRetrievedDomainsThisSession());
|
||||||
|
|
||||||
// L0 精确匹配摘要
|
|
||||||
if (!l0Matches.isEmpty()) {
|
if (!l0Matches.isEmpty()) {
|
||||||
KnowledgeEntry top = l0Matches.get(0);
|
KnowledgeEntry top = l0Matches.get(0);
|
||||||
log.info("<<< [L0 主结果] 标题: {}", top.getTitle());
|
log.info("<<< [L0 主结果] 标题: {}", top.getTitle());
|
||||||
log.info("<<< [L0 主结果] 来源: {}", top.getFilePath());
|
log.info("<<< [L0 主结果] 来源: {}", top.getFilePath());
|
||||||
|
log.info("<<< [L0 主结果] 域: {}", top.getCategory());
|
||||||
if (top.getSummary() != null) {
|
if (top.getSummary() != null) {
|
||||||
log.info("<<< [L0 主结果] 摘要: {}", top.getSummary());
|
log.info("<<< [L0 主结果] 摘要: {}", top.getSummary());
|
||||||
}
|
}
|
||||||
if (top.getKeywords() != null && !top.getKeywords().isEmpty()) {
|
|
||||||
log.info("<<< [L0 主结果] 关键词: {}", String.join(", ", top.getKeywords()));
|
|
||||||
}
|
|
||||||
// 内容概况:长度 + 章节数
|
|
||||||
String content = result.getPrimary() != null ? result.getPrimary().getContent() : null;
|
String content = result.getPrimary() != null ? result.getPrimary().getContent() : null;
|
||||||
if (content != null) {
|
if (content != null) {
|
||||||
int headingCount = countMdHeadings(content);
|
int headingCount = countMdHeadings(content);
|
||||||
log.info("<<< [L0 主结果] 内容: {} 字符, {} 个章节",
|
log.info("<<< [L0 主结果] 内容: {} 字符, {} 个章节", content.length(), headingCount);
|
||||||
content.length(), headingCount);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// L1 语义检索摘要
|
|
||||||
if (l1Results != null && !l1Results.isEmpty()) {
|
if (l1Results != null && !l1Results.isEmpty()) {
|
||||||
VectorSearchService.SearchResult topL1 = l1Results.get(0);
|
VectorSearchService.SearchResult topL1 = l1Results.get(0);
|
||||||
log.info("<<< [L1 补充] 来源: {}", topL1.getMetadata() != null ? topL1.getMetadata() : topL1.getId());
|
log.info("<<< [L1 补充] 来源: {}", topL1.getMetadata() != null ? topL1.getMetadata() : topL1.getId());
|
||||||
log.info("<<< [L1 补充] 相似度: {}", String.format("%.4f", topL1.getScore()));
|
log.info("<<< [L1 补充] L2距离: {}, similarity: {}",
|
||||||
if (topL1.getContent() != null) {
|
String.format("%.4f", topL1.getScore()),
|
||||||
String snippet = extractFirstMeaningfulLine(topL1.getContent(), 120);
|
String.format("%.4f", normalizeL2(topL1.getScore())));
|
||||||
log.info("<<< [L1 补充] 内容片段: {}", snippet);
|
|
||||||
log.info("<<< [L1 补充] 片段长度: {} 字符", topL1.getContent().length());
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
log.info("========================================");
|
log.info("========================================");
|
||||||
|
|
||||||
// 记录 tool_invocation(持久化检索明细)
|
// 记录 tool_invocation
|
||||||
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result);
|
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result, domain, null);
|
||||||
|
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ==================== 归一化层 ====================
|
||||||
|
|
||||||
|
/**
|
||||||
|
* L2 距离 Min-Max 归一化到 [0,1] similarity
|
||||||
|
* BGE-M3 输出 L2 归一化单位向量,L2 距离硬上界 = 2.0
|
||||||
|
* similarity = 1 - min(score, maxL2Distance) / maxL2Distance
|
||||||
|
* score=0 → 1.0(完全相同),score=2.0 → 0.0(完全相反)
|
||||||
|
*/
|
||||||
|
double normalizeL2(float l2Score) {
|
||||||
|
double clamped = Math.min(l2Score, maxL2Distance);
|
||||||
|
return 1.0 - clamped / maxL2Distance;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 归一化质量等级判定
|
||||||
|
*
|
||||||
|
* @param l0MatchCount L0 匹配数
|
||||||
|
* @param l1TopScore L1 最高分(L2 距离),无 L1 结果时传 Float.MAX_VALUE
|
||||||
|
* @return RelevanceAssessment(level + hint)
|
||||||
|
*/
|
||||||
|
RelevanceAssessment computeRelevance(int l0MatchCount, float l1TopScore) {
|
||||||
|
double l1Similarity = (l1TopScore != Float.MAX_VALUE) ? normalizeL2(l1TopScore) : 0.0;
|
||||||
|
|
||||||
|
// L0 唯一匹配 → PRECISE
|
||||||
|
if (l0MatchCount == 1) {
|
||||||
|
return new RelevanceAssessment(LEVEL_PRECISE, HINT_PRECISE);
|
||||||
|
}
|
||||||
|
|
||||||
|
// L0 命中 + L1 高分 → HIGHLY_RELEVANT
|
||||||
|
if (l0MatchCount > 1 && l1Similarity >= highlyRelevantThreshold) {
|
||||||
|
return new RelevanceAssessment(LEVEL_HIGHLY_RELEVANT, HINT_HIGHLY_RELEVANT);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 仅 L1 高分 → HIGHLY_RELEVANT
|
||||||
|
if (l0MatchCount == 0 && l1Similarity >= highlyRelevantThreshold) {
|
||||||
|
return new RelevanceAssessment(LEVEL_HIGHLY_RELEVANT, HINT_HIGHLY_RELEVANT);
|
||||||
|
}
|
||||||
|
|
||||||
|
// L0 多匹配 + L1 中分 → REFERENCE
|
||||||
|
if (l0MatchCount > 1 && l1Similarity >= referenceThreshold) {
|
||||||
|
return new RelevanceAssessment(LEVEL_REFERENCE, HINT_REFERENCE);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 仅 L1 中分 → REFERENCE
|
||||||
|
if (l0MatchCount == 0 && l1Similarity >= referenceThreshold) {
|
||||||
|
return new RelevanceAssessment(LEVEL_REFERENCE, HINT_REFERENCE);
|
||||||
|
}
|
||||||
|
|
||||||
|
// L0 多匹配 + 无 L1 / L1 低分 → REFERENCE(L0 命中本身有价值)
|
||||||
|
if (l0MatchCount > 1) {
|
||||||
|
return new RelevanceAssessment(LEVEL_REFERENCE, HINT_REFERENCE);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 无有效结果
|
||||||
|
return new RelevanceAssessment(null, null);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 归一化评估结果
|
||||||
|
*/
|
||||||
|
record RelevanceAssessment(String level, String hint) {}
|
||||||
|
|
||||||
|
// ==================== 域提取 ====================
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 从检索结果中提取域信息
|
||||||
|
* 优先使用 L0 的 category,兜底从 L1 metadata 解析
|
||||||
|
*/
|
||||||
|
private String extractDomain(List<KnowledgeEntry> l0Matches, List<VectorSearchService.SearchResult> l1Results) {
|
||||||
|
// 优先 L0
|
||||||
|
if (l0Matches != null && !l0Matches.isEmpty()) {
|
||||||
|
String category = l0Matches.get(0).getCategory();
|
||||||
|
if (category != null && !category.isBlank()) {
|
||||||
|
return category;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 兜底 L1:从 metadata JSON 中解析 category
|
||||||
|
if (l1Results != null && !l1Results.isEmpty()) {
|
||||||
|
try {
|
||||||
|
String metadata = l1Results.get(0).getMetadata();
|
||||||
|
if (metadata != null && metadata.contains("category")) {
|
||||||
|
var node = objectMapper.readTree(metadata);
|
||||||
|
if (node.has("category")) {
|
||||||
|
return node.get("category").asText();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (Exception e) {
|
||||||
|
log.debug("L1 metadata 解析 category 失败: {}", e.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ==================== 入库 ====================
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 保存工具调用明细到 tool_invocation 表
|
* 保存工具调用明细到 tool_invocation 表
|
||||||
*/
|
*/
|
||||||
private void saveToolInvocation(String query, List<KnowledgeEntry> l0Matches,
|
private void saveToolInvocation(String query, List<KnowledgeEntry> l0Matches,
|
||||||
List<VectorSearchService.SearchResult> l1Results,
|
List<VectorSearchService.SearchResult> l1Results,
|
||||||
boolean highConfidence, long startTime, LookupResult result) {
|
boolean highConfidence, long startTime,
|
||||||
|
LookupResult result, String domain, String dedupReason) {
|
||||||
try {
|
try {
|
||||||
String sessionId = SessionContextHolder.getSessionId();
|
String sessionId = SessionContextHolder.getSessionId();
|
||||||
if (sessionId == null) return; // 非会话上下文不记录
|
if (sessionId == null) return;
|
||||||
|
|
||||||
boolean hasL0 = l0Matches != null && !l0Matches.isEmpty();
|
boolean hasL0 = l0Matches != null && !l0Matches.isEmpty();
|
||||||
boolean hasL1 = l1Results != null && !l1Results.isEmpty();
|
boolean hasL1 = l1Results != null && !l1Results.isEmpty();
|
||||||
@@ -201,7 +334,7 @@ public class LookupKnowledgeTool {
|
|||||||
layer = null;
|
layer = null;
|
||||||
}
|
}
|
||||||
|
|
||||||
// 拼接 output_preview(前500字符)
|
// output_preview
|
||||||
if (result != null && result.getPrimary() != null && result.getPrimary().getContent() != null) {
|
if (result != null && result.getPrimary() != null && result.getPrimary().getContent() != null) {
|
||||||
String content = result.getPrimary().getContent();
|
String content = result.getPrimary().getContent();
|
||||||
outputLength = content.length();
|
outputLength = content.length();
|
||||||
@@ -222,24 +355,48 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 构建检索明细 JSON
|
// L1 top score + similarity
|
||||||
|
float l1TopScore = (hasL1) ? l1Results.get(0).getScore() : -1;
|
||||||
|
double l1TopSimilarity = (hasL1) ? normalizeL2(l1TopScore) : -1;
|
||||||
|
|
||||||
|
// 构建检索明细 JSON(扩展版)
|
||||||
StringBuilder details = new StringBuilder("{");
|
StringBuilder details = new StringBuilder("{");
|
||||||
if (hasL0) {
|
if (hasL0) {
|
||||||
|
details.append("\"l0_match_count\":").append(l0Count).append(",");
|
||||||
details.append("\"l0_titles\":[");
|
details.append("\"l0_titles\":[");
|
||||||
for (int i = 0; i < Math.min(3, l0Matches.size()); i++) {
|
for (int i = 0; i < Math.min(3, l0Matches.size()); i++) {
|
||||||
if (i > 0) details.append(",");
|
if (i > 0) details.append(",");
|
||||||
details.append("\"").append(escapeJson(l0Matches.get(i).getTitle())).append("\"");
|
details.append("\"").append(escapeJson(l0Matches.get(i).getTitle())).append("\"");
|
||||||
}
|
}
|
||||||
details.append("]");
|
details.append("],");
|
||||||
}
|
}
|
||||||
if (hasL1) {
|
if (hasL1) {
|
||||||
if (hasL0) details.append(",");
|
details.append("\"l1_top_score\":").append(String.format("%.4f", l1TopScore)).append(",");
|
||||||
|
details.append("\"l1_top_similarity\":").append(String.format("%.4f", l1TopSimilarity)).append(",");
|
||||||
|
details.append("\"l1_match_count\":").append(l1Count).append(",");
|
||||||
details.append("\"l1_scores\":[");
|
details.append("\"l1_scores\":[");
|
||||||
for (int i = 0; i < Math.min(3, l1Results.size()); i++) {
|
for (int i = 0; i < Math.min(3, l1Results.size()); i++) {
|
||||||
if (i > 0) details.append(",");
|
if (i > 0) details.append(",");
|
||||||
details.append(l1Results.get(i).getScore());
|
details.append(String.format("%.4f", l1Results.get(i).getScore()));
|
||||||
}
|
}
|
||||||
details.append("]");
|
details.append("],");
|
||||||
|
}
|
||||||
|
// 归一化信息
|
||||||
|
if (result != null && result.getRelevanceLevel() != null) {
|
||||||
|
details.append("\"relevance_level\":\"").append(result.getRelevanceLevel()).append("\",");
|
||||||
|
details.append("\"completeness_hint\":\"").append(escapeJson(result.getCompletenessHint())).append("\",");
|
||||||
|
}
|
||||||
|
// 域信息
|
||||||
|
if (domain != null) {
|
||||||
|
details.append("\"retrieved_domains\":[\"").append(escapeJson(domain)).append("\"],");
|
||||||
|
}
|
||||||
|
// 去重原因
|
||||||
|
if (dedupReason != null) {
|
||||||
|
details.append("\"dedup_reason\":\"").append(dedupReason).append("\",");
|
||||||
|
}
|
||||||
|
// 移除末尾逗号
|
||||||
|
if (details.charAt(details.length() - 1) == ',') {
|
||||||
|
details.setLength(details.length() - 1);
|
||||||
}
|
}
|
||||||
details.append("}");
|
details.append("}");
|
||||||
|
|
||||||
@@ -254,12 +411,15 @@ public class LookupKnowledgeTool {
|
|||||||
.l1MatchCount(hasL1 ? l1Count : null)
|
.l1MatchCount(hasL1 ? l1Count : null)
|
||||||
.isTruncated(truncated)
|
.isTruncated(truncated)
|
||||||
.retrievalDetails(details.toString())
|
.retrievalDetails(details.toString())
|
||||||
|
.relevanceLevel(result != null ? result.getRelevanceLevel() : null)
|
||||||
|
.dedupReason(dedupReason)
|
||||||
.durationMs((int) duration)
|
.durationMs((int) duration)
|
||||||
.success(true)
|
.success(true)
|
||||||
.build();
|
.build();
|
||||||
|
|
||||||
toolInvocationRepository.save(inv);
|
toolInvocationRepository.save(inv);
|
||||||
log.debug("tool_invocation 已保存: sessionId={}, layer={}, duration={}ms", sessionId, layer, duration);
|
log.debug("tool_invocation 已保存: sessionId={}, layer={}, relevanceLevel={}, duration={}ms",
|
||||||
|
sessionId, layer, result != null ? result.getRelevanceLevel() : null, duration);
|
||||||
} catch (Exception e) {
|
} catch (Exception e) {
|
||||||
log.error("保存 tool_invocation 失败", e);
|
log.error("保存 tool_invocation 失败", e);
|
||||||
}
|
}
|
||||||
@@ -274,14 +434,8 @@ public class LookupKnowledgeTool {
|
|||||||
.replace("\t", "\\t");
|
.replace("\t", "\\t");
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
// ==================== 结果组装 ====================
|
||||||
* 组装查询结果
|
|
||||||
*
|
|
||||||
* @param l0Matches L0 匹配结果
|
|
||||||
* @param l1Results L1 检索结果
|
|
||||||
* @param highConfidence 是否高置信度
|
|
||||||
* @return 组装后的结果
|
|
||||||
*/
|
|
||||||
private LookupResult buildResult(
|
private LookupResult buildResult(
|
||||||
List<KnowledgeEntry> l0Matches,
|
List<KnowledgeEntry> l0Matches,
|
||||||
List<VectorSearchService.SearchResult> l1Results,
|
List<VectorSearchService.SearchResult> l1Results,
|
||||||
@@ -294,8 +448,6 @@ public class LookupKnowledgeTool {
|
|||||||
if (l0Matches != null && !l0Matches.isEmpty()) {
|
if (l0Matches != null && !l0Matches.isEmpty()) {
|
||||||
KnowledgeEntry first = l0Matches.get(0);
|
KnowledgeEntry first = l0Matches.get(0);
|
||||||
boolean hasL1 = l1Results != null && !l1Results.isEmpty();
|
boolean hasL1 = l1Results != null && !l1Results.isEmpty();
|
||||||
|
|
||||||
// 场景决策:唯一匹配或 L1 无结果 → LLM 需要正文内容;多匹配且有 L1 → 只需元数据
|
|
||||||
boolean needFullContent = highConfidence || !hasL1;
|
boolean needFullContent = highConfidence || !hasL1;
|
||||||
String content = needFullContent
|
String content = needFullContent
|
||||||
? buildCompactSummary(first)
|
? buildCompactSummary(first)
|
||||||
@@ -307,7 +459,7 @@ public class LookupKnowledgeTool {
|
|||||||
.source(first.getFilePath())
|
.source(first.getFilePath())
|
||||||
.matchType("exact_L0")
|
.matchType("exact_L0")
|
||||||
.confidence(highConfidence ? "high" : "low")
|
.confidence(highConfidence ? "high" : "low")
|
||||||
.availableSections(null) // MVP 返回 null
|
.availableSections(null)
|
||||||
.build();
|
.build();
|
||||||
log.debug("L0结果已构建: source={}, contentLength={}", first.getFilePath(), content.length());
|
log.debug("L0结果已构建: source={}, contentLength={}", first.getFilePath(), content.length());
|
||||||
} else {
|
} else {
|
||||||
@@ -330,16 +482,12 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
builder.supplement(supplement);
|
builder.supplement(supplement);
|
||||||
|
|
||||||
// 判断是否找到结果(primary 或 supplement 至少有一个)
|
|
||||||
boolean found = (primary != null) || (supplement != null);
|
boolean found = (primary != null) || (supplement != null);
|
||||||
builder.found(found);
|
builder.found(found);
|
||||||
|
|
||||||
return builder.build();
|
return builder.build();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* 统计 MD 文档中的章节数(二级标题 ## 数量)
|
|
||||||
*/
|
|
||||||
private int countMdHeadings(String content) {
|
private int countMdHeadings(String content) {
|
||||||
if (content == null) return 0;
|
if (content == null) return 0;
|
||||||
return (int) content.lines()
|
return (int) content.lines()
|
||||||
@@ -347,15 +495,10 @@ public class LookupKnowledgeTool {
|
|||||||
.count();
|
.count();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* 构建紧凑文档摘要(替代原始 MD 全文,节省上下文窗口)
|
|
||||||
* 组合:title/summary + 章节结构 + 正文片段(~500 字符)
|
|
||||||
*/
|
|
||||||
private String buildCompactSummary(KnowledgeEntry entry) {
|
private String buildCompactSummary(KnowledgeEntry entry) {
|
||||||
String rawContent = knowledgeIndexService.readDocument(entry.getFilePath(), 2000);
|
String rawContent = knowledgeIndexService.readDocument(entry.getFilePath(), 2000);
|
||||||
if (rawContent == null) return null;
|
if (rawContent == null) return null;
|
||||||
|
|
||||||
// 跳过 YAML frontmatter 得到正文
|
|
||||||
String body = rawContent;
|
String body = rawContent;
|
||||||
if (body.startsWith("---")) {
|
if (body.startsWith("---")) {
|
||||||
int end = body.indexOf("---", 3);
|
int end = body.indexOf("---", 3);
|
||||||
@@ -365,14 +508,11 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
StringBuilder sb = new StringBuilder();
|
StringBuilder sb = new StringBuilder();
|
||||||
|
|
||||||
// 1. 元数据头(始终包含)
|
|
||||||
sb.append("文档: ").append(entry.getTitle()).append("\n");
|
sb.append("文档: ").append(entry.getTitle()).append("\n");
|
||||||
if (entry.getSummary() != null) {
|
if (entry.getSummary() != null) {
|
||||||
sb.append("摘要: ").append(entry.getSummary()).append("\n");
|
sb.append("摘要: ").append(entry.getSummary()).append("\n");
|
||||||
}
|
}
|
||||||
|
|
||||||
// 2. 章节结构(## 标题列表)
|
|
||||||
String headings = body.lines()
|
String headings = body.lines()
|
||||||
.filter(l -> l.trim().startsWith("##"))
|
.filter(l -> l.trim().startsWith("##"))
|
||||||
.map(l -> " - " + l.trim().replaceAll("^#+\\s*", ""))
|
.map(l -> " - " + l.trim().replaceAll("^#+\\s*", ""))
|
||||||
@@ -382,13 +522,11 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
sb.append("---\n");
|
sb.append("---\n");
|
||||||
|
|
||||||
// 3. 正文片段(去标题行、去空行,智能截断)
|
|
||||||
String textContent = body.lines()
|
String textContent = body.lines()
|
||||||
.filter(l -> !l.trim().startsWith("#") && !l.trim().isEmpty())
|
.filter(l -> !l.trim().startsWith("#") && !l.trim().isEmpty())
|
||||||
.collect(Collectors.joining("\n"))
|
.collect(Collectors.joining("\n"))
|
||||||
.trim();
|
.trim();
|
||||||
|
|
||||||
// 短文档保留更多内容,长文档节省上下文
|
|
||||||
int maxBodyChars = body.length() < 500 ? 800 : 500;
|
int maxBodyChars = body.length() < 500 ? 800 : 500;
|
||||||
if (textContent.length() > maxBodyChars) {
|
if (textContent.length() > maxBodyChars) {
|
||||||
sb.append(textContent, 0, maxBodyChars).append("...");
|
sb.append(textContent, 0, maxBodyChars).append("...");
|
||||||
@@ -399,10 +537,6 @@ public class LookupKnowledgeTool {
|
|||||||
return sb.toString();
|
return sb.toString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* 构建纯元数据摘要(不读文件,仅用内存索引信息)
|
|
||||||
* 多匹配且有 L1 补充时使用,L0 只需告知 LLM 命中了哪些文档
|
|
||||||
*/
|
|
||||||
private String buildMetadataOnlySummary(KnowledgeEntry entry) {
|
private String buildMetadataOnlySummary(KnowledgeEntry entry) {
|
||||||
StringBuilder sb = new StringBuilder();
|
StringBuilder sb = new StringBuilder();
|
||||||
sb.append("文档: ").append(entry.getTitle()).append("\n");
|
sb.append("文档: ").append(entry.getTitle()).append("\n");
|
||||||
@@ -416,14 +550,10 @@ public class LookupKnowledgeTool {
|
|||||||
return sb.toString();
|
return sb.toString();
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* 提取 MD 内容中第一个有意义的文本行(跳过 frontmatter 和标题行)
|
|
||||||
*/
|
|
||||||
private String extractFirstMeaningfulLine(String content, int maxLen) {
|
private String extractFirstMeaningfulLine(String content, int maxLen) {
|
||||||
if (content == null || content.isBlank()) return "(空)";
|
if (content == null || content.isBlank()) return "(空)";
|
||||||
|
|
||||||
String text = content.trim();
|
String text = content.trim();
|
||||||
// 跳过 YAML frontmatter (--- ... ---)
|
|
||||||
if (text.startsWith("---")) {
|
if (text.startsWith("---")) {
|
||||||
int end = text.indexOf("---", 3);
|
int end = text.indexOf("---", 3);
|
||||||
if (end != -1) {
|
if (end != -1) {
|
||||||
@@ -431,7 +561,6 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 查找第一个非空、非标题行
|
|
||||||
String[] lines = text.split("\n");
|
String[] lines = text.split("\n");
|
||||||
for (String line : lines) {
|
for (String line : lines) {
|
||||||
String tl = line.trim();
|
String tl = line.trim();
|
||||||
@@ -440,7 +569,6 @@ public class LookupKnowledgeTool {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// 兜底:第一行非空行
|
|
||||||
for (String line : lines) {
|
for (String line : lines) {
|
||||||
if (!line.trim().isEmpty()) {
|
if (!line.trim().isEmpty()) {
|
||||||
String tl = line.trim();
|
String tl = line.trim();
|
||||||
|
|||||||
@@ -3,34 +3,87 @@ package com.superbiz.agent.tool;
|
|||||||
import org.springframework.stereotype.Component;
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
import java.util.Collections;
|
import java.util.Collections;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Map;
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
import java.util.concurrent.ConcurrentHashMap;
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.stream.Collectors;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* session 级已召回文档追踪器
|
* session 级已召回文档追踪器
|
||||||
* 防止同一 session 内重复召回相同文档
|
* 支持文档级去重 + 域级行动记忆
|
||||||
|
*
|
||||||
|
* 数据结构:sessionId → { domain → Set<filePath> }
|
||||||
|
* - 域级:控制"不要重复查同域",提供行动记忆给 LLM
|
||||||
|
* - 文档级:控制"不要重复召回同文档"(替代原有单层结构)
|
||||||
*/
|
*/
|
||||||
@Component
|
@Component
|
||||||
public class RetrievedDocTracker {
|
public class RetrievedDocTracker {
|
||||||
|
|
||||||
// key: sessionId, value: 已召回文档的 filePath 集合
|
// key: sessionId, value: { domain → Set<filePath> }
|
||||||
private final ConcurrentHashMap<String, Set<String>> retrieved = new ConcurrentHashMap<>();
|
private final ConcurrentHashMap<String, Map<String, Set<String>>> sessionRetrievals = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
public boolean isAlreadyRetrieved(String sessionId, String filePath) {
|
/**
|
||||||
if (sessionId == null || filePath == null) return false;
|
* 记录一次检索(域级 + 文档级)
|
||||||
Set<String> docs = retrieved.get(sessionId);
|
*/
|
||||||
return docs != null && docs.contains(filePath);
|
public void markRetrieved(String sessionId, String domain, String filePath) {
|
||||||
}
|
|
||||||
|
|
||||||
public void markRetrieved(String sessionId, String filePath) {
|
|
||||||
if (sessionId == null || filePath == null) return;
|
if (sessionId == null || filePath == null) return;
|
||||||
retrieved.computeIfAbsent(sessionId,
|
|
||||||
k -> Collections.newSetFromMap(new ConcurrentHashMap<>()))
|
sessionRetrievals.computeIfAbsent(sessionId,
|
||||||
|
k -> new ConcurrentHashMap<>())
|
||||||
|
.computeIfAbsent(domain != null ? domain : "_unknown",
|
||||||
|
d -> Collections.newSetFromMap(new ConcurrentHashMap<>()))
|
||||||
.add(filePath);
|
.add(filePath);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 文档级去重:检查 filePath 是否已在本会话中检索过
|
||||||
|
*/
|
||||||
|
public boolean isDocRetrieved(String sessionId, String filePath) {
|
||||||
|
if (sessionId == null || filePath == null) return false;
|
||||||
|
Map<String, Set<String>> domains = sessionRetrievals.get(sessionId);
|
||||||
|
if (domains == null) return false;
|
||||||
|
return domains.values().stream().anyMatch(docs -> docs.contains(filePath));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 域级检查:检查 domain 是否已在本会话中检索过
|
||||||
|
*/
|
||||||
|
public boolean isDomainRetrieved(String sessionId, String domain) {
|
||||||
|
if (sessionId == null || domain == null) return false;
|
||||||
|
Map<String, Set<String>> domains = sessionRetrievals.get(sessionId);
|
||||||
|
return domains != null && domains.containsKey(domain);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 获取本次会话已检索的域列表(行动记忆,返回给 LLM)
|
||||||
|
*/
|
||||||
|
public List<String> getRetrievedDomains(String sessionId) {
|
||||||
|
if (sessionId == null) return List.of();
|
||||||
|
Map<String, Set<String>> domains = sessionRetrievals.get(sessionId);
|
||||||
|
if (domains == null) return List.of();
|
||||||
|
return List.copyOf(domains.keySet());
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 向后兼容:文档级去重(委托给 isDocRetrieved)
|
||||||
|
*/
|
||||||
|
public boolean isAlreadyRetrieved(String sessionId, String filePath) {
|
||||||
|
return isDocRetrieved(sessionId, filePath);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 向后兼容:旧版 markRetrieved(domain 设为 null,归入 _unknown)
|
||||||
|
*/
|
||||||
|
public void markRetrieved(String sessionId, String filePath) {
|
||||||
|
markRetrieved(sessionId, null, filePath);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 清理会话
|
||||||
|
*/
|
||||||
public void clearSession(String sessionId) {
|
public void clearSession(String sessionId) {
|
||||||
if (sessionId == null) return;
|
if (sessionId == null) return;
|
||||||
retrieved.remove(sessionId);
|
sessionRetrievals.remove(sessionId);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -119,6 +119,13 @@ document:
|
|||||||
rag:
|
rag:
|
||||||
top-k: 3 # 检索返回的最相似文档数量
|
top-k: 3 # 检索返回的最相似文档数量
|
||||||
|
|
||||||
|
# 检索归一化配置
|
||||||
|
retrieval:
|
||||||
|
normalization:
|
||||||
|
max-l2-distance: 2.0 # L2 距离上界(BGE-M3 单位向量 = 2.0)
|
||||||
|
highly-relevant-threshold: 0.75 # similarity >= 0.75 → HIGHLY_RELEVANT
|
||||||
|
reference-threshold: 0.5 # similarity >= 0.5 → REFERENCE
|
||||||
|
|
||||||
# Prometheus 配置
|
# Prometheus 配置
|
||||||
prometheus:
|
prometheus:
|
||||||
base-url: http://localhost:9090
|
base-url: http://localhost:9090
|
||||||
|
|||||||
@@ -0,0 +1,6 @@
|
|||||||
|
-- V010: 新增 relevance_level 和 dedup_reason 列到 tool_invocation 表
|
||||||
|
-- 用于检索归一化等级和去重原因的可观测性
|
||||||
|
|
||||||
|
ALTER TABLE tool_invocation
|
||||||
|
ADD COLUMN relevance_level VARCHAR(20) COMMENT '归一化质量等级:PRECISE/HIGHLY_RELEVANT/REFERENCE/DEDUPED',
|
||||||
|
ADD COLUMN dedup_reason VARCHAR(32) COMMENT '去重原因:doc_retrieved/domain_retrieved/null';
|
||||||
@@ -2,11 +2,38 @@
|
|||||||
|
|
||||||
## 职责
|
## 职责
|
||||||
- 按步骤执行具体的查询任务
|
- 按步骤执行具体的查询任务
|
||||||
- 使用知识库查询、日志查询等工具获取信息
|
- 需要外部信息时调用工具,但须遵守下方的检索约束
|
||||||
- 将执行结果汇总,给出完整的最终答案
|
- 不要凭记忆回答,必须基于工具返回的真实数据
|
||||||
|
- 执行完成后,综合所有结果给出完整的答案
|
||||||
|
|
||||||
## 规则
|
## 规则
|
||||||
- 按顺序执行,不可跳过步骤
|
- 按顺序执行,不可跳过步骤
|
||||||
- 所有需要外部信息的地方,都必须调用对应的工具
|
|
||||||
- 不要凭记忆回答,必须基于工具返回的真实数据
|
- 不要凭记忆回答,必须基于工具返回的真实数据
|
||||||
- 执行完成后,综合所有结果给出完整的答案
|
- 执行完成后,综合所有结果给出完整的答案
|
||||||
|
|
||||||
|
## 检索约束
|
||||||
|
|
||||||
|
### 1. 判断重复:基于已检索上下文
|
||||||
|
每次 lookup_knowledge 返回值中包含 `retrievedDomainsThisSession`,
|
||||||
|
表示本次会话已检索过的知识域。如果当前问题与已检索域语义重叠,
|
||||||
|
**禁止再次调用 lookup_knowledge**。
|
||||||
|
|
||||||
|
### 2. 重复了该怎么办
|
||||||
|
如果当前想检索的内容与【已检索上下文】语义相似:
|
||||||
|
- 禁止换关键词重新检索
|
||||||
|
- 直接基于已有事实回答
|
||||||
|
- 如果信息不足,先明确指出缺少什么具体维度
|
||||||
|
(如:"缺少 HikariCP 具体配置参数"、"缺少连接池耗尽的日志样例"),
|
||||||
|
再针对该维度进行一次定向补充检索——而非盲目换词重查
|
||||||
|
|
||||||
|
### 3. 合法出口:允许信息不全时给出结论
|
||||||
|
如果你认为已有信息足以回答核心问题,即使细节不全,
|
||||||
|
也请直接给出结论并说明局限性(如:"基于已有信息,连接池配置建议如下,
|
||||||
|
但具体参数值需结合实际负载调整")。
|
||||||
|
**不查全不会被追责,重复检索才会被惩罚。**
|
||||||
|
|
||||||
|
### 4. 利用质量信号判断
|
||||||
|
- relevanceLevel=PRECISE → 信息精准,直接使用,不再检索
|
||||||
|
- relevanceLevel=HIGHLY_RELEVANT + 域已在 retrievedDomainsThisSession → 禁止再次调用
|
||||||
|
- relevanceLevel=REFERENCE → 先指出缺什么维度,再定向补充一次
|
||||||
|
- completenessHint 是知识库给你的天花板信号,信任它
|
||||||
|
|||||||
@@ -86,8 +86,15 @@ class FullPipelineSmokeTest {
|
|||||||
boolean hasNonZero = vector.stream().anyMatch(v -> Math.abs(v) > 1e-6);
|
boolean hasNonZero = vector.stream().anyMatch(v -> Math.abs(v) > 1e-6);
|
||||||
assertTrue(hasNonZero, "向量不能全为零");
|
assertTrue(hasNonZero, "向量不能全为零");
|
||||||
|
|
||||||
|
// L2 范数校验:BGE-M3 输出应为 L2 归一化的单位向量
|
||||||
|
double norm = Math.sqrt(vector.stream().mapToDouble(v -> (double) v * v).sum());
|
||||||
|
|
||||||
System.out.println("维度: " + vector.size());
|
System.out.println("维度: " + vector.size());
|
||||||
System.out.println("前5维: " + vector.subList(0, Math.min(5, vector.size())));
|
System.out.println("前5维: " + vector.subList(0, Math.min(5, vector.size())));
|
||||||
|
System.out.println("L2 范数: " + String.format("%.10f", norm));
|
||||||
|
System.out.println("是否归一化 (|norm - 1.0| < 0.01): " + (Math.abs(norm - 1.0) < 0.01));
|
||||||
|
|
||||||
|
assertEquals(1.0, norm, 0.01, "BGE-M3 向量应为 L2 归一化单位向量,实际范数=" + norm);
|
||||||
System.out.println("Embedding ✓");
|
System.out.println("Embedding ✓");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user