feat(graph): add diagnosis real nodes
This commit is contained in:
zhuyongxin
@@ -4,6 +4,7 @@
|
||||
|
||||
| 日期 | slug | 说明 | 领域 | 关键词 | 关联 OpenSpec | 状态 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 2026-07-17 | chat-diagnosis-stategraph-real-nodes | 接入真实 Agent/Java Nodes、显式 Gatekeeper、可信输入投影、关键证据补查与安全 Fallback，暂不切换生产入口。 | Chat diagnosis orchestration/nodes | ReactAgent adapter, Gatekeeper node, verified input, evidence retry, safe fallback, CompiledGraph | openspec/changes/archive/2026-07-17-chat-diagnosis-stategraph-real-nodes | archived |
|
||||
| 2026-07-17 | chat-diagnosis-stategraph-routing-skeleton | 实现未接生产入口的 Diagnosis StateGraph 骨架、有限路由和 Fake Node 测试。 | Chat diagnosis orchestration/graph | StateGraph, fake node, conditional edge, retry counter, orchestration events, trace builder | openspec/changes/archive/2026-07-17-chat-diagnosis-stategraph-routing-skeleton | archived |
|
||||
| 2026-07-17 | chat-diagnosis-stategraph-design-freeze | 冻结 ISS-011 的 Graph State、条件边、有限重试、安全降级、审计和测试迁移边界。 | Chat diagnosis orchestration/design | StateGraph, runId, Gatekeeper, verified evidence, fallback, orchestration trace, test migration | openspec/changes/archive/2026-07-17-chat-diagnosis-stategraph-design-freeze | archived |
|
||||
| 2026-07-10 | session-run-trace-isolation | 拆分会话态和运行态，引入 runId 隔离 Trace、Feedback、AIOps 和 demo 链路。 | Trace/session/run isolation | chat_session, diagnosis_run, runId, trace exact run, feedback fallback, AIOps SSE metadata, baseline drift | openspec/changes/archive/2026-07-10-session-run-trace-isolation | archived |
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
# Chat Diagnosis StateGraph Real Nodes 验收
|
||||
|
||||
## 结果
|
||||
|
||||
已接受。OpenSpec tasks 27/27 完成；真实 Nodes 可构造和测试，旧生产 Sequential 路径保持可用，阶段 2 未切换生产入口。
|
||||
|
||||
## 静态验证
|
||||
|
||||
- `openspec validate chat-diagnosis-stategraph-real-nodes --strict`：通过。
|
||||
- `openspec validate --specs --strict`：13 passed，0 failed。
|
||||
- `git diff --check`：通过；只有 LF/CRLF 转换提示，无 whitespace error。
|
||||
- 源码/引用检查：`ChatService` Graph 引用 0；新增源码 forbidden refs 0；protocol 反向依赖 0；DB/Trace/Prompt diff 0。
|
||||
- 装配对齐：无核心 TODO/FIXME/placeholder；Graph factory 继续拥有所有 retry counter 与 Planner reset。
|
||||
|
||||
## 脚本验证
|
||||
|
||||
- 新 protocol/Node/CompiledGraph/Router/Trace focused suite：通过。
|
||||
- `mvn -q "-Dtest=ChatServiceSequentialAgentTest,VerifierInputHookTest,ExecutorGatekeeperServiceTest" test`：通过。
|
||||
- 两组测试合计：100 tests，0 failures，0 errors，0 skipped。
|
||||
- `mvn -q -DskipTests test-compile`：通过。
|
||||
|
||||
## 浏览器/人工验证
|
||||
|
||||
- 未运行。阶段 2 不改变用户入口或 UI，没有独立人工验收价值。
|
||||
|
||||
## 未验证
|
||||
|
||||
- Maven live E2E、`logs/` 日志和 `scripts/query_mysql.py` 数据库核验未运行。
|
||||
- 原因：用户明确要求只在阶段 5 全部实现后统一做最终 E2E；阶段 2 尚未切换生产入口。
|
||||
- 风险：当前验收只证明组件/Graph 契约与旧路径回归，不证明真实生产装配、模型、DB 和 Trace 全链路。
|
||||
|
||||
## 已完成范围
|
||||
|
||||
- 中立共享 protocol 与旧路径行为保持型委托。
|
||||
- 四个 Agent adapters、显式 Gatekeeper、可信投影、关键证据补查、Composer 与两类 Fallback。
|
||||
- 真实 CompiledGraph 装配、critical gap 路由修复和完整 focused test 证据。
|
||||
|
||||
## 交接
|
||||
|
||||
- 下一步：归档本 change、独立提交阶段 2，然后启动阶段 3 `chat-diagnosis-stategraph-chatservice-cutover`。
|
||||
- OpenSpec 归档：已同步主 specs，并归档到 `openspec/changes/archive/2026-07-17-chat-diagnosis-stategraph-real-nodes/`。
|
||||
- 归档授权：用户已明确“直接实现吧，不用找我授权了”，授权后续阶段在门禁通过后直接归档和提交。
|
||||
@@ -0,0 +1,21 @@
|
||||
# Chat Diagnosis StateGraph Real Nodes Brief
|
||||
|
||||
## 背景
|
||||
|
||||
- 用户目标：将 ISS-011 阶段 2 作为独立 sm-flow，接入真实 Agent/Java Nodes，并在归档、验收和独立 Git 提交后才进入阶段 3。
|
||||
- 当前问题：阶段 1 只有 Fake Node 路由骨架；Executor/Verifier/Composer 协议逻辑分散在 Hook 与 ChatService，真实 Graph 尚不能安全调用 Agent、Gatekeeper 或投影可信材料。
|
||||
- 关联 OpenSpec：`openspec/changes/archive/2026-07-17-chat-diagnosis-stategraph-real-nodes/`
|
||||
- devflow 分档：complex
|
||||
|
||||
## 范围
|
||||
|
||||
- 本次要做：共享无状态 protocol 组件；Planner/Executor/Verifier/Composer adapters；显式 Gatekeeper、Verified Input、Evidence Retry、Fallback Nodes；真实 CompiledGraph 装配和 focused tests。
|
||||
- 本次不做：不切换 ChatService 生产入口，不修改 DB、Trace API、Prompt 契约，不删除旧 Sequential/Hook，不运行 live E2E。
|
||||
- 影响区域：`diagnosis.protocol`、`graph.diagnosis`、`VerifierInputHook`、`ChatService` 共享逻辑委托及对应测试。
|
||||
|
||||
## OpenSpec 对齐
|
||||
|
||||
- proposal 覆盖状态：已覆盖。
|
||||
- design 覆盖状态：已覆盖；接口影响为 L2，生产切换明确延期到阶段 3。
|
||||
- specs 覆盖状态：已覆盖真实 Nodes 安全边界，并修正 critical evidence gap 路由条件。
|
||||
- tasks 覆盖状态：27/27 完成。
|
||||
@@ -0,0 +1,218 @@
|
||||
# Chat Diagnosis StateGraph Real Nodes Decisions
|
||||
|
||||
## Question Pool
|
||||
|
||||
| # | 维度 | 问题 | 模式 | 状态 |
|
||||
|---|---|---|---|---|
|
||||
| Q1 | 边界 | 阶段 2 是否切换 ChatService/DB/Trace？ | user-interview（六阶段已确认） | 已解决 |
|
||||
| Q2 | 复用 | 新 Nodes 如何避免复制 Hook/ChatService 解析与安全渲染？ | evidence-driven | 已解决 |
|
||||
| Q3 | Agent | Adapter 如何调用真实 ReactAgent 并保留 Hook/ToolCallback/run config？ | evidence-driven | 已解决 |
|
||||
| Q4 | Gatekeeper | 显式 Node 如何保证当前 run、单次调用和 fail-closed 标准化？ | evidence-driven | 已解决 |
|
||||
| Q5 | 安全 | passed checked bindings 如何投影为 verified output/evidence？ | evidence-driven | 已解决 |
|
||||
| Q6 | 补证据 | 哪些 facts 可触发 evidence retry，completed queries 如何表达？ | evidence-driven | 已解决 |
|
||||
| Q7 | Fallback | 前置验证失败与 Composer 后置失败可分别使用哪些材料？ | evidence-driven | 已解决 |
|
||||
| Q8 | Prompt | 阶段 2 是否立即修改共享 Verifier Prompt？ | evidence-driven | 已解决 |
|
||||
| Q9 | 验收 | 阶段 2 是否需要单元/集成测试与 E2E？ | evidence-driven + user rule | 已解决 |
|
||||
|
||||
## Evidence-driven
|
||||
|
||||
| 结论 | 证据来源 | 是否已汇报用户 |
|
||||
|---|---|---|
|
||||
| ReactAgent.call(input, config) 返回 AssistantMessage，现有 AgentLoggingHook 从 config metadata 读取 sessionId/runId | 本地 1.1.2.0 javap、AgentLoggingHook | 已汇报 |
|
||||
| Executor parser 当前在 VerifierInputHook，Verifier/Composer parser 与 renderer 当前在 ChatService | 定向源码阅读和 references | 已汇报 |
|
||||
| checked_bindings 提供 claim_id/tool_name/source_invocation_id/raw_path/matched_text/status | ExecutorGatekeeperService | 已汇报 |
|
||||
| Gatekeeper.validateRun 使用 run-scoped ToolInvocation repository | ExecutorGatekeeperService tests/code | 已汇报 |
|
||||
| Graph path不能注册旧 VerifierInputHook，否则 Gatekeeper 会双执行且输入包含 full tool trace | 阶段 0 ADR、Hook 源码 | 已汇报 |
|
||||
| evidence retry 只允许 critical no_evidence/indirect_support | 阶段 0 design/ISS-011 | 已汇报 |
|
||||
| 阶段 1 router 未检查 is_critical，属于实现偏离 | Router 源码与阶段 0 baseline 对照 | 已汇报 |
|
||||
| 共享 Prompt 暂时仍服务旧 Sequential path，本阶段直接修改会提前破坏生产 payload | ChatService agent builder + VerifierInputHook + prompt | 已汇报 |
|
||||
| Node 输入/安全边界新增且旧解析会重构，单元和 focused regression 必要；E2E 不必要 | 阶段边界和用户规则 | 已汇报 |
|
||||
|
||||
## User-interview
|
||||
|
||||
| 问题原文 | 用户原话 | 确认状态 | OpenSpec 回写 |
|
||||
|---|---|---|---|
|
||||
| 阶段 2 是否独立 sm-flow？ | “iss-011里每个阶段，都是一个sm-flow” | 已确认 | 独立 change |
|
||||
| 是否可提前切生产入口？ | “每个阶段需要归档完并提交才能进入下一个阶段” | 已确认不可提前阶段 3 | Out of Scope |
|
||||
| 阶段 2 是否执行 E2E？ | “端到端只在最后阶段全部完成后才验证” | 已确认不执行 | Acceptance |
|
||||
| 是否添加单元测试？ | “如果有必要添加单元测试验收的话，就加” | 已确认规则；本阶段判定必要 | Acceptance |
|
||||
|
||||
## Context And Handoff
|
||||
|
||||
- 阶段 0 archive/commit：design baseline / `581daff`
|
||||
- 阶段 1 archive/commit：routing skeleton / `42ba204`
|
||||
- 当前 change：`chat-diagnosis-stategraph-real-nodes`
|
||||
- 后续 change：`chat-diagnosis-stategraph-chatservice-cutover`，只能在本阶段 archive + commit 后创建。
|
||||
|
||||
## Technical Decisions
|
||||
|
||||
### Shared protocol components
|
||||
|
||||
- 抽取 Executor、Verifier、Composer 解析器，旧 Hook/ChatService 委托新组件。
|
||||
- 抽取 Composer safe input builder / renderer，旧 ChatService 保持相同输出。
|
||||
- JSON sanitization 只存在一份共享实现，不在每个 Node copy。
|
||||
- 抽取过程是行为保持 refactor；现有 focused tests 是回归门禁。
|
||||
|
||||
### Agent adapter boundary
|
||||
|
||||
- `DiagnosisAgentInvoker` 是最小 port：`invoke(String, RunnableConfig) -> String`。
|
||||
- `ReactAgentDiagnosisInvoker` 只包装 `ReactAgent.call(...).getText()`。
|
||||
- Planner/Executor/Verifier/Composer adapters 各自拥有白名单 input projector、parser 和 status event。
|
||||
- tests 使用 fake invoker，另有 ReactAgent wrapper test。
|
||||
|
||||
### Legacy Hook coexistence
|
||||
|
||||
- 旧 Sequential path 在阶段 3 前仍通过 VerifierInputHook 执行 Gatekeeper。
|
||||
- Graph Verifier Agent 不注册该 Hook；显式 Gatekeeper Node 是 Graph 中唯一 validation 入口。
|
||||
- Parser/enricher 可共享，但 Hook 的 legacy payload/prompt 暂不改变。
|
||||
- 阶段 3 切换生产入口并同步 Verifier Prompt；阶段 5 删除旧隐式结构。
|
||||
|
||||
### Gatekeeper normalization
|
||||
|
||||
- raw pass → PASS。
|
||||
- raw fail + severity low_confid → LOW_CONFID。
|
||||
- raw fail + severity reject → REJECT。
|
||||
- 缺失、unknown、异常或不一致 → REJECT。
|
||||
- verified_binding_count 只统计 checked_bindings.status=pass。
|
||||
|
||||
### Verified projection
|
||||
|
||||
- 通过项按 claim_id + source_invocation_id + tool_name + raw_path 与原 binding 精确匹配。
|
||||
- verified_executor_output 只包含 answer_version 与至少一条 passed binding 的 filtered claims；合法零 claim 保持空列表。
|
||||
- verified_evidence 只含 claim_id/source_invocation_id/tool_name/raw_path/matched_text。
|
||||
- hypotheses、失败 binding、未引用 ToolInvocation、raw Executor output 不投影。
|
||||
|
||||
### Evidence retry
|
||||
|
||||
- extractor 要求 `is_critical=true` 且 verification 为 no_evidence/indirect_support。
|
||||
- gap 字段：claim_id（从 `claim-id: text` 提取或空）、fact、verification、reason。
|
||||
- completed queries 由 verified evidence 的 tool/invocation/path 去重生成。
|
||||
- prior verified output/evidence 原样只读进入 retry_context。
|
||||
- 约束固定：max_retry=1、do_not_repeat_successful_queries、only_execute_incremental_queries、preserve_prior_verified_claims。
|
||||
- Executor 输入声明“增量执行、完整输出”，Java 不合并 claims。
|
||||
|
||||
### Interface impact
|
||||
|
||||
- 等级：L2 internal interface；共享 parser 委托保持旧行为。
|
||||
- 新消费者：阶段 3 Graph orchestrator/Agent factory。
|
||||
- 当前外部 API/DB/生产路由：无变化。
|
||||
- 回滚：revert 本阶段提交；旧 Sequential path仍完整。
|
||||
- 有意修正：Router 只允许 critical gap，属于对冻结基线的代码修复。
|
||||
|
||||
## Risks Accepted
|
||||
|
||||
- 真实模型尚未执行；Node contract 通过 fake invoker/real service tests证明，阶段 5 才 E2E。
|
||||
- Prompt 输入说明与 Graph input 的最终同步推迟到阶段 3，避免当前旧生产 Hook 提前不兼容。
|
||||
- 旧 Hook 暂时仍存在，但不进入 Graph action graph；阶段 5 必须删除。
|
||||
|
||||
## Architecture Audit
|
||||
|
||||
### Module and ownership map
|
||||
|
||||
`Diagnosis Context + RunnableConfig` → Agent adapters / deterministic Nodes → owned Graph State fields → `DiagnosisGraphRouter` → next Node or END → `final_answer + orchestration_events`。阶段 2 只提供这条可构造链，阶段 3 才由 `ChatService` 创建 Run、构造 Agent 实例并调用 Graph。
|
||||
|
||||
| 模块 | 数据所有权 | 允许依赖 |
|
||||
|---|---|---|
|
||||
| `diagnosis.protocol` | JSON contract、纯解析结果、安全 input/rendering | Jackson 与纯 DTO；不依赖 Graph/Hook/ChatService/ThreadLocal |
|
||||
| `graph.diagnosis` Agent adapters | 白名单输入、Agent attempt status/event | protocol、注入的 invoker、Graph API |
|
||||
| Gatekeeper Node | raw Gatekeeper result、normalized status/count | ExecutorGatekeeperService、当前 run config |
|
||||
| Verified Input / Retry Prepare | verified projection、critical gaps、retry context | raw result 的只读投影与 protocol DTO |
|
||||
| Router / Factory | 条件边、有限计数、调用次序 | Graph State；不解析 Agent raw output |
|
||||
| legacy Hook / ChatService | 阶段 3 前的生产 Sequential 流程 | 只委托 protocol；不得消费真实 Graph action set |
|
||||
|
||||
### Lifecycle and coupling audit
|
||||
|
||||
Graph State 和 events 都是 invocation-scoped，runId 只从当前 RunnableConfig 获取，Nodes 不持有跨 Run 可变状态。旧路径与 Graph 路径阶段性共享的只有无状态 protocol 组件和 Gatekeeper service，不共享 ThreadLocal 或 Agent output。ReactAgent/invoker 必须构造注入，阶段 2 不复制 ChatService Prompt/Agent factory。唯一有意的阶段性耦合是 legacy consumers 改为委托 protocol，这由旧 focused tests 和完整 revert 保护。阶段 3 前生产入口、DB、Trace、Prompt 均保持隔离。
|
||||
|
||||
### Cross-artifact alignment
|
||||
|
||||
| 对齐链 | 结果 | 证据 |
|
||||
|---|---|---|
|
||||
| brief/proposal 目标、范围、非目标 → proposal | 已对齐 | 独立阶段边界、真实 Nodes、无生产切换均明确 |
|
||||
| proposal 承诺与约束 → design | 已对齐 | invoker、共享组件、Gatekeeper、投影、retry、Fallback、L2 均有决策 |
|
||||
| design 架构/接口结论 → specs/tasks | 已对齐 | 中立 protocol 包、构造注入、fail-closed 和生产隔离均有任务/行为 |
|
||||
| specs 可观察行为 → tasks 可执行切片 | 已对齐 | 每个 requirement 至少由一个实现任务和一个测试/验收任务覆盖 |
|
||||
|
||||
### Audit result
|
||||
|
||||
审计发现共享协议组件包所有权与 Agent 实例装配边界需要显式化，已回写 design/tasks。未发现状态字段、路由计数、run 生命周期或阶段边界的新冲突。接口影响维持 L2，消费者都在本 change 与下一阶段明确范围内。剩余风险是共享抽取的旧行为漂移和 binding 投影泄漏，均有 focused regression 与 mixed-binding tests。架构风险可接受，无未解决问题。
|
||||
|
||||
## Commit Gate
|
||||
|
||||
- proposal/design/specs/tasks：全部存在，OpenSpec status `isComplete=true`。
|
||||
- 当前 change strict validation：通过。
|
||||
- 主 specs strict validation：13 passed，0 failed。
|
||||
- 规格结构：9 requirements、22 scenarios；tasks：27 个 checkbox 切片。
|
||||
- Cross-artifact：4/4 已对齐，gap=0。
|
||||
- 接口影响：L2，已在 design 独立章节记录消费者、兼容和回滚。
|
||||
- Question pool：所有 evidence-driven 已汇报；所有 user-interview 已确认；无未决项。
|
||||
- Preflight scope：`git diff --check` 通过；Commit checkpoint 未修改业务代码。
|
||||
- 结论：Draft OpenSpec 达到可执行状态，创建 `.committed`。
|
||||
|
||||
## Apply Authorization
|
||||
|
||||
- 用户原话：“直接实现吧，不用找我授权了”。
|
||||
- 解释：阶段 2–5 后续 checkpoint 可在前置门禁通过后直接继续，不再因 Apply 或 Archive 授权暂停。
|
||||
- 不扩大范围：每阶段仍须独立 OpenSpec、验收归档、Git commit；阶段 0–4 不做 E2E，阶段 5 才统一执行。
|
||||
|
||||
## Pre-apply Research
|
||||
|
||||
### Reference implementations read
|
||||
|
||||
- `src/main/java/com/superbiz/agent/hook/VerifierInputHook.java`：Executor JSON sanitization、parse status、tool-name normalization、唯一 invocation 回填与 legacy Gatekeeper payload。
|
||||
- `src/main/java/com/superbiz/agent/service/ChatService.java`：Verifier claim/fact parser、Gatekeeper ceiling、Composer allowed-material builder、Composer parser/safe renderer、旧 retry context。
|
||||
- `src/main/java/com/superbiz/agent/service/ExecutorGatekeeperService.java`：`validateRun`、severity normalization source、checked binding 与 matched_text 结构。
|
||||
- `src/main/java/com/superbiz/agent/graph/diagnosis/DiagnosisGraphFactory.java`：config-aware action ports、technical retry/evidence retry counter 所有权和 recursion limit。
|
||||
- `src/main/java/com/superbiz/agent/graph/diagnosis/OrchestrationEvent.java` 与 `src/test/java/com/superbiz/agent/graph/diagnosis/ScriptedDiagnosisGraphActions.java`：每 attempt 单 event 形态。
|
||||
- `src/main/java/com/superbiz/agent/hook/AgentLoggingHook.java`：RunnableConfig metadata 中 sessionId/runId 的审计读取方式。
|
||||
- `src/test/java/com/superbiz/agent/hook/VerifierInputHookTest.java`、`ChatServiceSequentialAgentTest.java`、`DiagnosisGraphRoutingTest.java`：JUnit 5、Mockito 边界替身和 CompiledGraph observable behavior 测试风格。
|
||||
- 本地 `1.1.2.0` JAR `javap`：`ReactAgent.call(String,RunnableConfig)` 与 `RunnableConfig.threadId/metadata` 公共 API。
|
||||
|
||||
### Technology inventory
|
||||
|
||||
| 类别 | 项目标准 / 本阶段使用 |
|
||||
|---|---|
|
||||
| JSON contract | Jackson `ObjectMapper`、`LinkedHashMap` 保持稳定字段顺序；共享 sanitization 只存在一份 |
|
||||
| Graph Node | `AsyncNodeActionWithConfig` 返回 `CompletableFuture<Map<String,Object>>`；state 默认 Replace、events Append |
|
||||
| Agent boundary | 构造注入的 `DiagnosisAgentInvoker`；ReactAgent wrapper 透传同一个 RunnableConfig |
|
||||
| Run scope | `config.metadata("runId")` 为 Gatekeeper 唯一运行边界；缺失即 fail closed |
|
||||
| Error handling | 合法 contract/invalid output/temporary/permanent 分离；unknown exception 不推断为可重试 |
|
||||
| Tests | JUnit 5；只在 ReactAgent、repository 等系统边界使用 fake/mock；Node/CompiledGraph 走公开 action/graph 接口 |
|
||||
| Request/response | 本阶段不改 Controller/DTO/API，不适用 |
|
||||
| MQ/Consumer | 本阶段不涉及，不适用 |
|
||||
| DB/Trace/Prompt | 本阶段禁止修改，阶段 3 处理 |
|
||||
|
||||
### Reuse and new infrastructure
|
||||
|
||||
- 新建中立 `com.superbiz.agent.diagnosis.protocol`：`JsonPayloadSupport`、Executor/Verifier/Composer protocol、safe input/rendering、`EvidenceGapExtractor`；不得依赖 Graph/Hook/ChatService/ThreadLocal。
|
||||
- 新建 `graph.diagnosis` Node 层：invoker wrapper、failure classifier、四个 Agent adapters、Gatekeeper、Verified Input、Retry Prepare、Fallback 和 action assembly。
|
||||
- 不新建 Prompt/Agent factory；阶段 3 通过构造注入已有 ReactAgent 实例。
|
||||
- 不新增 Maven dependency、数据库迁移、配置项或生产 consumer。
|
||||
|
||||
### Pre-apply conclusion
|
||||
|
||||
参考实现、API 签名、异常和测试标准已足以指导实现；未发现 devflow/OpenSpec 冲突。进入 TDD tracer bullet，先锁定共享 Executor parser 的合法 no-evidence 与 malformed 行为。
|
||||
|
||||
## Apply Completion And Verification
|
||||
|
||||
### Assembly alignment
|
||||
|
||||
- 首个共享 protocol 模块与最终真实 Graph action assembly 均逐项对照 design/specs：invoker 显式透传 RunnableConfig，Gatekeeper 单次 fail-closed，Verified Input 精确投影，Verifier/Composer 固定输入重试，critical evidence retry 与两类 Fallback 边界全部落地。
|
||||
- `DiagnosisGraphFactory` 继续独占技术重试计数、evidence retry 计数和 Planner mode/reset；Node 不重复拥有编排计数。
|
||||
- 新增源码中无 `ThreadLocal`、`VerifierInputHook`、`tool_trace_summary`、`raw_executor`、TODO 或 FIXME；protocol 包无 Graph/Hook/ChatService 反向依赖。
|
||||
- `ChatService` 无 Diagnosis Graph/CompiledGraph 引用，生产切换保持在阶段 3；DB migration、Trace DTO/entity/repository 和 prompts 均无 diff。
|
||||
|
||||
### Automated verification
|
||||
|
||||
- 新 protocol/Node/真实 CompiledGraph/Router/Trace focused suite：通过。
|
||||
- 旧 `ChatServiceSequentialAgentTest`、`VerifierInputHookTest`、`ExecutorGatekeeperServiceTest` 回归：通过。
|
||||
- 合计 100 tests，0 failures，0 errors，0 skipped。
|
||||
- `mvn -q -DskipTests test-compile`：通过。
|
||||
- `openspec validate chat-diagnosis-stategraph-real-nodes --strict`：通过。
|
||||
- `openspec validate --specs --strict`：13 passed，0 failed。
|
||||
- `git diff --check`：通过；仅报告 Git 既有 LF/CRLF 转换提示，无 whitespace error。
|
||||
|
||||
### Deferred final verification
|
||||
|
||||
- 阶段 2 按用户确认不运行 Maven live E2E，不启动应用，不检查 `logs/`，不查询数据库。
|
||||
- 上述端到端、日志和 `scripts/query_mysql.py` 数据库核验统一保留到阶段 5 全部实现完成后执行。
|
||||
@@ -0,0 +1,25 @@
|
||||
# Chat Diagnosis StateGraph Real Nodes Evidence
|
||||
|
||||
## 证据
|
||||
|
||||
| 来源 | 证据 | 结论 | 是否已汇报 |
|
||||
|---|---|---|---|
|
||||
| `VerifierInputHook.java`、`ChatService.java` | Executor/Verifier/Composer 解析与安全渲染原实现 | 抽取到中立 protocol 并让旧路径委托，避免双真理源 | 是 |
|
||||
| `ExecutorGatekeeperService.java` | `validateRun` 与 checked binding/matched_text 契约 | Graph Gatekeeper 必须按当前 runId 单次调用并 fail closed | 是 |
|
||||
| 本地 Graph/ReactAgent 1.1.2.0 API | `ReactAgent.call(String,RunnableConfig)` 与 config-aware Graph action | 最小 invoker 可精确透传输入和当前 Run metadata | 是 |
|
||||
| 阶段 0/1 OpenSpec archives | 路由、计数、状态与安全边界冻结 | 阶段 2 不改变 Graph counter 所有权或生产入口 | 是 |
|
||||
| 新 protocol/Node/CompiledGraph tests | PASS、REJECT、LOW_CONFID、critical retry、固定输入重试和安全 fallback | 真实 Nodes 的可观察路径和材料边界已覆盖 | 是 |
|
||||
| 旧 Sequential/Hook/Gatekeeper tests | 共享抽取后的旧路径回归 | 阶段 2 未破坏当前生产控制流 | 是 |
|
||||
|
||||
## Evidence-driven 结论
|
||||
|
||||
- Graph Verifier 不得注册旧 `VerifierInputHook`，否则会双执行 Gatekeeper 并泄漏完整 tool trace。
|
||||
- Verified Input 必须用 claim/invocation/tool/path 精确关联 passed binding；不能复刻 Gatekeeper 判断或读取未引用工具结果。
|
||||
- Router 与 Retry Prepare 必须共用 critical-gap 提取规则：仅 `is_critical=true` 的 `no_evidence`/`indirect_support`。
|
||||
- 技术重试输入必须字节一致，且不得重跑前序 Node；计数仍由 Graph factory 统一拥有。
|
||||
- 当前 ChatService 无 Graph 引用，DB/Trace/Prompt 无 diff，满足阶段 2 的生产隔离要求。
|
||||
|
||||
## 风险与后续证据
|
||||
|
||||
- 本阶段使用 fake invoker 和 focused tests，不证明真实模型/外部基础设施联通；阶段 5 最终 E2E 统一补证。
|
||||
- Prompt 输入说明、生产装配、Run/Trace 持久化属于阶段 3，不能提前从阶段 2 证据推断已完成。
|
||||
+3
@@ -0,0 +1,3 @@
|
||||
ready_at: 2026-07-17
|
||||
devflow: devflow/projects/2026-07-17-chat-diagnosis-stategraph-real-nodes
|
||||
authorization: user-requested-per-stage-archive
|
||||
@@ -0,0 +1 @@
|
||||
Committed by sm-flow after stage 2 Commit checkpoint validation on 2026-07-17.
|
||||
@@ -0,0 +1,106 @@
|
||||
## Context
|
||||
|
||||
阶段 1 已交付未接生产入口的 Diagnosis StateGraph 状态、拓扑、路由和 trace builder。当前复杂 Chat 仍由 `ChatService`、`SequentialAgent`、`VerifierInputHook` 和 `VerifierContextHolder` 编排；Executor 解析在 Hook 内，Verifier/Composer 解析与安全渲染在 ChatService 内。阶段 2 只把真实 ReactAgent 和确定性 Java 服务接入 Graph action ports，并抽出可复用协议组件，生产切换留到阶段 3。
|
||||
|
||||
本 change 涉及 `graph.diagnosis`、Hook 和 ChatService 内部协作，接口影响为 L2。`/api/chat`、数据库、Trace API、Prompt 业务协议和当前生产路由均不改变。
|
||||
|
||||
## Goals / Non-Goals
|
||||
|
||||
**Goals:**
|
||||
|
||||
- 提供显式透传 `RunnableConfig` 的 Planner、Executor、Verifier、Composer adapters。
|
||||
- 将 Executor、Verifier、Composer 协议解析和安全渲染抽为单一共享实现，并让旧路径委托以保持行为。
|
||||
- 将 Gatekeeper、可信输入投影、evidence retry prepare 和两类 Fallback 实现为确定性 Java Node。
|
||||
- 保证 Verifier 只接收通过 binding 投影的材料，并将执行状态、模型 verdict 和 effective verdict 分离。
|
||||
- 修正 LOW_CONFID evidence retry 仅接受 critical gap 的阶段 1 偏差。
|
||||
|
||||
**Non-Goals:**
|
||||
|
||||
- 不切换 `ChatService.executeChatComplex` 到 StateGraph。
|
||||
- 不修改 `/api/chat`、数据库、Run/Trace DTO 或持久化。
|
||||
- 不删除 SequentialAgent、VerifierInputHook 或 VerifierContextHolder。
|
||||
- 不修改共享 Verifier Prompt；其输入说明在阶段 3 切换时同步。
|
||||
- 不运行 Maven live E2E、日志或数据库验收。
|
||||
|
||||
## Decisions
|
||||
|
||||
### 1. Agent 调用经最小 port 隔离
|
||||
|
||||
`DiagnosisAgentInvoker` 只暴露 `invoke(String, RunnableConfig) -> String`；`ReactAgentDiagnosisInvoker` 包装 `ReactAgent.call(input, config)` 并返回消息文本。各 Agent adapter 自己负责白名单输入序列化、输出解析、状态映射和 orchestration event，不读取完整父 Graph State。
|
||||
|
||||
选择最小 port 而不是直接使用 `ReactAgent.asNode(...)`，因为显式字符串输入便于证明输入白名单、固定技术重试输入和 run config 透传，也避免父 Graph messages/private state 泄漏。ReactAgent/invoker 通过构造注入；本阶段不复制 ChatService 的 Prompt 或 Agent factory，生产实例装配属于阶段 3。
|
||||
|
||||
### 2. 共享协议组件替代复制
|
||||
|
||||
在中立的 `com.superbiz.agent.diagnosis.protocol` 包抽取无状态的 `JsonPayloadSupport`、Executor/Verifier/Composer parser、Composer safe input builder 和 fallback renderer。旧 `VerifierInputHook`/`ChatService` 委托这些组件，新 Nodes 使用同一实现；抽取本身不得改变旧可观察行为。共享组件不得依赖 `graph.diagnosis`、Hook、ChatService 或 ThreadLocal。
|
||||
|
||||
选择共享组件而不是在 Graph 包复制旧私有方法，避免旧 Sequential 与新 Graph 对同一 JSON 契约产生两个真理源。阶段 2 保留旧控制流只是迁移顺序，不形成长期双轨。
|
||||
|
||||
### 3. Node failure 分类 fail closed
|
||||
|
||||
Adapter 将合法结构映射为 COMPLETED，将可识别的临时调用失败映射为 RETRYABLE_FAILED，将非法 JSON/结构映射为 INVALID_OUTPUT；未知或明确不可重试异常映射为 NON_RETRYABLE_FAILED/FAILED。Executor 不论失败类型都不重试，合法 no-evidence 结构仍为 COMPLETED。
|
||||
|
||||
分类器允许注入以便测试。无法确定的异常不猜测为可重试，防止无限或扩大副作用。
|
||||
|
||||
### 4. Gatekeeper 是 Graph 唯一验证入口
|
||||
|
||||
Graph Verifier ReactAgent 不注册旧 `VerifierInputHook`。显式 Gatekeeper Node 从 `RunnableConfig.metadata.runId` 和 `executor_output` 调用 `ExecutorGatekeeperService.validateRun` 恰好一次，同时保存 raw result 与 normalized status：pass 为 PASS；fail/low_confid 为 LOW_CONFID；fail/reject 为 REJECT；缺失、unknown、异常或自相矛盾均为 REJECT。
|
||||
|
||||
旧 Sequential 路径在阶段 3 前仍由 Hook 调用 Gatekeeper。两个入口服务于互斥的控制流，不允许同一次 Graph run 双执行。
|
||||
|
||||
### 5. Verified Input 按 binding 精确投影
|
||||
|
||||
Builder 只接受 Gatekeeper `checked_bindings.status=pass`。它以 `claim_id + source_invocation_id + tool_name + raw_path` 精确关联 Executor claim binding，输出过滤后的 `verified_executor_output` 和只含 `claim_id/source_invocation_id/tool_name/raw_path/matched_text` 的 `verified_evidence`。
|
||||
|
||||
失败 binding、hypotheses、未引用工具结果、完整 `tool_trace_summary` 和 raw Executor 文本不得进入 Verifier。合法零 claim 输出保留空列表，但 LOW_CONFID 零可信 binding 已由 Router 在 Builder 前阻断。
|
||||
|
||||
### 6. Verifier 状态与 verdict 分离
|
||||
|
||||
Verifier parser 产出执行状态和模型 verdict；Adapter 再应用 Gatekeeper ceiling 得到 effective verdict。技术失败只写 `verifier_status`，不得伪造诊断 verdict。ceiling=LOW_CONFID 时模型 PASS 最高只能得到 LOW_CONFID。
|
||||
|
||||
Verifier technical retry 复用首次生成的完全相同输入字符串；Composer technical retry 同样复用首次 allowed-material 输入。重试不得读取变化后的前序 raw state。
|
||||
|
||||
### 7. Evidence retry 使用共享 critical-gap extractor
|
||||
|
||||
`EvidenceGapExtractor` 同时被 Router guard 与 Retry Prepare Node 使用，唯一资格为 `is_critical=true` 且 `verification` 为 `no_evidence` 或 `indirect_support`。这样修复阶段 1 Router 漏检 critical 标记，又避免路由判断与 retry payload 漂移。
|
||||
|
||||
Retry context 包含 prior verified output/evidence、结构化 gaps、从 verified evidence 去重得到的 completed query refs，以及固定约束：最多一次、不重复成功查询、只做增量查询、保留 prior verified claims。第二轮 Executor 被要求执行增量查询但输出完整 `executor_evidence_v2` 快照；Java 不合并 claim 文本，完整快照重新经过 Gatekeeper。
|
||||
|
||||
### 8. 两类 Fallback 使用不同材料边界
|
||||
|
||||
前置验证 Fallback 只使用 reason code、校验状态、工具概况和人工查看 Trace 建议，绝不输出 Executor claim。Composer 后置 Fallback 只使用 Verifier 已允许的 claims、missing info 和 recommendations，不读取 raw Executor/tool output。
|
||||
|
||||
Fallback 由确定性 renderer 生成；若连安全答案都无法生成，异常交给阶段 3 的外层 Run failure handling。
|
||||
|
||||
### 9. 阶段内装配与测试边界
|
||||
|
||||
新增真实 Node action set/factory 装配入口，但不让 ChatService 成为消费者。单元测试使用 fake invoker 验证 Node 契约，使用真实 `ExecutorGatekeeperService` mock 边界验证单次调用，并以 CompiledGraph 场景验证 adapters、critical evidence retry 和 Fallback 路由。旧 Sequential/Hook/Gatekeeper focused tests作为共享抽取回归门禁。
|
||||
|
||||
## Interface Impact
|
||||
|
||||
- 等级：L2 内部接口。
|
||||
- 新内部消费者：阶段 3 的 Graph orchestrator/Agent factory。
|
||||
- 旧内部消费者：VerifierInputHook 与 ChatService 改为委托共享 parser/renderer，公开方法与外部响应不变。
|
||||
- 外部 API、DB、Trace、Prompt 输出协议：无变化。
|
||||
- 回滚：revert 本阶段提交；生产仍走旧 Sequential 路径。
|
||||
|
||||
## Risks / Trade-offs
|
||||
|
||||
- [共享解析器抽取造成旧行为漂移] → 保留原输入/输出形态并运行旧 focused tests。
|
||||
- [binding 关联不精确导致证据泄漏] → 使用四元键精确匹配并覆盖 mixed pass/fail binding。
|
||||
- [Gatekeeper 双执行] → Graph Verifier 不注册旧 Hook，并以调用次数测试锁定。
|
||||
- [异常误判为可重试] → 未知异常默认 fail closed；分类器用显式用例覆盖。
|
||||
- [阶段 2 Prompt 与 Node payload 说明暂不一致] → 本阶段不接生产入口；阶段 3 切换时同一 change 更新 Prompt。
|
||||
|
||||
## Migration Plan
|
||||
|
||||
1. 先抽共享协议组件并让旧 Hook/ChatService 委托，运行旧 focused tests。
|
||||
2. 接入 invoker、Agent adapters、Gatekeeper、Verified Input、Retry Prepare 和 Fallback Nodes。
|
||||
3. 修正 Router critical-gap guard，装配未接生产入口的真实 Graph 并运行 Node/Graph tests。
|
||||
4. 归档并提交本阶段；阶段 3 再切换 ChatService、Prompt、DB 和 Trace。
|
||||
|
||||
回滚为完整 revert 本阶段提交；不需要数据库回滚，不存在外部协议迁移。
|
||||
|
||||
## Open Questions
|
||||
|
||||
无。阶段 3 前不得扩大范围到生产入口或 Prompt 切换。
|
||||
@@ -0,0 +1,82 @@
|
||||
# Chat Diagnosis StateGraph Real Nodes
|
||||
|
||||
## Why
|
||||
|
||||
阶段 1 已提供可编译的 StateGraph 骨架和 35 个 Fake Node 路由测试，但所有 action ports 仍是测试脚本，尚不能调用现有 ReactAgent、Executor Gatekeeper 或构造安全的 Verifier/Composer 输入。阶段 2 需要把现有 Agent 协议和确定性服务接到 Graph ports，同时保持 ChatService 生产入口仍走旧链路，避免把真实 Node 接入与生产切换混成一个不可回滚阶段。
|
||||
|
||||
## What Changes
|
||||
|
||||
- 新增 `DiagnosisAgentInvoker` 及 ReactAgent adapter，所有 Agent Node 显式传递 `RunnableConfig`。
|
||||
- 新增 Planner、Executor、Verifier、Composer Node Adapters 和可注入失败分类器。
|
||||
- 抽取 Executor/Verifier/Composer 共享解析组件，旧 Hook/ChatService 委托它们，避免复制协议逻辑。
|
||||
- 新增显式 Gatekeeper Node，调用 `ExecutorGatekeeperService.validateRun` 并 fail-closed 标准化状态。
|
||||
- 新增 Verified Input Builder，只投影 Gatekeeper passed bindings 对应的 claims 与 `matched_text` evidence。
|
||||
- 新增 Evidence Gap Extractor / Retry Prepare Node，仅处理关键 `no_evidence` / `indirect_support` facts。
|
||||
- 新增 Composer Safe Input Builder 和两类固定 Fallback Node 表达。
|
||||
- 修正阶段 1 Router：evidence retry guard 必须要求关键 evidence gap。
|
||||
- 新增 Node 契约和真实 CompiledGraph 集成测试，并保留旧 Sequential/Hook/Gatekeeper 回归。
|
||||
|
||||
本 change 不切换 ChatService 到 StateGraph，不修改 DB/Trace API，也不删除旧 Hook/Sequential 结构。
|
||||
|
||||
## Capabilities
|
||||
|
||||
### New Capabilities
|
||||
|
||||
- `chat-diagnosis-stategraph-real-nodes`：提供真实 Agent/Java Node adapters、可信输入投影、关键证据补查上下文和安全 Fallback。
|
||||
|
||||
### Modified Capabilities
|
||||
|
||||
- `chat-diagnosis-stategraph-routing-skeleton`：将 LOW_CONFID evidence retry guard 收紧为仅接受 `is_critical=true` 的 `no_evidence` 或 `indirect_support` facts，与阶段 0 冻结基线一致。
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- `com.superbiz.agent.graph.diagnosis` 下的 Agent adapters、Gatekeeper、Verified Input、Evidence Retry、Composer Input 和 Fallback。
|
||||
- 可被旧 Hook/ChatService 与新 Nodes 共用的输出解析/安全渲染组件。
|
||||
- 旧 Hook/ChatService 的行为保持型委托重构。
|
||||
- `DiagnosisGraphRouter` 的 critical gap guard 修正。
|
||||
- Node 单元测试、真实 CompiledGraph adapter tests、旧路径 focused regressions。
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- 将 `ChatService.executeChatComplex` 替换为 Graph invocation。
|
||||
- 修改 Agent 基础 Prompt 业务语义或输出协议。
|
||||
- 移除 `VerifierInputHook` / `VerifierContextHolder` / SequentialAgent。
|
||||
- Flyway、DiagnosisRun orchestration trace 字段、Trace DTO/API。
|
||||
- 最终全量 Graph 测试体系替换。
|
||||
- Maven E2E、`logs/` 和数据库核验。
|
||||
|
||||
## Context Constraints
|
||||
|
||||
- 阶段 0/1 archives 是执行基线；Node 不得改变条件边和计数所有权。
|
||||
- Graph Verifier 只接收 `verified_executor_output`、`verified_evidence`、gatekeeper audit/ceiling、query 和 retry context，不接收完整 tool trace 或 Executor raw text。
|
||||
- Gatekeeper validation 使用当前 `runId`，缺失/未知/异常一律 REJECT。
|
||||
- Gatekeeper raw result 与 normalized status 分开保存。
|
||||
- Verifier execution status、model verdict 和 effective verdict 分离；status 永远不写进 verdict。
|
||||
- Planner/Verifier/Composer retry 输入由相同 state 白名单投影构造，技术 retry 不重跑前序节点。
|
||||
- Executor 不重试；合法 no-evidence 仍为 COMPLETED。
|
||||
- Evidence Retry Context 保留 prior verified output/evidence、关键 gaps、completed query refs 和固定约束；Java 不合并 claim 文本。
|
||||
- 旧 Sequential path 在阶段 3 前继续工作；共享解析器重构必须通过旧 focused tests。
|
||||
- 本阶段不修改 `/api/chat`、DB 或当前运行路由。
|
||||
|
||||
## Acceptance
|
||||
|
||||
- ReactAgent invoker 透传输入与 RunnableConfig，现有 Agent Hook/ToolCallback 可继续工作。
|
||||
- Planner 合法 JSON 为 COMPLETED；非法 JSON/结构为 INVALID_OUTPUT；临时失败与不可重试失败分开。
|
||||
- Executor 合法 `executor_evidence_v2`（包括 no-evidence）为 COMPLETED；非法结构、工具阻断、其他失败正确映射且不重试。
|
||||
- Gatekeeper Node 每轮只调用一次 `validateRun`，unknown/error fail closed。
|
||||
- PASS 与可继续 LOW_CONFID 都经过 Verified Input Builder；失败 binding、未引用工具结果和 raw Executor text 不进入 Verifier。
|
||||
- Verifier input 只有白名单字段；输出分离 status/model/effective verdict，ceiling 正确限制 PASS。
|
||||
- Verifier/Composer technical retry 使用相同序列化输入。
|
||||
- Evidence retry 只从关键 gap 生成，包含 prior verified data/completed refs/约束，最多一次。
|
||||
- 第二轮 Executor 输入明确要求增量查询和完整 `executor_evidence_v2` 快照，不由 Java 合并 claims。
|
||||
- Pre-verification Fallback 不输出 Executor claim；post-verification Composer fallback 只使用 allowed material。
|
||||
- 新 Node/Graph tests 和旧路径 focused regressions 通过；不运行 Maven E2E。
|
||||
|
||||
## Risks
|
||||
|
||||
- 共享解析器抽取可能改变旧 Sequential 行为；旧 ChatService/Hook tests 必须同批通过。
|
||||
- Gatekeeper checked binding 与 claim binding 关联错误可能泄漏失败 evidence；使用 claim_id + invocation/tool/path 精确匹配并测试 mixed binding。
|
||||
- 异常分类依赖 cause/message；未知异常默认不可重试或 FAILED，安全优先。
|
||||
- Prompt 当前仍描述旧 Hook payload；本阶段只验证 Node input contract，阶段 3 在生产切换时同步 Verifier Prompt 输入说明，避免旧生产路径提前不兼容。
|
||||
+132
@@ -0,0 +1,132 @@
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Agent adapters SHALL invoke real agents through explicit run config
|
||||
|
||||
The system SHALL provide Planner, Executor, Verifier, and Composer adapters that invoke their configured ReactAgent through a minimal invoker port with an explicit `RunnableConfig`. Each adapter SHALL serialize only its declared input fields and append one terminal orchestration event per attempt.
|
||||
|
||||
#### Scenario: Adapter invokes an agent
|
||||
|
||||
- **WHEN** an adapter receives valid Graph State and a RunnableConfig containing the current run metadata
|
||||
- **THEN** it SHALL pass its projected input and the same RunnableConfig to the configured invoker
|
||||
- **AND** existing Agent hooks and ToolCallbacks SHALL be able to observe the current run metadata
|
||||
|
||||
#### Scenario: Parent state is inspected
|
||||
|
||||
- **WHEN** an adapter builds an Agent input
|
||||
- **THEN** it SHALL NOT serialize undeclared Graph State, raw prompts, other Agent private state, or orchestration events
|
||||
|
||||
### Requirement: Agent outputs SHALL be parsed into explicit execution statuses
|
||||
|
||||
The system SHALL use shared Executor, Verifier, and Composer protocol parsers for both Graph Nodes and the legacy path. Legal structured output SHALL map to COMPLETED; invalid JSON or contract shape SHALL map to INVALID_OUTPUT; recognized transient invocation failure SHALL map to RETRYABLE_FAILED where that Agent supports technical retry; unknown or permanent failure SHALL fail closed. A legal Executor no-evidence result SHALL be COMPLETED.
|
||||
|
||||
#### Scenario: Legal no-evidence Executor output
|
||||
|
||||
- **WHEN** Executor returns a valid `executor_evidence_v2` document containing a legal no-evidence result
|
||||
- **THEN** Executor status SHALL be COMPLETED
|
||||
- **AND** the Graph SHALL continue to Gatekeeper
|
||||
|
||||
#### Scenario: Invalid structured output
|
||||
|
||||
- **WHEN** an Agent returns malformed JSON or violates its required output structure
|
||||
- **THEN** its adapter SHALL set INVALID_OUTPUT
|
||||
- **AND** it SHALL NOT fabricate a diagnostic verdict or evidence
|
||||
|
||||
#### Scenario: Legacy parser behavior is exercised
|
||||
|
||||
- **WHEN** the existing Sequential path parses the same payloads after shared component extraction
|
||||
- **THEN** its externally observable parser and safe-rendering behavior SHALL remain unchanged
|
||||
|
||||
### Requirement: Gatekeeper Node SHALL validate exactly once and fail closed
|
||||
|
||||
The Gatekeeper Node SHALL call `ExecutorGatekeeperService.validateRun` exactly once for the current run and Executor structured output. It SHALL preserve the raw result separately from normalized status. Raw pass SHALL normalize to PASS; fail with low-confid severity SHALL normalize to LOW_CONFID; fail with reject severity SHALL normalize to REJECT; missing, unknown, inconsistent, or exceptional results SHALL normalize to REJECT.
|
||||
|
||||
#### Scenario: Gatekeeper passes output
|
||||
|
||||
- **WHEN** `validateRun` returns a valid pass result
|
||||
- **THEN** the Node SHALL store the raw result and status PASS
|
||||
- **AND** validation SHALL have been called exactly once with the current runId
|
||||
|
||||
#### Scenario: Gatekeeper result cannot be trusted
|
||||
|
||||
- **WHEN** validation throws or returns a missing, unknown, or inconsistent result
|
||||
- **THEN** the Node SHALL normalize status to REJECT
|
||||
- **AND** Verifier SHALL NOT receive unverified Executor material
|
||||
|
||||
### Requirement: Verified Input Builder SHALL project only passed bindings
|
||||
|
||||
PASS and continuable LOW_CONFID results SHALL pass through a Verified Input Builder. The Builder SHALL match passed checked bindings to Executor claims by `claim_id`, `source_invocation_id`, `tool_name`, and `raw_path`, and SHALL produce only filtered `verified_executor_output` plus `verified_evidence` entries containing the matched binding fields and `matched_text`.
|
||||
|
||||
#### Scenario: Mixed checked bindings are projected
|
||||
|
||||
- **WHEN** Gatekeeper returns both passed and failed checked bindings
|
||||
- **THEN** only claims and evidence matching passed bindings SHALL be projected
|
||||
- **AND** failed bindings, hypotheses, unreferenced tool results, and raw Executor text SHALL be absent
|
||||
|
||||
#### Scenario: Verifier input is serialized
|
||||
|
||||
- **WHEN** the Verifier adapter builds its input
|
||||
- **THEN** it SHALL include only diagnosis query context, verified Executor output, verified evidence, Gatekeeper audit/ceiling, and permitted retry context
|
||||
- **AND** it SHALL NOT include complete `tool_trace_summary` or raw Executor output
|
||||
|
||||
### Requirement: Verifier SHALL separate execution status from diagnostic verdict
|
||||
|
||||
The Verifier adapter SHALL store `verifier_status`, `verifier_model_verdict`, and `effective_verdict` as separate values. A Gatekeeper LOW_CONFID ceiling SHALL prevent model PASS from producing effective PASS. Technical retry SHALL reuse the exact same serialized verified input and SHALL NOT rerun any preceding Node.
|
||||
|
||||
#### Scenario: Ceiling limits model verdict
|
||||
|
||||
- **WHEN** Gatekeeper ceiling is LOW_CONFID and the model verdict is PASS
|
||||
- **THEN** effective verdict SHALL be LOW_CONFID
|
||||
- **AND** verifier execution status SHALL remain COMPLETED
|
||||
|
||||
#### Scenario: Verifier technical retry occurs
|
||||
|
||||
- **WHEN** the first Verifier attempt returns INVALID_OUTPUT or RETRYABLE_FAILED
|
||||
- **THEN** its single retry SHALL receive the same serialized input
|
||||
- **AND** Executor, Gatekeeper, Verified Input, and tools SHALL NOT rerun
|
||||
|
||||
### Requirement: Evidence retry SHALL contain only structured critical gaps and incremental constraints
|
||||
|
||||
The system SHALL extract evidence gaps only from facts with `is_critical=true` and verification `no_evidence` or `indirect_support`. Retry Prepare SHALL include prior verified output/evidence, structured gaps, deduplicated completed query references, and fixed constraints requiring at most one incremental retry without repeating successful queries. The second Executor invocation SHALL be instructed to return a complete `executor_evidence_v2` snapshot; Java code SHALL NOT merge claim text.
|
||||
|
||||
#### Scenario: Critical gaps prepare a retry
|
||||
|
||||
- **WHEN** effective verdict is LOW_CONFID, ceiling is PASS, evidence retry count is zero, and at least one qualifying critical gap exists
|
||||
- **THEN** Retry Prepare SHALL build the bounded retry context and increment evidence retry count once
|
||||
- **AND** the new Planner stage SHALL use EVIDENCE_GAP_ONLY mode
|
||||
|
||||
#### Scenario: Non-critical gap is present
|
||||
|
||||
- **WHEN** facts contain only non-critical no-evidence or indirect-support items
|
||||
- **THEN** no evidence retry SHALL occur
|
||||
- **AND** the Graph SHALL continue to Composer
|
||||
|
||||
#### Scenario: Second Executor input is built
|
||||
|
||||
- **WHEN** Planner produces an evidence-gap-only incremental plan
|
||||
- **THEN** Executor input SHALL prohibit repeating completed queries and require a complete output snapshot preserving prior verified claims
|
||||
- **AND** the Java layer SHALL NOT semantically merge old and new claims
|
||||
|
||||
### Requirement: Composer and Fallback SHALL use only allowed material
|
||||
|
||||
Composer SHALL receive only effective verdict and Verifier-allowed claims, missing information, and recommendations. Composer technical retry SHALL reuse the exact same serialized input. A pre-verification Fallback SHALL never output Executor claims; a post-verification Composer Fallback SHALL use only Verifier-allowed material.
|
||||
|
||||
#### Scenario: Pre-verification path degrades
|
||||
|
||||
- **WHEN** Planner, Executor, Gatekeeper, Verified Input, or Verifier cannot establish trusted material
|
||||
- **THEN** deterministic Fallback output SHALL contain no Executor claim or raw tool output
|
||||
|
||||
#### Scenario: Composer retry is exhausted
|
||||
|
||||
- **WHEN** Composer fails after its one technical retry and Verifier-allowed material exists
|
||||
- **THEN** deterministic Fallback SHALL express only the allowed claims, missing information, and recommendations
|
||||
- **AND** it SHALL NOT read raw Executor or tool output
|
||||
|
||||
### Requirement: Real Nodes SHALL remain isolated from the production Chat path in stage 2
|
||||
|
||||
The real Node action set and CompiledGraph SHALL be constructible and testable, but ChatService, database, Trace API, shared Agent prompts, and the current production routing SHALL remain unchanged until the stage 3 change.
|
||||
|
||||
#### Scenario: Stage 2 production isolation is inspected
|
||||
|
||||
- **WHEN** this change is accepted
|
||||
- **THEN** no production ChatService code path SHALL invoke the real Diagnosis Graph
|
||||
- **AND** no database migration, Trace API field, or shared Prompt contract SHALL be changed
|
||||
+33
@@ -0,0 +1,33 @@
|
||||
## MODIFIED Requirements
|
||||
|
||||
### Requirement: Verifier routing SHALL separate technical retry from evidence retry
|
||||
|
||||
Verifier INVALID_OUTPUT and RETRYABLE_FAILED SHALL self-retry once with the same verified input. COMPLETED PASS or REJECT SHALL route to Composer. COMPLETED LOW_CONFID SHALL route to one Evidence Retry only when all frozen guards are true, including at least one critical evidence gap; otherwise it SHALL route to Composer. Other outcomes SHALL fail closed.
|
||||
|
||||
#### Scenario: Verifier first technical failure
|
||||
|
||||
- **WHEN** Verifier first returns INVALID_OUTPUT or RETRYABLE_FAILED
|
||||
- **THEN** only Verifier SHALL run again
|
||||
- **AND** Gatekeeper, Executor, and tools SHALL NOT rerun
|
||||
|
||||
#### Scenario: Verifier retry is exhausted
|
||||
|
||||
- **WHEN** Verifier returns a technical failure after `verifier_retry_count=1`
|
||||
- **THEN** the Graph SHALL route to Fallback
|
||||
|
||||
#### Scenario: Verifier verdict reaches Composer
|
||||
|
||||
- **WHEN** Verifier completes with effective PASS or REJECT
|
||||
- **THEN** Composer SHALL run
|
||||
|
||||
#### Scenario: LOW_CONFID qualifies for evidence retry
|
||||
|
||||
- **WHEN** Verifier completes LOW_CONFID with ceiling PASS, at least one fact having `is_critical=true` and verification `no_evidence` or `indirect_support`, and `evidence_retry_count=0`
|
||||
- **THEN** Evidence Retry SHALL run once and return to a new Planner stage
|
||||
- **AND** `planner_retry_count` SHALL reset to 0
|
||||
- **AND** `evidence_retry_count` SHALL become 1
|
||||
|
||||
#### Scenario: LOW_CONFID does not qualify for evidence retry
|
||||
|
||||
- **WHEN** ceiling is LOW_CONFID, facts contain no critical valid gap, or evidence retry count is already 1
|
||||
- **THEN** Composer SHALL run without another Planner cycle
|
||||
@@ -0,0 +1,44 @@
|
||||
## 1. Shared Protocol Components
|
||||
|
||||
- [x] 1.1 Extract stateless JSON payload and Executor evidence parsing components into neutral `com.superbiz.agent.diagnosis.protocol`, preserving legal no-evidence behavior and introducing no dependency on Graph, Hook, ChatService, or ThreadLocal.
|
||||
- [x] 1.2 Extract Verifier and Composer output parsers, effective-verdict calculation, Composer safe-input construction, and deterministic safe rendering from ChatService.
|
||||
- [x] 1.3 Make VerifierInputHook and ChatService delegate to the shared components without changing their public methods or legacy payload/output behavior.
|
||||
- [x] 1.4 Run `VerifierInputHookTest`, `ChatServiceSequentialAgentTest`, and `ExecutorGatekeeperServiceTest` after the extraction and fix only behavior regressions in scope.
|
||||
|
||||
## 2. Agent Invocation And Adapters
|
||||
|
||||
- [x] 2.1 Implement constructor-injected `DiagnosisAgentInvoker` and a ReactAgent wrapper that forwards the exact input and RunnableConfig and returns AssistantMessage text, without adding a second Prompt/Agent factory.
|
||||
- [x] 2.2 Implement a fail-closed, injectable Node failure classifier for invalid output, retryable invocation failure, and non-retryable failure.
|
||||
- [x] 2.3 Implement Planner adapter with NORMAL/EVIDENCE_GAP_ONLY input projection, structured plan parsing, status mapping, and one event per attempt.
|
||||
- [x] 2.4 Implement Executor adapter with incremental retry instructions, legal no-evidence completion, no technical retry, and complete-snapshot output parsing.
|
||||
- [x] 2.5 Add focused adapter tests proving input whitelists, RunnableConfig identity, failure classification, event shape, legal no-evidence handling, and absence of parent-state leakage.
|
||||
|
||||
## 3. Gatekeeper And Verified Projection
|
||||
|
||||
- [x] 3.1 Implement the explicit Gatekeeper Node using current runId and `ExecutorGatekeeperService.validateRun`, preserving raw result separately from normalized status.
|
||||
- [x] 3.2 Normalize missing, unknown, inconsistent, and exceptional Gatekeeper results to REJECT and record a deterministic reason code.
|
||||
- [x] 3.3 Implement Verified Input Builder matching passed bindings by claim/invocation/tool/path and projecting only filtered claims plus matched-text evidence.
|
||||
- [x] 3.4 Add mixed-binding tests proving one Gatekeeper call, correct PASS/LOW_CONFID/REJECT normalization, precise pass projection, and exclusion of failed, unreferenced, hypothesis, tool-summary, and raw materials.
|
||||
|
||||
## 4. Verifier, Evidence Retry, Composer And Fallback
|
||||
|
||||
- [x] 4.1 Implement Verifier adapter with whitelisted verified input, separate execution/model/effective fields, ceiling enforcement, and byte-identical technical retry input.
|
||||
- [x] 4.2 Implement shared `EvidenceGapExtractor` and update `DiagnosisGraphRouter` so only critical no-evidence/indirect-support facts qualify.
|
||||
- [x] 4.3 Implement Evidence Retry Prepare Node with prior verified data, structured gaps, deduplicated completed query refs, bounded constraints, and planner retry reset.
|
||||
- [x] 4.4 Implement Composer adapter and safe-input builder with byte-identical technical retry input and no raw Executor/tool access.
|
||||
- [x] 4.5 Implement distinct deterministic pre-verification and post-verification Fallback inputs/rendering with their allowed-material boundaries.
|
||||
- [x] 4.6 Add focused tests for ceiling, fixed retry inputs, non-critical gap rejection, critical gap context, no Java claim merge, and both Fallback safety boundaries.
|
||||
|
||||
## 5. Real Graph Assembly
|
||||
|
||||
- [x] 5.1 Extend the diagnosis action set/factory to construct a CompiledGraph from the real Agent and deterministic Node dependencies without registering legacy VerifierInputHook on the Graph Verifier.
|
||||
- [x] 5.2 Add CompiledGraph integration tests for PASS, legal no-evidence, Gatekeeper REJECT/LOW_CONFID, critical evidence retry, exhausted Agent technical retry, and Composer fallback paths.
|
||||
- [x] 5.3 Verify Graph events/transitions, per-node invocation counts, same-input retries, one full-snapshot Gatekeeper revalidation after evidence retry, and bounded termination.
|
||||
- [x] 5.4 Confirm the first completed Node module and final assembly against design/specs, with no core TODO or placeholder implementation.
|
||||
|
||||
## 6. Stage 2 Verification And Handoff
|
||||
|
||||
- [x] 6.1 Run new Node/Graph focused tests plus `DiagnosisGraphRoutingTest` and `DiagnosisOrchestrationTraceBuilderTest`.
|
||||
- [x] 6.2 Run legacy focused regressions `ChatServiceSequentialAgentTest`, `VerifierInputHookTest`, and `ExecutorGatekeeperServiceTest`, then run Maven test compilation.
|
||||
- [x] 6.3 Run OpenSpec strict validation, `git diff --check`, and source/reference checks proving ChatService does not invoke the real Graph and no DB/Trace/Prompt contract changed.
|
||||
- [x] 6.4 Record that Maven live E2E, `logs/`, and database verification were intentionally not run in stage 2 and are reserved for stage 5.
|
||||
@@ -0,0 +1,135 @@
|
||||
# chat-diagnosis-stategraph-real-nodes Specification
|
||||
|
||||
## Purpose
|
||||
TBD - created by archiving change chat-diagnosis-stategraph-real-nodes. Update Purpose after archive.
|
||||
## Requirements
|
||||
### Requirement: Agent adapters SHALL invoke real agents through explicit run config
|
||||
|
||||
The system SHALL provide Planner, Executor, Verifier, and Composer adapters that invoke their configured ReactAgent through a minimal invoker port with an explicit `RunnableConfig`. Each adapter SHALL serialize only its declared input fields and append one terminal orchestration event per attempt.
|
||||
|
||||
#### Scenario: Adapter invokes an agent
|
||||
|
||||
- **WHEN** an adapter receives valid Graph State and a RunnableConfig containing the current run metadata
|
||||
- **THEN** it SHALL pass its projected input and the same RunnableConfig to the configured invoker
|
||||
- **AND** existing Agent hooks and ToolCallbacks SHALL be able to observe the current run metadata
|
||||
|
||||
#### Scenario: Parent state is inspected
|
||||
|
||||
- **WHEN** an adapter builds an Agent input
|
||||
- **THEN** it SHALL NOT serialize undeclared Graph State, raw prompts, other Agent private state, or orchestration events
|
||||
|
||||
### Requirement: Agent outputs SHALL be parsed into explicit execution statuses
|
||||
|
||||
The system SHALL use shared Executor, Verifier, and Composer protocol parsers for both Graph Nodes and the legacy path. Legal structured output SHALL map to COMPLETED; invalid JSON or contract shape SHALL map to INVALID_OUTPUT; recognized transient invocation failure SHALL map to RETRYABLE_FAILED where that Agent supports technical retry; unknown or permanent failure SHALL fail closed. A legal Executor no-evidence result SHALL be COMPLETED.
|
||||
|
||||
#### Scenario: Legal no-evidence Executor output
|
||||
|
||||
- **WHEN** Executor returns a valid `executor_evidence_v2` document containing a legal no-evidence result
|
||||
- **THEN** Executor status SHALL be COMPLETED
|
||||
- **AND** the Graph SHALL continue to Gatekeeper
|
||||
|
||||
#### Scenario: Invalid structured output
|
||||
|
||||
- **WHEN** an Agent returns malformed JSON or violates its required output structure
|
||||
- **THEN** its adapter SHALL set INVALID_OUTPUT
|
||||
- **AND** it SHALL NOT fabricate a diagnostic verdict or evidence
|
||||
|
||||
#### Scenario: Legacy parser behavior is exercised
|
||||
|
||||
- **WHEN** the existing Sequential path parses the same payloads after shared component extraction
|
||||
- **THEN** its externally observable parser and safe-rendering behavior SHALL remain unchanged
|
||||
|
||||
### Requirement: Gatekeeper Node SHALL validate exactly once and fail closed
|
||||
|
||||
The Gatekeeper Node SHALL call `ExecutorGatekeeperService.validateRun` exactly once for the current run and Executor structured output. It SHALL preserve the raw result separately from normalized status. Raw pass SHALL normalize to PASS; fail with low-confid severity SHALL normalize to LOW_CONFID; fail with reject severity SHALL normalize to REJECT; missing, unknown, inconsistent, or exceptional results SHALL normalize to REJECT.
|
||||
|
||||
#### Scenario: Gatekeeper passes output
|
||||
|
||||
- **WHEN** `validateRun` returns a valid pass result
|
||||
- **THEN** the Node SHALL store the raw result and status PASS
|
||||
- **AND** validation SHALL have been called exactly once with the current runId
|
||||
|
||||
#### Scenario: Gatekeeper result cannot be trusted
|
||||
|
||||
- **WHEN** validation throws or returns a missing, unknown, or inconsistent result
|
||||
- **THEN** the Node SHALL normalize status to REJECT
|
||||
- **AND** Verifier SHALL NOT receive unverified Executor material
|
||||
|
||||
### Requirement: Verified Input Builder SHALL project only passed bindings
|
||||
|
||||
PASS and continuable LOW_CONFID results SHALL pass through a Verified Input Builder. The Builder SHALL match passed checked bindings to Executor claims by `claim_id`, `source_invocation_id`, `tool_name`, and `raw_path`, and SHALL produce only filtered `verified_executor_output` plus `verified_evidence` entries containing the matched binding fields and `matched_text`.
|
||||
|
||||
#### Scenario: Mixed checked bindings are projected
|
||||
|
||||
- **WHEN** Gatekeeper returns both passed and failed checked bindings
|
||||
- **THEN** only claims and evidence matching passed bindings SHALL be projected
|
||||
- **AND** failed bindings, hypotheses, unreferenced tool results, and raw Executor text SHALL be absent
|
||||
|
||||
#### Scenario: Verifier input is serialized
|
||||
|
||||
- **WHEN** the Verifier adapter builds its input
|
||||
- **THEN** it SHALL include only diagnosis query context, verified Executor output, verified evidence, Gatekeeper audit/ceiling, and permitted retry context
|
||||
- **AND** it SHALL NOT include complete `tool_trace_summary` or raw Executor output
|
||||
|
||||
### Requirement: Verifier SHALL separate execution status from diagnostic verdict
|
||||
|
||||
The Verifier adapter SHALL store `verifier_status`, `verifier_model_verdict`, and `effective_verdict` as separate values. A Gatekeeper LOW_CONFID ceiling SHALL prevent model PASS from producing effective PASS. Technical retry SHALL reuse the exact same serialized verified input and SHALL NOT rerun any preceding Node.
|
||||
|
||||
#### Scenario: Ceiling limits model verdict
|
||||
|
||||
- **WHEN** Gatekeeper ceiling is LOW_CONFID and the model verdict is PASS
|
||||
- **THEN** effective verdict SHALL be LOW_CONFID
|
||||
- **AND** verifier execution status SHALL remain COMPLETED
|
||||
|
||||
#### Scenario: Verifier technical retry occurs
|
||||
|
||||
- **WHEN** the first Verifier attempt returns INVALID_OUTPUT or RETRYABLE_FAILED
|
||||
- **THEN** its single retry SHALL receive the same serialized input
|
||||
- **AND** Executor, Gatekeeper, Verified Input, and tools SHALL NOT rerun
|
||||
|
||||
### Requirement: Evidence retry SHALL contain only structured critical gaps and incremental constraints
|
||||
|
||||
The system SHALL extract evidence gaps only from facts with `is_critical=true` and verification `no_evidence` or `indirect_support`. Retry Prepare SHALL include prior verified output/evidence, structured gaps, deduplicated completed query references, and fixed constraints requiring at most one incremental retry without repeating successful queries. The second Executor invocation SHALL be instructed to return a complete `executor_evidence_v2` snapshot; Java code SHALL NOT merge claim text.
|
||||
|
||||
#### Scenario: Critical gaps prepare a retry
|
||||
|
||||
- **WHEN** effective verdict is LOW_CONFID, ceiling is PASS, evidence retry count is zero, and at least one qualifying critical gap exists
|
||||
- **THEN** Retry Prepare SHALL build the bounded retry context and increment evidence retry count once
|
||||
- **AND** the new Planner stage SHALL use EVIDENCE_GAP_ONLY mode
|
||||
|
||||
#### Scenario: Non-critical gap is present
|
||||
|
||||
- **WHEN** facts contain only non-critical no-evidence or indirect-support items
|
||||
- **THEN** no evidence retry SHALL occur
|
||||
- **AND** the Graph SHALL continue to Composer
|
||||
|
||||
#### Scenario: Second Executor input is built
|
||||
|
||||
- **WHEN** Planner produces an evidence-gap-only incremental plan
|
||||
- **THEN** Executor input SHALL prohibit repeating completed queries and require a complete output snapshot preserving prior verified claims
|
||||
- **AND** the Java layer SHALL NOT semantically merge old and new claims
|
||||
|
||||
### Requirement: Composer and Fallback SHALL use only allowed material
|
||||
|
||||
Composer SHALL receive only effective verdict and Verifier-allowed claims, missing information, and recommendations. Composer technical retry SHALL reuse the exact same serialized input. A pre-verification Fallback SHALL never output Executor claims; a post-verification Composer Fallback SHALL use only Verifier-allowed material.
|
||||
|
||||
#### Scenario: Pre-verification path degrades
|
||||
|
||||
- **WHEN** Planner, Executor, Gatekeeper, Verified Input, or Verifier cannot establish trusted material
|
||||
- **THEN** deterministic Fallback output SHALL contain no Executor claim or raw tool output
|
||||
|
||||
#### Scenario: Composer retry is exhausted
|
||||
|
||||
- **WHEN** Composer fails after its one technical retry and Verifier-allowed material exists
|
||||
- **THEN** deterministic Fallback SHALL express only the allowed claims, missing information, and recommendations
|
||||
- **AND** it SHALL NOT read raw Executor or tool output
|
||||
|
||||
### Requirement: Real Nodes SHALL remain isolated from the production Chat path in stage 2
|
||||
|
||||
The real Node action set and CompiledGraph SHALL be constructible and testable, but ChatService, database, Trace API, shared Agent prompts, and the current production routing SHALL remain unchanged until the stage 3 change.
|
||||
|
||||
#### Scenario: Stage 2 production isolation is inspected
|
||||
|
||||
- **WHEN** this change is accepted
|
||||
- **THEN** no production ChatService code path SHALL invoke the real Diagnosis Graph
|
||||
- **AND** no database migration, Trace API field, or shared Prompt contract SHALL be changed
|
||||
@@ -74,7 +74,7 @@ Executor SHALL route only COMPLETED output to Gatekeeper and SHALL never retry.
|
||||
|
||||
### Requirement: Verifier routing SHALL separate technical retry from evidence retry
|
||||
|
||||
Verifier INVALID_OUTPUT and RETRYABLE_FAILED SHALL self-retry once with the same verified input. COMPLETED PASS or REJECT SHALL route to Composer. COMPLETED LOW_CONFID SHALL route to one Evidence Retry only when all frozen guards are true; otherwise it SHALL route to Composer. Other outcomes SHALL fail closed.
|
||||
Verifier INVALID_OUTPUT and RETRYABLE_FAILED SHALL self-retry once with the same verified input. COMPLETED PASS or REJECT SHALL route to Composer. COMPLETED LOW_CONFID SHALL route to one Evidence Retry only when all frozen guards are true, including at least one critical evidence gap; otherwise it SHALL route to Composer. Other outcomes SHALL fail closed.
|
||||
|
||||
#### Scenario: Verifier first technical failure
|
||||
|
||||
@@ -94,14 +94,14 @@ Verifier INVALID_OUTPUT and RETRYABLE_FAILED SHALL self-retry once with the same
|
||||
|
||||
#### Scenario: LOW_CONFID qualifies for evidence retry
|
||||
|
||||
- **WHEN** Verifier completes LOW_CONFID with ceiling PASS, valid no_evidence or indirect_support facts, and `evidence_retry_count=0`
|
||||
- **WHEN** Verifier completes LOW_CONFID with ceiling PASS, at least one fact having `is_critical=true` and verification `no_evidence` or `indirect_support`, and `evidence_retry_count=0`
|
||||
- **THEN** Evidence Retry SHALL run once and return to a new Planner stage
|
||||
- **AND** `planner_retry_count` SHALL reset to 0
|
||||
- **AND** `evidence_retry_count` SHALL become 1
|
||||
|
||||
#### Scenario: LOW_CONFID does not qualify for evidence retry
|
||||
|
||||
- **WHEN** ceiling is LOW_CONFID, facts contain no valid gap, or evidence retry count is already 1
|
||||
- **WHEN** ceiling is LOW_CONFID, facts contain no critical valid gap, or evidence retry count is already 1
|
||||
- **THEN** Composer SHALL run without another Planner cycle
|
||||
|
||||
### Requirement: Composer routing SHALL allow one technical retry and then terminate safely
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
public final class ComposerOutputParser {
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
private final SafeFallbackRenderer fallbackRenderer;
|
||||
|
||||
public ComposerOutputParser() {
|
||||
this(new ObjectMapper(), new SafeFallbackRenderer());
|
||||
}
|
||||
|
||||
public ComposerOutputParser(
|
||||
ObjectMapper objectMapper,
|
||||
SafeFallbackRenderer fallbackRenderer) {
|
||||
this.objectMapper = Objects.requireNonNull(objectMapper, "objectMapper");
|
||||
this.fallbackRenderer = Objects.requireNonNull(
|
||||
fallbackRenderer, "fallbackRenderer");
|
||||
}
|
||||
|
||||
public ComposerRenderResult parse(
|
||||
String rawOutput,
|
||||
Map<String, Object> composerInput,
|
||||
List<String> defaultSuggestions) {
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(
|
||||
JsonPayloadSupport.sanitizeObject(rawOutput));
|
||||
String answerSummary = root.path("answer_summary").asText("");
|
||||
String userFacingAnswer = root.path("user_facing_answer").asText("");
|
||||
if (answerSummary.isBlank()
|
||||
|| userFacingAnswer.isBlank()
|
||||
|| !root.path("recommended_actions").isArray()) {
|
||||
return fallback(composerInput, defaultSuggestions,
|
||||
"composer_schema_invalid");
|
||||
}
|
||||
Map<String, Object> audit = new LinkedHashMap<>();
|
||||
audit.put("status", "valid");
|
||||
audit.put("answer_summary", answerSummary);
|
||||
audit.put("recommended_actions", parseActions(
|
||||
root.path("recommended_actions")));
|
||||
audit.put("user_facing_answer", userFacingAnswer);
|
||||
return new ComposerRenderResult(userFacingAnswer, audit);
|
||||
} catch (Exception ignored) {
|
||||
return fallback(composerInput, defaultSuggestions, "composer_malformed");
|
||||
}
|
||||
}
|
||||
|
||||
public ComposerRenderResult fallback(
|
||||
Map<String, Object> composerInput,
|
||||
List<String> defaultSuggestions,
|
||||
String status) {
|
||||
String answer = fallbackRenderer.render(composerInput, defaultSuggestions);
|
||||
Map<String, Object> audit = new LinkedHashMap<>();
|
||||
audit.put("status", status);
|
||||
audit.put("detail", "used safe fallback rendering");
|
||||
audit.put("answer_summary", firstSentence(answer));
|
||||
audit.put("recommended_actions", composerInput == null
|
||||
? List.of()
|
||||
: composerInput.getOrDefault("recommended_actions", List.of()));
|
||||
audit.put("user_facing_answer", answer);
|
||||
return new ComposerRenderResult(answer, audit);
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseActions(JsonNode node) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
if (!node.isArray()) {
|
||||
return result;
|
||||
}
|
||||
for (JsonNode actionNode : node) {
|
||||
Map<String, Object> action = new LinkedHashMap<>();
|
||||
action.put("action_text", actionNode.path("action_text").asText(""));
|
||||
action.put("reason", actionNode.path("reason").asText(""));
|
||||
result.add(action);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private String firstSentence(String text) {
|
||||
if (text == null || text.isBlank()) {
|
||||
return "";
|
||||
}
|
||||
int end = text.indexOf('\n');
|
||||
return end < 0 ? text : text.substring(0, end);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
public record ComposerRenderResult(String answer, Map<String, Object> audit) {
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
public final class ComposerSafeInputBuilder {
|
||||
|
||||
public Map<String, Object> build(
|
||||
String originalQuery,
|
||||
VerifierDecision decision,
|
||||
Map<String, Object> executorOutput) {
|
||||
Map<String, Object> input = new LinkedHashMap<>();
|
||||
input.put("original_query", originalQuery);
|
||||
input.put("verdict", decision.verdict());
|
||||
|
||||
Map<String, Map<String, Object>> claimsById = indexClaims(executorOutput);
|
||||
List<Map<String, Object>> allowedClaims = new ArrayList<>();
|
||||
List<Map<String, Object>> allowedHypotheses = new ArrayList<>();
|
||||
List<String> missingInfo = extractMissingInfo(executorOutput);
|
||||
List<Map<String, Object>> recommendedActions =
|
||||
extractRecommendedActions(executorOutput);
|
||||
|
||||
if (decision.claimChecks().isEmpty()) {
|
||||
addLegacyFacts(decision, allowedClaims, allowedHypotheses, missingInfo);
|
||||
} else {
|
||||
for (Map<String, Object> check : decision.claimChecks()) {
|
||||
String verification = text(check.getOrDefault(
|
||||
"verification", "unsupported"));
|
||||
String claimId = text(check.get("claim_id"));
|
||||
String claimText = text(check.get("claim_text"));
|
||||
String detail = text(check.get("detail"));
|
||||
Map<String, Object> claim = buildClaim(claimsById.get(claimId), check);
|
||||
switch (verification) {
|
||||
case "direct_observation", "reasonable_inference" ->
|
||||
allowedClaims.add(claim);
|
||||
case "overstated" -> allowedHypotheses.add(Map.of(
|
||||
"hypothesis_text", claimText,
|
||||
"basis", detail.isBlank()
|
||||
? "当前证据只能支持部分判断，不能作为确认结论"
|
||||
: detail));
|
||||
default -> addMissingInfo(missingInfo, claimText, detail);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ("REJECT".equals(decision.verdict())) {
|
||||
allowedHypotheses = List.of();
|
||||
}
|
||||
input.put("allowed_claims", allowedClaims);
|
||||
input.put("allowed_hypotheses", allowedHypotheses);
|
||||
input.put("missing_info", missingInfo);
|
||||
input.put("recommended_actions", recommendedActions);
|
||||
input.put("rationale", decision.rationale());
|
||||
return input;
|
||||
}
|
||||
|
||||
private Map<String, Map<String, Object>> indexClaims(
|
||||
Map<String, Object> executorOutput) {
|
||||
Map<String, Map<String, Object>> result = new LinkedHashMap<>();
|
||||
Object claimsValue = executorOutput == null ? null : executorOutput.get("claims");
|
||||
if (!(claimsValue instanceof List<?> claims)) {
|
||||
return result;
|
||||
}
|
||||
for (Object value : claims) {
|
||||
if (!(value instanceof Map<?, ?> rawClaim)) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> claim = copy(rawClaim);
|
||||
String claimId = text(claim.get("claim_id"));
|
||||
if (!claimId.isBlank()) {
|
||||
result.put(claimId, claim);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private Map<String, Object> buildClaim(
|
||||
Map<String, Object> executorClaim,
|
||||
Map<String, Object> claimCheck) {
|
||||
Map<String, Object> claim = new LinkedHashMap<>();
|
||||
claim.put("claim_id", valueFrom(executorClaim, claimCheck, "claim_id"));
|
||||
claim.put("claim_type", valueFrom(executorClaim, claimCheck, "claim_type"));
|
||||
claim.put("claim_text", valueFrom(executorClaim, claimCheck, "claim_text"));
|
||||
claim.put("support_level", executorClaim == null
|
||||
? ""
|
||||
: text(executorClaim.get("support_level")));
|
||||
claim.put("verification", text(claimCheck.get("verification")));
|
||||
claim.put("detail", text(claimCheck.get("detail")));
|
||||
return claim;
|
||||
}
|
||||
|
||||
private String valueFrom(
|
||||
Map<String, Object> primary,
|
||||
Map<String, Object> fallback,
|
||||
String key) {
|
||||
String value = primary == null ? "" : text(primary.get(key));
|
||||
return value.isBlank() ? text(fallback.get(key)) : value;
|
||||
}
|
||||
|
||||
private List<String> extractMissingInfo(Map<String, Object> executorOutput) {
|
||||
List<String> result = new ArrayList<>();
|
||||
Object value = executorOutput == null ? null : executorOutput.get("missing_info");
|
||||
if (value instanceof List<?> items) {
|
||||
for (Object item : items) {
|
||||
String text = text(item);
|
||||
if (!text.isBlank() && !result.contains(text)) {
|
||||
result.add(text);
|
||||
}
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> extractRecommendedActions(
|
||||
Map<String, Object> executorOutput) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
Object value = executorOutput == null
|
||||
? null
|
||||
: executorOutput.get("recommended_actions");
|
||||
if (value instanceof List<?> items) {
|
||||
for (Object item : items) {
|
||||
if (item instanceof Map<?, ?> action) {
|
||||
Map<String, Object> filtered = new LinkedHashMap<>();
|
||||
filtered.put("action_text", text(action.get("action_text")));
|
||||
filtered.put("reason", text(action.get("reason")));
|
||||
if (!text(filtered.get("action_text")).isBlank()) {
|
||||
result.add(filtered);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private void addLegacyFacts(
|
||||
VerifierDecision decision,
|
||||
List<Map<String, Object>> allowedClaims,
|
||||
List<Map<String, Object>> allowedHypotheses,
|
||||
List<String> missingInfo) {
|
||||
for (Map<String, Object> fact : decision.factsChecked()) {
|
||||
String verification = text(fact.get("verification"));
|
||||
String factText = text(fact.get("fact"));
|
||||
String detail = text(fact.get("detail"));
|
||||
if ("direct_evidence".equals(verification)) {
|
||||
Map<String, Object> claim = new LinkedHashMap<>();
|
||||
claim.put("claim_id", "");
|
||||
claim.put("claim_type", "");
|
||||
claim.put("claim_text", factText);
|
||||
claim.put("support_level", "direct");
|
||||
claim.put("verification", verification);
|
||||
claim.put("detail", detail);
|
||||
allowedClaims.add(claim);
|
||||
} else if ("indirect_support".equals(verification)) {
|
||||
allowedHypotheses.add(Map.of(
|
||||
"hypothesis_text", factText,
|
||||
"basis", detail.isBlank()
|
||||
? "当前仅有间接支持，不能作为确认结论"
|
||||
: detail));
|
||||
} else {
|
||||
addMissingInfo(missingInfo, factText, detail);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void addMissingInfo(List<String> target, String text, String detail) {
|
||||
if (text == null || text.isBlank()) {
|
||||
return;
|
||||
}
|
||||
String value = detail == null || detail.isBlank()
|
||||
? text
|
||||
: text + "：" + detail;
|
||||
if (!target.contains(value)) {
|
||||
target.add(value);
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, Object> copy(Map<?, ?> source) {
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
source.forEach((key, value) -> result.put(String.valueOf(key), value));
|
||||
return result;
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
if (value == null) {
|
||||
return "";
|
||||
}
|
||||
String text = String.valueOf(value);
|
||||
return "null".equals(text) ? "" : text;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
|
||||
public final class EvidenceGapExtractor {
|
||||
|
||||
public List<Map<String, Object>> extract(Map<String, Object> verifierOutput) {
|
||||
Object factsValue = verifierOutput == null
|
||||
? null
|
||||
: verifierOutput.get("facts_checked");
|
||||
if (!(factsValue instanceof List<?> facts)) {
|
||||
return List.of();
|
||||
}
|
||||
List<Map<String, Object>> gaps = new ArrayList<>();
|
||||
for (Object value : facts) {
|
||||
if (!(value instanceof Map<?, ?> fact)
|
||||
|| !isCritical(fact.get("is_critical"))) {
|
||||
continue;
|
||||
}
|
||||
String statement = text(fact.get("fact"));
|
||||
String verification = text(fact.get("verification"))
|
||||
.toLowerCase(Locale.ROOT);
|
||||
if (statement.isBlank()
|
||||
|| !("no_evidence".equals(verification)
|
||||
|| "indirect_support".equals(verification))) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> gap = new LinkedHashMap<>();
|
||||
gap.put("claim_id", claimId(statement));
|
||||
gap.put("fact", statement);
|
||||
gap.put("verification", verification);
|
||||
gap.put("reason", text(fact.get("detail")));
|
||||
gaps.add(gap);
|
||||
}
|
||||
return gaps;
|
||||
}
|
||||
|
||||
public boolean hasCriticalGap(Map<String, Object> verifierOutput) {
|
||||
return !extract(verifierOutput).isEmpty();
|
||||
}
|
||||
|
||||
private boolean isCritical(Object value) {
|
||||
return Boolean.TRUE.equals(value)
|
||||
|| "true".equalsIgnoreCase(text(value));
|
||||
}
|
||||
|
||||
private String claimId(String statement) {
|
||||
int separator = statement.indexOf(':');
|
||||
if (separator <= 0) {
|
||||
return "";
|
||||
}
|
||||
return statement.substring(0, separator).trim();
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
return value == null ? "" : String.valueOf(value).trim();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,190 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.Set;
|
||||
|
||||
public final class ExecutorEvidenceParser {
|
||||
|
||||
private static final TypeReference<Map<String, Object>> MAP_TYPE = new TypeReference<>() {
|
||||
};
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public ExecutorEvidenceParser() {
|
||||
this(new ObjectMapper());
|
||||
}
|
||||
|
||||
public ExecutorEvidenceParser(ObjectMapper objectMapper) {
|
||||
this.objectMapper = Objects.requireNonNull(objectMapper, "objectMapper");
|
||||
}
|
||||
|
||||
public ParseResult parse(String rawOutput) {
|
||||
if (rawOutput == null || rawOutput.isBlank()) {
|
||||
return new ParseResult(null, status("missing", "executor_final_answer is blank"));
|
||||
}
|
||||
|
||||
String sanitized = JsonPayloadSupport.sanitizeObject(rawOutput);
|
||||
if (!JsonPayloadSupport.looksLikeObject(sanitized)) {
|
||||
return new ParseResult(null, status("missing", "executor output is not JSON"));
|
||||
}
|
||||
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(sanitized);
|
||||
if (!root.isObject() || !root.path("claims").isArray()) {
|
||||
return new ParseResult(null, status("malformed",
|
||||
"executor output JSON does not match evidence-attribution contract"));
|
||||
}
|
||||
Map<String, Object> structuredOutput = objectMapper.convertValue(root, MAP_TYPE);
|
||||
return new ParseResult(structuredOutput,
|
||||
status("valid", "parsed executor evidence contract"));
|
||||
} catch (Exception exception) {
|
||||
return new ParseResult(null, status("malformed", exception.getMessage()));
|
||||
}
|
||||
}
|
||||
|
||||
public ParseResult enrich(ParseResult parsed,
|
||||
List<Map<String, Object>> toolTraceSummary) {
|
||||
Objects.requireNonNull(parsed, "parsed");
|
||||
Map<String, Object> structuredOutput = parsed.structuredOutput();
|
||||
if (structuredOutput == null) {
|
||||
return parsed;
|
||||
}
|
||||
|
||||
Map<String, List<Long>> invocationIdsByTool =
|
||||
invocationIdsByTool(toolTraceSummary);
|
||||
List<Map<String, Object>> warnings = new ArrayList<>();
|
||||
enrichBindings(structuredOutput.get("claims"), invocationIdsByTool, warnings);
|
||||
enrichBindings(structuredOutput.get("recommended_actions"), invocationIdsByTool, warnings);
|
||||
if (!warnings.isEmpty()) {
|
||||
structuredOutput.put("_gatekeeper_warnings", warnings);
|
||||
}
|
||||
return new ParseResult(structuredOutput, parsed.status());
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
private void enrichBindings(Object sectionValue,
|
||||
Map<String, List<Long>> invocationIdsByTool,
|
||||
List<Map<String, Object>> warnings) {
|
||||
if (!(sectionValue instanceof List<?> items)) {
|
||||
return;
|
||||
}
|
||||
for (Object itemValue : items) {
|
||||
if (!(itemValue instanceof Map<?, ?> item)) {
|
||||
continue;
|
||||
}
|
||||
Object bindingsValue = item.get("evidence_bindings");
|
||||
if (!(bindingsValue instanceof List<?> bindings)) {
|
||||
continue;
|
||||
}
|
||||
for (Object bindingValue : bindings) {
|
||||
if (!(bindingValue instanceof Map<?, ?> rawBinding)) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> binding = (Map<String, Object>) rawBinding;
|
||||
String normalizedToolName = normalizeToolName(binding.get("tool_name"));
|
||||
if (!normalizedToolName.isBlank()) {
|
||||
binding.put("tool_name", normalizedToolName);
|
||||
}
|
||||
if (!hasInvocationId(binding)) {
|
||||
List<Long> ids = invocationIdsByTool.getOrDefault(
|
||||
normalizedToolName, List.of());
|
||||
if (ids.size() == 1) {
|
||||
binding.put("source_invocation_id", ids.get(0));
|
||||
warnings.add(Map.of(
|
||||
"rule", "evidence.invocation_auto_backfill",
|
||||
"message", "source_invocation_id was auto-filled from the unique tool invocation candidate; raw_path remains missing if Executor did not provide it",
|
||||
"tool_name", normalizedToolName,
|
||||
"source_invocation_id", ids.get(0)));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, List<Long>> invocationIdsByTool(
|
||||
List<Map<String, Object>> toolTraceSummary) {
|
||||
Map<String, Set<Long>> idsByTool = new LinkedHashMap<>();
|
||||
List<Map<String, Object>> summaries = toolTraceSummary == null
|
||||
? List.of()
|
||||
: toolTraceSummary;
|
||||
for (Map<String, Object> summary : summaries) {
|
||||
String toolName = normalizeToolName(summary.get("tool_name"));
|
||||
if (toolName.isBlank()) {
|
||||
continue;
|
||||
}
|
||||
List<Long> ids = toLongList(summary.get("source_invocation_ids"));
|
||||
if (!ids.isEmpty()) {
|
||||
idsByTool.computeIfAbsent(toolName, ignored -> new LinkedHashSet<>())
|
||||
.addAll(ids);
|
||||
}
|
||||
}
|
||||
Map<String, List<Long>> result = new LinkedHashMap<>();
|
||||
idsByTool.forEach((tool, ids) -> result.put(tool, new ArrayList<>(ids)));
|
||||
return result;
|
||||
}
|
||||
|
||||
private boolean hasInvocationId(Map<String, Object> binding) {
|
||||
return asLong(binding.get("source_invocation_id")) != null
|
||||
|| toLongList(binding.get("source_invocation_ids")).size() == 1;
|
||||
}
|
||||
|
||||
private List<Long> toLongList(Object value) {
|
||||
if (!(value instanceof List<?> values)) {
|
||||
return List.of();
|
||||
}
|
||||
List<Long> ids = new ArrayList<>();
|
||||
for (Object item : values) {
|
||||
Long id = asLong(item);
|
||||
if (id != null) {
|
||||
ids.add(id);
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
private Long asLong(Object value) {
|
||||
if (value instanceof Number number) {
|
||||
return number.longValue();
|
||||
}
|
||||
if (value instanceof String text) {
|
||||
try {
|
||||
return Long.parseLong(text);
|
||||
} catch (NumberFormatException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private String normalizeToolName(Object value) {
|
||||
String toolName = value == null ? "" : String.valueOf(value);
|
||||
return switch (toolName) {
|
||||
case "lookupKnowledge" -> "lookup_knowledge";
|
||||
case "queryLogs" -> "query_logs";
|
||||
case "queryPrometheusAlerts" -> "query_metrics";
|
||||
case "getAvailableLogTopics" -> "get_available_log_topics";
|
||||
default -> toolName;
|
||||
};
|
||||
}
|
||||
|
||||
private Map<String, Object> status(String value, String detail) {
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("status", value);
|
||||
result.put("detail", detail == null ? "" : detail);
|
||||
return result;
|
||||
}
|
||||
|
||||
public record ParseResult(
|
||||
Map<String, Object> structuredOutput,
|
||||
Map<String, Object> status) {
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
public final class JsonPayloadSupport {
|
||||
|
||||
private JsonPayloadSupport() {
|
||||
}
|
||||
|
||||
public static String sanitizeObject(String raw) {
|
||||
if (raw == null) {
|
||||
return "";
|
||||
}
|
||||
String trimmed = raw.trim();
|
||||
int fenceStart = trimmed.indexOf("```");
|
||||
if (fenceStart >= 0) {
|
||||
int firstNewline = trimmed.indexOf('\n', fenceStart);
|
||||
int lastFence = firstNewline < 0
|
||||
? -1
|
||||
: trimmed.indexOf("```", firstNewline + 1);
|
||||
if (firstNewline >= 0 && lastFence > firstNewline) {
|
||||
return trimmed.substring(firstNewline + 1, lastFence).trim();
|
||||
}
|
||||
}
|
||||
int objectStart = trimmed.indexOf('{');
|
||||
int objectEnd = trimmed.lastIndexOf('}');
|
||||
if (objectStart >= 0 && objectEnd > objectStart) {
|
||||
return trimmed.substring(objectStart, objectEnd + 1).trim();
|
||||
}
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
public static boolean looksLikeObject(String text) {
|
||||
return text != null && text.startsWith("{") && text.endsWith("}");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
public final class SafeFallbackRenderer {
|
||||
|
||||
public static final String LOW_CONFID_DISCLAIMER =
|
||||
"以下结论基于当前已获取证据，仍存在部分证据缺口，请谨慎参考。";
|
||||
public static final String DEGRADED_PREFIX =
|
||||
"当前无法基于已获取证据生成可靠结论，建议人工介入。";
|
||||
|
||||
public String render(Map<String, Object> input, List<String> defaultSuggestions) {
|
||||
Map<String, Object> safeInput = input == null ? Map.of() : input;
|
||||
String verdict = text(safeInput.getOrDefault("verdict", "LOW_CONFID"));
|
||||
List<Map<String, Object>> allowedClaims = maps(safeInput.get("allowed_claims"));
|
||||
List<Map<String, Object>> allowedHypotheses = maps(
|
||||
safeInput.get("allowed_hypotheses"));
|
||||
List<String> missingInfo = strings(safeInput.get("missing_info"));
|
||||
List<Map<String, Object>> recommendedActions = maps(
|
||||
safeInput.get("recommended_actions"));
|
||||
|
||||
StringBuilder output = new StringBuilder();
|
||||
if ("REJECT".equals(verdict)) {
|
||||
output.append(DEGRADED_PREFIX);
|
||||
} else if ("LOW_CONFID".equals(verdict)) {
|
||||
output.append(LOW_CONFID_DISCLAIMER);
|
||||
}
|
||||
|
||||
output.append("\n\n已确认信息：");
|
||||
if (allowedClaims.isEmpty()) {
|
||||
output.append("\n- 暂无可稳定确认的信息");
|
||||
} else {
|
||||
for (Map<String, Object> claim : allowedClaims) {
|
||||
appendBullet(output, text(claim.get("claim_text")));
|
||||
}
|
||||
}
|
||||
|
||||
if (!"REJECT".equals(verdict) && !allowedHypotheses.isEmpty()) {
|
||||
output.append("\n\n可能方向：");
|
||||
for (Map<String, Object> hypothesis : allowedHypotheses) {
|
||||
String hypothesisText = text(hypothesis.get("hypothesis_text"));
|
||||
if (hypothesisText.isBlank()) {
|
||||
continue;
|
||||
}
|
||||
output.append("\n- ").append(hypothesisText);
|
||||
String basis = text(hypothesis.get("basis"));
|
||||
if (!basis.isBlank()) {
|
||||
output.append("（").append(basis).append("）");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output.append("\n\n")
|
||||
.append("REJECT".equals(verdict) ? "证据缺口：" : "当前缺口：");
|
||||
if (missingInfo.isEmpty()) {
|
||||
output.append("\n- 当前缺少足够的直接证据支撑核心结论");
|
||||
} else {
|
||||
missingInfo.forEach(gap -> appendBullet(output, gap));
|
||||
}
|
||||
|
||||
output.append("\n\n建议下一步：");
|
||||
if (recommendedActions.isEmpty()) {
|
||||
List<String> suggestions = defaultSuggestions == null
|
||||
? List.of()
|
||||
: defaultSuggestions;
|
||||
if (suggestions.isEmpty()) {
|
||||
suggestions = List.of("围绕上述证据缺口补充只读查询，再由人工复核最终结论");
|
||||
}
|
||||
suggestions.forEach(suggestion -> appendBullet(output, suggestion));
|
||||
} else {
|
||||
for (Map<String, Object> action : recommendedActions) {
|
||||
String actionText = text(action.get("action_text"));
|
||||
if (actionText.isBlank()) {
|
||||
continue;
|
||||
}
|
||||
output.append("\n- ").append(actionText);
|
||||
String reason = text(action.get("reason"));
|
||||
if (!reason.isBlank()) {
|
||||
output.append("：").append(reason);
|
||||
}
|
||||
}
|
||||
}
|
||||
return output.toString().trim();
|
||||
}
|
||||
|
||||
private void appendBullet(StringBuilder target, String value) {
|
||||
if (value != null && !value.isBlank()) {
|
||||
target.append("\n- ").append(value);
|
||||
}
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> maps(Object value) {
|
||||
if (!(value instanceof List<?> items)) {
|
||||
return List.of();
|
||||
}
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
for (Object item : items) {
|
||||
if (item instanceof Map<?, ?> map) {
|
||||
Map<String, Object> copy = new LinkedHashMap<>();
|
||||
map.forEach((key, nested) -> copy.put(String.valueOf(key), nested));
|
||||
result.add(copy);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<String> strings(Object value) {
|
||||
if (!(value instanceof List<?> items)) {
|
||||
return List.of();
|
||||
}
|
||||
List<String> result = new ArrayList<>();
|
||||
for (Object item : items) {
|
||||
String text = text(item);
|
||||
if (!text.isBlank()) {
|
||||
result.add(text);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
if (value == null) {
|
||||
return "";
|
||||
}
|
||||
String text = String.valueOf(value);
|
||||
return "null".equals(text) ? "" : text;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
public record VerifierDecision(
|
||||
String verdict,
|
||||
double groundednessScore,
|
||||
int criticalFactCount,
|
||||
List<Map<String, Object>> claimChecks,
|
||||
List<Map<String, Object>> factsChecked,
|
||||
String rationale,
|
||||
int round) {
|
||||
|
||||
public VerifierDecision {
|
||||
claimChecks = claimChecks == null ? List.of() : List.copyOf(claimChecks);
|
||||
factsChecked = factsChecked == null ? List.of() : List.copyOf(factsChecked);
|
||||
rationale = rationale == null ? "" : rationale;
|
||||
}
|
||||
|
||||
public VerifierDecision withVerdict(String effectiveVerdict) {
|
||||
return new VerifierDecision(
|
||||
effectiveVerdict,
|
||||
groundednessScore,
|
||||
criticalFactCount,
|
||||
claimChecks,
|
||||
factsChecked,
|
||||
rationale,
|
||||
round);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,227 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
|
||||
public final class VerifierOutputParser {
|
||||
|
||||
private static final String VERDICT_PASS = "PASS";
|
||||
private static final String VERDICT_LOW_CONFID = "LOW_CONFID";
|
||||
private static final String VERDICT_REJECT = "REJECT";
|
||||
private static final String RULE_INVOCATION_REF = "evidence.invocation_ref";
|
||||
|
||||
private final ObjectMapper objectMapper;
|
||||
|
||||
public VerifierOutputParser() {
|
||||
this(new ObjectMapper());
|
||||
}
|
||||
|
||||
public VerifierOutputParser(ObjectMapper objectMapper) {
|
||||
this.objectMapper = Objects.requireNonNull(objectMapper, "objectMapper");
|
||||
}
|
||||
|
||||
public VerifierDecision parse(String rawOutput, int round) {
|
||||
if (rawOutput == null || rawOutput.isBlank()) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(
|
||||
JsonPayloadSupport.sanitizeObject(rawOutput));
|
||||
List<Map<String, Object>> claimChecks =
|
||||
parseClaimChecks(root.path("claim_checks"));
|
||||
List<Map<String, Object>> factsChecked = claimChecks.isEmpty()
|
||||
? parseFactsChecked(root.path("facts_checked"))
|
||||
: mapClaimChecksToFactsChecked(claimChecks);
|
||||
int criticalFactCount = root.path("critical_fact_count")
|
||||
.asInt(countCriticalFacts(factsChecked));
|
||||
return new VerifierDecision(
|
||||
normalizeVerdict(root.path("verdict").asText(VERDICT_LOW_CONFID)),
|
||||
root.path("groundedness_score").asDouble(0.0),
|
||||
criticalFactCount,
|
||||
claimChecks,
|
||||
factsChecked,
|
||||
root.path("rationale").asText(""),
|
||||
round);
|
||||
} catch (Exception ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
public VerifierDecision applyCeiling(
|
||||
VerifierDecision modelDecision,
|
||||
String executorParseStatus,
|
||||
Map<String, Object> gatekeeperResult) {
|
||||
if (modelDecision == null) {
|
||||
return null;
|
||||
}
|
||||
String verdict = normalizeVerdict(modelDecision.verdict());
|
||||
if (("missing".equals(executorParseStatus)
|
||||
|| "malformed".equals(executorParseStatus))
|
||||
&& VERDICT_PASS.equals(verdict)) {
|
||||
verdict = VERDICT_LOW_CONFID;
|
||||
}
|
||||
|
||||
if (gatekeeperResult != null
|
||||
&& "fail".equals(String.valueOf(gatekeeperResult.get("status")))) {
|
||||
String severity = String.valueOf(
|
||||
gatekeeperResult.getOrDefault("severity", ""));
|
||||
if ("reject".equals(severity)
|
||||
|| containsRule(gatekeeperResult.get("failed_rules"),
|
||||
RULE_INVOCATION_REF)) {
|
||||
verdict = VERDICT_REJECT;
|
||||
} else if (VERDICT_PASS.equals(verdict)) {
|
||||
verdict = VERDICT_LOW_CONFID;
|
||||
}
|
||||
}
|
||||
return modelDecision.withVerdict(verdict);
|
||||
}
|
||||
|
||||
public VerifierDecision fallback(int round, String rationale) {
|
||||
return new VerifierDecision(
|
||||
VERDICT_LOW_CONFID,
|
||||
0.0,
|
||||
0,
|
||||
List.of(),
|
||||
List.of(),
|
||||
rationale,
|
||||
round);
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseClaimChecks(JsonNode node) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
if (!node.isArray()) {
|
||||
return result;
|
||||
}
|
||||
for (JsonNode claimNode : node) {
|
||||
Map<String, Object> claim = new LinkedHashMap<>();
|
||||
claim.put("claim_id", claimNode.path("claim_id").asText(""));
|
||||
claim.put("claim_text", claimNode.path("claim_text").asText(""));
|
||||
claim.put("claim_type", claimNode.path("claim_type").asText(""));
|
||||
claim.put("verification", normalizeClaimVerification(
|
||||
claimNode.path("verification").asText("unsupported")));
|
||||
claim.put("detail", claimNode.path("detail").asText(""));
|
||||
claim.put("evidence_refs", parseEvidenceRefs(
|
||||
claimNode.path("evidence_refs")));
|
||||
result.add(claim);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> mapClaimChecksToFactsChecked(
|
||||
List<Map<String, Object>> claimChecks) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
for (Map<String, Object> claimCheck : claimChecks) {
|
||||
String claimId = text(claimCheck.get("claim_id"));
|
||||
String claimText = text(claimCheck.get("claim_text"));
|
||||
String claimType = text(claimCheck.get("claim_type"));
|
||||
Map<String, Object> fact = new LinkedHashMap<>();
|
||||
fact.put("fact", claimId.isBlank()
|
||||
? claimText
|
||||
: claimId + ": " + claimText);
|
||||
fact.put("is_critical", isCriticalClaimType(claimType));
|
||||
fact.put("verification", mapClaimVerificationToFactVerification(
|
||||
text(claimCheck.get("verification"))));
|
||||
fact.put("detail", claimCheck.getOrDefault("detail", ""));
|
||||
fact.put("evidence_refs", claimCheck.getOrDefault(
|
||||
"evidence_refs", List.of()));
|
||||
result.add(fact);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseFactsChecked(JsonNode node) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
if (!node.isArray()) {
|
||||
return result;
|
||||
}
|
||||
for (JsonNode factNode : node) {
|
||||
Map<String, Object> fact = new LinkedHashMap<>();
|
||||
fact.put("fact", factNode.path("fact").asText(""));
|
||||
fact.put("is_critical", factNode.path("is_critical").asBoolean(false));
|
||||
fact.put("verification", factNode.path("verification").asText(""));
|
||||
fact.put("detail", factNode.path("detail").asText(""));
|
||||
fact.put("evidence_refs", parseEvidenceRefs(
|
||||
factNode.path("evidence_refs")));
|
||||
result.add(fact);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseEvidenceRefs(JsonNode node) {
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
if (!node.isArray()) {
|
||||
return result;
|
||||
}
|
||||
for (JsonNode refNode : node) {
|
||||
Map<String, Object> ref = new LinkedHashMap<>();
|
||||
ref.put("trace_ref", refNode.path("trace_ref").asText(""));
|
||||
ref.put("tool_name", refNode.path("tool_name").asText(""));
|
||||
ref.put("topic_domain", refNode.path("topic_domain").asText(""));
|
||||
ref.put("note", refNode.path("note").asText(""));
|
||||
List<Long> ids = new ArrayList<>();
|
||||
JsonNode idsNode = refNode.path("source_invocation_ids");
|
||||
if (idsNode.isArray()) {
|
||||
for (JsonNode idNode : idsNode) {
|
||||
if (idNode.canConvertToLong()) {
|
||||
ids.add(idNode.asLong());
|
||||
}
|
||||
}
|
||||
}
|
||||
ref.put("source_invocation_ids", ids);
|
||||
result.add(ref);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private int countCriticalFacts(List<Map<String, Object>> facts) {
|
||||
return (int) facts.stream()
|
||||
.filter(fact -> Boolean.TRUE.equals(fact.get("is_critical")))
|
||||
.count();
|
||||
}
|
||||
|
||||
private String normalizeVerdict(String verdict) {
|
||||
return switch (verdict) {
|
||||
case VERDICT_PASS, VERDICT_LOW_CONFID, VERDICT_REJECT -> verdict;
|
||||
default -> VERDICT_LOW_CONFID;
|
||||
};
|
||||
}
|
||||
|
||||
private String normalizeClaimVerification(String verification) {
|
||||
return switch (verification) {
|
||||
case "direct_observation", "reasonable_inference", "overstated",
|
||||
"unsupported", "external_unknown", "contradicted" -> verification;
|
||||
default -> "unsupported";
|
||||
};
|
||||
}
|
||||
|
||||
private boolean isCriticalClaimType(String claimType) {
|
||||
return "root_cause".equals(claimType)
|
||||
|| "symptom".equals(claimType)
|
||||
|| "impact".equals(claimType)
|
||||
|| "risk".equals(claimType);
|
||||
}
|
||||
|
||||
private String mapClaimVerificationToFactVerification(String verification) {
|
||||
return switch (verification) {
|
||||
case "direct_observation" -> "direct_evidence";
|
||||
case "reasonable_inference", "overstated" -> "indirect_support";
|
||||
case "contradicted" -> "contradicted";
|
||||
default -> "no_evidence";
|
||||
};
|
||||
}
|
||||
|
||||
private boolean containsRule(Object value, String expected) {
|
||||
return value instanceof List<?> rules
|
||||
&& rules.stream().anyMatch(rule -> expected.equals(String.valueOf(rule)));
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
return value == null ? "" : String.valueOf(value);
|
||||
}
|
||||
}
|
||||
+94
@@ -0,0 +1,94 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.action.AsyncNodeActionWithConfig;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
abstract class AbstractDiagnosisAgentNodeAdapter
|
||||
implements AsyncNodeActionWithConfig {
|
||||
|
||||
protected final ObjectMapper objectMapper;
|
||||
|
||||
private final String nodeName;
|
||||
private final String retryCountKey;
|
||||
private final DiagnosisAgentInvoker invoker;
|
||||
private final DiagnosisNodeFailureClassifier failureClassifier;
|
||||
|
||||
AbstractDiagnosisAgentNodeAdapter(
|
||||
String nodeName,
|
||||
String retryCountKey,
|
||||
DiagnosisAgentInvoker invoker,
|
||||
ObjectMapper objectMapper,
|
||||
DiagnosisNodeFailureClassifier failureClassifier) {
|
||||
this.nodeName = Objects.requireNonNull(nodeName, "nodeName");
|
||||
this.retryCountKey = Objects.requireNonNull(retryCountKey, "retryCountKey");
|
||||
this.invoker = Objects.requireNonNull(invoker, "invoker");
|
||||
this.objectMapper = Objects.requireNonNull(objectMapper, "objectMapper");
|
||||
this.failureClassifier = Objects.requireNonNull(
|
||||
failureClassifier, "failureClassifier");
|
||||
}
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Map<String, Object>> apply(
|
||||
OverAllState state,
|
||||
RunnableConfig config) {
|
||||
int attempt = DiagnosisGraphState.intValue(state, retryCountKey) + 1;
|
||||
try {
|
||||
String input = objectMapper.writeValueAsString(projectInput(state));
|
||||
NodeResult result = parseOutput(state, invoker.invoke(input, config));
|
||||
return CompletableFuture.completedFuture(toUpdate(result, attempt));
|
||||
} catch (Throwable failure) {
|
||||
boolean retryable = failureClassifier.classify(failure)
|
||||
== DiagnosisNodeFailureClassifier.FailureKind.RETRYABLE;
|
||||
NodeResult result = failureResult(failure, retryable);
|
||||
return CompletableFuture.completedFuture(toUpdate(result, attempt));
|
||||
}
|
||||
}
|
||||
|
||||
protected NodeResult failureResult(Throwable failure, boolean retryable) {
|
||||
return new NodeResult(
|
||||
retryable ? retryableFailureStatus() : nonRetryableFailureStatus(),
|
||||
retryable ? retryableFailureReason() : nonRetryableFailureReason(),
|
||||
Map.of(DiagnosisGraphState.FAILURE_REASON,
|
||||
retryable ? retryableFailureReason() : nonRetryableFailureReason()));
|
||||
}
|
||||
|
||||
protected abstract Map<String, Object> projectInput(OverAllState state);
|
||||
|
||||
protected abstract NodeResult parseOutput(OverAllState state, String rawOutput);
|
||||
|
||||
protected abstract String retryableFailureStatus();
|
||||
|
||||
protected abstract String nonRetryableFailureStatus();
|
||||
|
||||
protected abstract String retryableFailureReason();
|
||||
|
||||
protected abstract String nonRetryableFailureReason();
|
||||
|
||||
private Map<String, Object> toUpdate(NodeResult result, int attempt) {
|
||||
Map<String, Object> update = new LinkedHashMap<>(result.values());
|
||||
update.put(DiagnosisGraphState.ORCHESTRATION_EVENTS,
|
||||
List.of(new OrchestrationEvent(
|
||||
nodeName, result.outcome(), result.reasonCode(), attempt)));
|
||||
return update;
|
||||
}
|
||||
|
||||
protected record NodeResult(
|
||||
String outcome,
|
||||
String reasonCode,
|
||||
Map<String, Object> values) {
|
||||
|
||||
protected NodeResult {
|
||||
Objects.requireNonNull(outcome, "outcome");
|
||||
Objects.requireNonNull(reasonCode, "reasonCode");
|
||||
values = values == null ? Map.of() : Map.copyOf(values);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,95 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerOutputParser;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerRenderResult;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerSafeInputBuilder;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.ComposerStatus;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
public final class ComposerNodeAdapter extends AbstractDiagnosisAgentNodeAdapter {
|
||||
|
||||
private final ComposerSafeInputBuilder inputBuilder;
|
||||
private final ComposerOutputParser outputParser;
|
||||
private final VerifierDecisionStateMapper stateMapper;
|
||||
|
||||
public ComposerNodeAdapter(DiagnosisAgentInvoker invoker) {
|
||||
this(invoker, new ObjectMapper(), DiagnosisNodeFailureClassifier.defaults());
|
||||
}
|
||||
|
||||
public ComposerNodeAdapter(
|
||||
DiagnosisAgentInvoker invoker,
|
||||
ObjectMapper objectMapper,
|
||||
DiagnosisNodeFailureClassifier failureClassifier) {
|
||||
super(
|
||||
DiagnosisGraphTopology.Node.COMPOSER,
|
||||
DiagnosisGraphState.COMPOSER_RETRY_COUNT,
|
||||
invoker,
|
||||
objectMapper,
|
||||
failureClassifier);
|
||||
this.inputBuilder = new ComposerSafeInputBuilder();
|
||||
this.outputParser = new ComposerOutputParser();
|
||||
this.stateMapper = new VerifierDecisionStateMapper();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected Map<String, Object> projectInput(OverAllState state) {
|
||||
return inputBuilder.build(
|
||||
stateMapper.originalQuery(state),
|
||||
stateMapper.decision(state),
|
||||
DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected NodeResult parseOutput(OverAllState state, String rawOutput) {
|
||||
Map<String, Object> safeInput = projectInput(state);
|
||||
ComposerRenderResult parsed = outputParser.parse(
|
||||
rawOutput,
|
||||
safeInput,
|
||||
List.of("围绕上述证据缺口补充只读查询，再由人工复核最终结论"));
|
||||
if (!"valid".equals(parsed.audit().get("status"))) {
|
||||
return new NodeResult(
|
||||
ComposerStatus.INVALID_OUTPUT.name(),
|
||||
"composer_invalid_output",
|
||||
Map.of(
|
||||
DiagnosisGraphState.COMPOSER_STATUS,
|
||||
ComposerStatus.INVALID_OUTPUT.name(),
|
||||
DiagnosisGraphState.FAILURE_REASON,
|
||||
"composer_invalid_output"));
|
||||
}
|
||||
return new NodeResult(
|
||||
ComposerStatus.COMPLETED.name(),
|
||||
"composer_completed",
|
||||
Map.of(
|
||||
DiagnosisGraphState.COMPOSER_STATUS,
|
||||
ComposerStatus.COMPLETED.name(),
|
||||
DiagnosisGraphState.COMPOSER_OUTPUT,
|
||||
parsed.audit(),
|
||||
DiagnosisGraphState.FINAL_ANSWER,
|
||||
parsed.answer()));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureStatus() {
|
||||
return ComposerStatus.RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureStatus() {
|
||||
return ComposerStatus.NON_RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureReason() {
|
||||
return "composer_retryable_failed";
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureReason() {
|
||||
return "composer_non_retryable_failed";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
|
||||
@FunctionalInterface
|
||||
public interface DiagnosisAgentInvoker {
|
||||
|
||||
String invoke(String input, RunnableConfig config) throws Exception;
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.superbiz.agent.diagnosis.protocol.EvidenceGapExtractor;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.ComposerStatus;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.ExecutorStatus;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.GatekeeperStatus;
|
||||
@@ -8,13 +9,14 @@ import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.PlannerStatus;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.Verdict;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.VerifierStatus;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
|
||||
public final class DiagnosisGraphRouter {
|
||||
|
||||
private static final EvidenceGapExtractor EVIDENCE_GAP_EXTRACTOR =
|
||||
new EvidenceGapExtractor();
|
||||
|
||||
private DiagnosisGraphRouter() {
|
||||
}
|
||||
|
||||
@@ -110,35 +112,8 @@ public final class DiagnosisGraphRouter {
|
||||
Optional<Verdict> ceiling = DiagnosisGraphState.enumValue(
|
||||
state, DiagnosisGraphState.VERIFIER_VERDICT_CEILING, Verdict.class);
|
||||
return ceiling.filter(value -> value == Verdict.PASS).isPresent()
|
||||
&& hasValidEvidenceGap(DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIER_OUTPUT));
|
||||
}
|
||||
|
||||
private static boolean hasValidEvidenceGap(Map<String, Object> verifierOutput) {
|
||||
Object factsValue = verifierOutput.get("facts_checked");
|
||||
if (!(factsValue instanceof List<?> facts)) {
|
||||
return false;
|
||||
}
|
||||
for (Object factValue : facts) {
|
||||
if (!(factValue instanceof Map<?, ?> fact)) {
|
||||
continue;
|
||||
}
|
||||
String statement = text(fact.get("fact"));
|
||||
String verification = text(fact.get("verification"));
|
||||
if (statement != null && verification != null
|
||||
&& ("no_evidence".equals(verification)
|
||||
|| "indirect_support".equals(verification))) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private static String text(Object value) {
|
||||
if (value == null) {
|
||||
return null;
|
||||
}
|
||||
String text = String.valueOf(value).trim().toLowerCase(Locale.ROOT);
|
||||
return text.isEmpty() ? null : text;
|
||||
&& EVIDENCE_GAP_EXTRACTOR.hasCriticalGap(
|
||||
DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIER_OUTPUT));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.concurrent.TimeoutException;
|
||||
|
||||
@FunctionalInterface
|
||||
public interface DiagnosisNodeFailureClassifier {
|
||||
|
||||
FailureKind classify(Throwable failure);
|
||||
|
||||
static DiagnosisNodeFailureClassifier defaults() {
|
||||
return DiagnosisNodeFailureClassifier::classifyDefault;
|
||||
}
|
||||
|
||||
private static FailureKind classifyDefault(Throwable failure) {
|
||||
Throwable current = failure;
|
||||
int depth = 0;
|
||||
while (current != null && depth++ < 8) {
|
||||
if (current instanceof TimeoutException
|
||||
|| current instanceof IOException) {
|
||||
return FailureKind.RETRYABLE;
|
||||
}
|
||||
current = current.getCause();
|
||||
}
|
||||
return FailureKind.NON_RETRYABLE;
|
||||
}
|
||||
|
||||
enum FailureKind {
|
||||
RETRYABLE,
|
||||
NON_RETRYABLE
|
||||
}
|
||||
}
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
|
||||
import java.util.Objects;
|
||||
|
||||
public final class DiagnosisRealGraphActionsFactory {
|
||||
|
||||
public DiagnosisGraphActions create(
|
||||
DiagnosisAgentInvoker plannerInvoker,
|
||||
DiagnosisAgentInvoker executorInvoker,
|
||||
DiagnosisAgentInvoker verifierInvoker,
|
||||
DiagnosisAgentInvoker composerInvoker,
|
||||
ExecutorGatekeeperService gatekeeperService) {
|
||||
Objects.requireNonNull(plannerInvoker, "plannerInvoker");
|
||||
Objects.requireNonNull(executorInvoker, "executorInvoker");
|
||||
Objects.requireNonNull(verifierInvoker, "verifierInvoker");
|
||||
Objects.requireNonNull(composerInvoker, "composerInvoker");
|
||||
Objects.requireNonNull(gatekeeperService, "gatekeeperService");
|
||||
|
||||
return new DiagnosisGraphActions(
|
||||
new PlannerNodeAdapter(plannerInvoker),
|
||||
new ExecutorNodeAdapter(executorInvoker),
|
||||
new GatekeeperNode(gatekeeperService),
|
||||
new VerifiedInputNode(),
|
||||
new VerifierNodeAdapter(verifierInvoker),
|
||||
new EvidenceRetryPrepareNode(),
|
||||
new ComposerNodeAdapter(composerInvoker),
|
||||
new FallbackNode());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.action.AsyncNodeActionWithConfig;
|
||||
import com.superbiz.agent.diagnosis.protocol.EvidenceGapExtractor;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
public final class EvidenceRetryPrepareNode implements AsyncNodeActionWithConfig {
|
||||
|
||||
private final EvidenceGapExtractor gapExtractor;
|
||||
|
||||
public EvidenceRetryPrepareNode() {
|
||||
this(new EvidenceGapExtractor());
|
||||
}
|
||||
|
||||
public EvidenceRetryPrepareNode(EvidenceGapExtractor gapExtractor) {
|
||||
this.gapExtractor = gapExtractor;
|
||||
}
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Map<String, Object>> apply(
|
||||
OverAllState state,
|
||||
RunnableConfig config) {
|
||||
Map<String, Object> priorOutput = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT);
|
||||
List<?> priorEvidence = DiagnosisGraphState.listValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EVIDENCE);
|
||||
List<Map<String, Object>> gaps = gapExtractor.extract(
|
||||
DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIER_OUTPUT));
|
||||
|
||||
Map<String, Object> context = new LinkedHashMap<>();
|
||||
context.put("prior_verified_executor_output", priorOutput);
|
||||
context.put("prior_verified_evidence", priorEvidence);
|
||||
context.put("evidence_gaps", gaps);
|
||||
context.put("completed_query_refs", completedQueryRefs(priorEvidence));
|
||||
context.put("constraints", Map.of(
|
||||
"max_retry", 1,
|
||||
"do_not_repeat_successful_queries", true,
|
||||
"only_execute_incremental_queries", true,
|
||||
"preserve_prior_verified_claims", true,
|
||||
"return_complete_executor_evidence_v2_snapshot", true));
|
||||
|
||||
int attempt = DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1;
|
||||
Map<String, Object> update = new LinkedHashMap<>();
|
||||
update.put(DiagnosisGraphState.RETRY_CONTEXT, context);
|
||||
update.put(DiagnosisGraphState.ORCHESTRATION_EVENTS,
|
||||
List.of(new OrchestrationEvent(
|
||||
DiagnosisGraphTopology.Node.EVIDENCE_RETRY,
|
||||
"COMPLETED",
|
||||
DiagnosisGraphTopology.Reason.EVIDENCE_RETRY,
|
||||
attempt)));
|
||||
return CompletableFuture.completedFuture(update);
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> completedQueryRefs(List<?> evidence) {
|
||||
Set<QueryRef> unique = new LinkedHashSet<>();
|
||||
for (Object value : evidence) {
|
||||
if (value instanceof Map<?, ?> item) {
|
||||
unique.add(new QueryRef(
|
||||
text(item.get("tool_name")),
|
||||
text(item.get("source_invocation_id")),
|
||||
text(item.get("raw_path"))));
|
||||
}
|
||||
}
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
for (QueryRef ref : unique) {
|
||||
if (ref.toolName().isBlank()
|
||||
|| ref.invocationId().isBlank()
|
||||
|| ref.rawPath().isBlank()) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> item = new LinkedHashMap<>();
|
||||
item.put("tool_name", ref.toolName());
|
||||
item.put("source_invocation_id", ref.invocationId());
|
||||
item.put("raw_path", ref.rawPath());
|
||||
result.add(item);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
return value == null ? "" : String.valueOf(value).trim();
|
||||
}
|
||||
|
||||
private record QueryRef(String toolName, String invocationId, String rawPath) {
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.diagnosis.protocol.ExecutorEvidenceParser;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.ExecutorStatus;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.PlannerMode;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Locale;
|
||||
import java.util.Map;
|
||||
|
||||
public final class ExecutorNodeAdapter extends AbstractDiagnosisAgentNodeAdapter {
|
||||
|
||||
private final ExecutorEvidenceParser evidenceParser;
|
||||
|
||||
public ExecutorNodeAdapter(DiagnosisAgentInvoker invoker) {
|
||||
this(invoker, new ObjectMapper(), DiagnosisNodeFailureClassifier.defaults());
|
||||
}
|
||||
|
||||
public ExecutorNodeAdapter(
|
||||
DiagnosisAgentInvoker invoker,
|
||||
ObjectMapper objectMapper,
|
||||
DiagnosisNodeFailureClassifier failureClassifier) {
|
||||
super(
|
||||
DiagnosisGraphTopology.Node.EXECUTOR,
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT,
|
||||
invoker,
|
||||
objectMapper,
|
||||
failureClassifier);
|
||||
this.evidenceParser = new ExecutorEvidenceParser(objectMapper);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected Map<String, Object> projectInput(OverAllState state) {
|
||||
Map<String, Object> input = new LinkedHashMap<>();
|
||||
input.put("diagnosis_context", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.DIAGNOSIS_CONTEXT));
|
||||
input.put("planner_plan", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.PLANNER_PLAN));
|
||||
String plannerMode = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.PLANNER_MODE);
|
||||
input.put("planner_mode", plannerMode == null
|
||||
? PlannerMode.NORMAL.name()
|
||||
: plannerMode);
|
||||
state.value(DiagnosisGraphState.RETRY_CONTEXT)
|
||||
.ifPresent(value -> input.put("retry_context", value));
|
||||
if (PlannerMode.EVIDENCE_GAP_ONLY.name().equals(plannerMode)) {
|
||||
input.put("execution_constraints", Map.of(
|
||||
"only_execute_incremental_queries", true,
|
||||
"do_not_repeat_successful_queries", true,
|
||||
"preserve_prior_verified_claims", true,
|
||||
"return_complete_executor_evidence_v2_snapshot", true));
|
||||
}
|
||||
return input;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected NodeResult parseOutput(OverAllState state, String rawOutput) {
|
||||
ExecutorEvidenceParser.ParseResult parsed = evidenceParser.parse(rawOutput);
|
||||
Map<String, Object> structured = parsed.structuredOutput();
|
||||
boolean valid = "valid".equals(parsed.status().get("status"))
|
||||
&& structured != null
|
||||
&& "executor_evidence_v2".equals(
|
||||
String.valueOf(structured.get("answer_version")));
|
||||
if (!valid) {
|
||||
return new NodeResult(
|
||||
ExecutorStatus.INVALID_OUTPUT.name(),
|
||||
"executor_invalid_output",
|
||||
Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_STATUS,
|
||||
ExecutorStatus.INVALID_OUTPUT.name(),
|
||||
DiagnosisGraphState.FAILURE_REASON,
|
||||
"executor_invalid_output"));
|
||||
}
|
||||
return new NodeResult(
|
||||
ExecutorStatus.COMPLETED.name(),
|
||||
"executor_completed",
|
||||
Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_STATUS,
|
||||
ExecutorStatus.COMPLETED.name(),
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT,
|
||||
structured));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected NodeResult failureResult(Throwable failure, boolean retryable) {
|
||||
if (isToolBlocked(failure)) {
|
||||
return new NodeResult(
|
||||
ExecutorStatus.TOOL_BLOCKED.name(),
|
||||
"executor_tool_blocked",
|
||||
Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_STATUS,
|
||||
ExecutorStatus.TOOL_BLOCKED.name(),
|
||||
DiagnosisGraphState.FAILURE_REASON,
|
||||
"executor_tool_blocked"));
|
||||
}
|
||||
return super.failureResult(failure, retryable);
|
||||
}
|
||||
|
||||
private boolean isToolBlocked(Throwable failure) {
|
||||
Throwable current = failure;
|
||||
int depth = 0;
|
||||
while (current != null && depth++ < 8) {
|
||||
String message = current.getMessage();
|
||||
if (message != null) {
|
||||
String normalized = message.toLowerCase(Locale.ROOT);
|
||||
if (normalized.contains("tool")
|
||||
&& (normalized.contains("blocked")
|
||||
|| normalized.contains("denied")
|
||||
|| normalized.contains("not allowed"))) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
current = current.getCause();
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureStatus() {
|
||||
return ExecutorStatus.FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureStatus() {
|
||||
return ExecutorStatus.FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureReason() {
|
||||
return "executor_failed";
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureReason() {
|
||||
return "executor_failed";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.action.AsyncNodeActionWithConfig;
|
||||
import com.superbiz.agent.diagnosis.protocol.SafeFallbackRenderer;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerSafeInputBuilder;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
public final class FallbackNode implements AsyncNodeActionWithConfig {
|
||||
|
||||
private final SafeFallbackRenderer renderer;
|
||||
private final ComposerSafeInputBuilder inputBuilder;
|
||||
private final VerifierDecisionStateMapper stateMapper;
|
||||
|
||||
public FallbackNode() {
|
||||
this(new SafeFallbackRenderer(), new ComposerSafeInputBuilder());
|
||||
}
|
||||
|
||||
public FallbackNode(
|
||||
SafeFallbackRenderer renderer,
|
||||
ComposerSafeInputBuilder inputBuilder) {
|
||||
this.renderer = renderer;
|
||||
this.inputBuilder = inputBuilder;
|
||||
this.stateMapper = new VerifierDecisionStateMapper();
|
||||
}
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Map<String, Object>> apply(
|
||||
OverAllState state,
|
||||
RunnableConfig config) {
|
||||
String failureReason = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.FAILURE_REASON);
|
||||
Map<String, Object> safeInput = postVerification(state)
|
||||
? inputBuilder.build(
|
||||
stateMapper.originalQuery(state),
|
||||
stateMapper.decision(state),
|
||||
DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT))
|
||||
: preVerificationInput(failureReason);
|
||||
String answer = renderer.render(safeInput, List.of());
|
||||
|
||||
int attempt = DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1;
|
||||
Map<String, Object> update = new LinkedHashMap<>();
|
||||
update.put(DiagnosisGraphState.FINAL_ANSWER, answer);
|
||||
update.put(DiagnosisGraphState.ORCHESTRATION_EVENTS,
|
||||
List.of(new OrchestrationEvent(
|
||||
DiagnosisGraphTopology.Node.FALLBACK,
|
||||
"COMPLETED",
|
||||
DiagnosisGraphTopology.Reason.FALLBACK_COMPLETED,
|
||||
attempt)));
|
||||
return CompletableFuture.completedFuture(update);
|
||||
}
|
||||
|
||||
private boolean postVerification(OverAllState state) {
|
||||
return DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.COMPOSER_STATUS) != null
|
||||
&& !DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIER_OUTPUT).isEmpty();
|
||||
}
|
||||
|
||||
private Map<String, Object> preVerificationInput(String failureReason) {
|
||||
Map<String, Object> safeInput = new LinkedHashMap<>();
|
||||
safeInput.put("verdict", "REJECT");
|
||||
safeInput.put("allowed_claims", List.of());
|
||||
safeInput.put("allowed_hypotheses", List.of());
|
||||
safeInput.put("missing_info", List.of(
|
||||
failureReason == null
|
||||
? "当前编排未能建立可信诊断材料"
|
||||
: "当前编排未能建立可信诊断材料（" + failureReason + "）"));
|
||||
safeInput.put("recommended_actions", List.of(Map.of(
|
||||
"action_text", "查看当前 Run Trace",
|
||||
"reason", "确认失败节点并补充可信证据")));
|
||||
return safeInput;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.action.AsyncNodeActionWithConfig;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.GatekeeperStatus;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.Verdict;
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Objects;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
public final class GatekeeperNode implements AsyncNodeActionWithConfig {
|
||||
|
||||
private static final Map<String, Object> VALID_PARSE_STATUS = Map.of(
|
||||
"status", "valid",
|
||||
"detail", "graph executor completed");
|
||||
|
||||
private final ExecutorGatekeeperService gatekeeperService;
|
||||
|
||||
public GatekeeperNode(ExecutorGatekeeperService gatekeeperService) {
|
||||
this.gatekeeperService = Objects.requireNonNull(
|
||||
gatekeeperService, "gatekeeperService");
|
||||
}
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Map<String, Object>> apply(
|
||||
OverAllState state,
|
||||
RunnableConfig config) {
|
||||
String runId = config.metadata("runId")
|
||||
.map(Object::toString)
|
||||
.filter(value -> !value.isBlank())
|
||||
.orElse(null);
|
||||
Map<String, Object> executorOutput = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.EXECUTOR_OUTPUT);
|
||||
|
||||
Map<String, Object> rawResult;
|
||||
if (runId == null) {
|
||||
rawResult = internalReject("gatekeeper.run_id_missing");
|
||||
} else {
|
||||
try {
|
||||
rawResult = gatekeeperService.validateRun(
|
||||
runId, executorOutput, VALID_PARSE_STATUS);
|
||||
} catch (Exception exception) {
|
||||
rawResult = internalReject("gatekeeper.internal_error");
|
||||
}
|
||||
}
|
||||
|
||||
GatekeeperStatus status = normalize(rawResult);
|
||||
int passedBindings = countPassedBindings(rawResult);
|
||||
Verdict ceiling = status == GatekeeperStatus.PASS
|
||||
? Verdict.PASS
|
||||
: Verdict.LOW_CONFID;
|
||||
int attempt = DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1;
|
||||
|
||||
Map<String, Object> update = new LinkedHashMap<>();
|
||||
update.put(DiagnosisGraphState.GATEKEEPER_RESULT, rawResult);
|
||||
update.put(DiagnosisGraphState.GATEKEEPER_STATUS, status.name());
|
||||
update.put(DiagnosisGraphState.VERIFIED_BINDING_COUNT, passedBindings);
|
||||
update.put(DiagnosisGraphState.VERIFIER_VERDICT_CEILING, ceiling.name());
|
||||
if (status == GatekeeperStatus.REJECT) {
|
||||
update.put(DiagnosisGraphState.FAILURE_REASON, reasonCode(status));
|
||||
}
|
||||
update.put(DiagnosisGraphState.ORCHESTRATION_EVENTS,
|
||||
List.of(new OrchestrationEvent(
|
||||
DiagnosisGraphTopology.Node.GATEKEEPER,
|
||||
status.name(),
|
||||
reasonCode(status),
|
||||
attempt)));
|
||||
return CompletableFuture.completedFuture(update);
|
||||
}
|
||||
|
||||
private GatekeeperStatus normalize(Map<String, Object> rawResult) {
|
||||
if (rawResult == null) {
|
||||
return GatekeeperStatus.REJECT;
|
||||
}
|
||||
String status = String.valueOf(rawResult.getOrDefault("status", ""));
|
||||
String severity = String.valueOf(rawResult.getOrDefault("severity", ""));
|
||||
if (ExecutorGatekeeperService.STATUS_PASS.equals(status)
|
||||
&& ExecutorGatekeeperService.SEVERITY_NONE.equals(severity)) {
|
||||
return GatekeeperStatus.PASS;
|
||||
}
|
||||
if (ExecutorGatekeeperService.STATUS_FAIL.equals(status)
|
||||
&& ExecutorGatekeeperService.SEVERITY_LOW_CONFID.equals(severity)) {
|
||||
return GatekeeperStatus.LOW_CONFID;
|
||||
}
|
||||
return GatekeeperStatus.REJECT;
|
||||
}
|
||||
|
||||
private int countPassedBindings(Map<String, Object> rawResult) {
|
||||
if (rawResult == null
|
||||
|| !(rawResult.get("checked_bindings") instanceof List<?> bindings)) {
|
||||
return 0;
|
||||
}
|
||||
int count = 0;
|
||||
for (Object value : bindings) {
|
||||
if (value instanceof Map<?, ?> binding
|
||||
&& ExecutorGatekeeperService.STATUS_PASS.equals(
|
||||
String.valueOf(binding.get("status")))) {
|
||||
count++;
|
||||
}
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
private String reasonCode(GatekeeperStatus status) {
|
||||
return switch (status) {
|
||||
case PASS -> "gatekeeper_pass";
|
||||
case LOW_CONFID -> "gatekeeper_low_confid";
|
||||
case REJECT -> "gatekeeper_reject";
|
||||
};
|
||||
}
|
||||
|
||||
private Map<String, Object> internalReject(String rule) {
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("status", ExecutorGatekeeperService.STATUS_FAIL);
|
||||
result.put("severity", ExecutorGatekeeperService.SEVERITY_REJECT);
|
||||
result.put("checked_bindings", List.of());
|
||||
result.put("failed_rules", List.of(rule));
|
||||
result.put("warnings", List.of());
|
||||
result.put("errors", List.of(Map.of(
|
||||
"rule", rule,
|
||||
"message", "Gatekeeper could not validate current run")));
|
||||
return result;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.diagnosis.protocol.JsonPayloadSupport;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.PlannerMode;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.PlannerStatus;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
public final class PlannerNodeAdapter extends AbstractDiagnosisAgentNodeAdapter {
|
||||
|
||||
private static final TypeReference<Map<String, Object>> MAP_TYPE =
|
||||
new TypeReference<>() {
|
||||
};
|
||||
|
||||
public PlannerNodeAdapter(DiagnosisAgentInvoker invoker) {
|
||||
this(invoker, new ObjectMapper(), DiagnosisNodeFailureClassifier.defaults());
|
||||
}
|
||||
|
||||
public PlannerNodeAdapter(
|
||||
DiagnosisAgentInvoker invoker,
|
||||
ObjectMapper objectMapper,
|
||||
DiagnosisNodeFailureClassifier failureClassifier) {
|
||||
super(
|
||||
DiagnosisGraphTopology.Node.PLANNER,
|
||||
DiagnosisGraphState.PLANNER_RETRY_COUNT,
|
||||
invoker,
|
||||
objectMapper,
|
||||
failureClassifier);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected Map<String, Object> projectInput(OverAllState state) {
|
||||
Map<String, Object> input = new LinkedHashMap<>();
|
||||
input.put("diagnosis_context", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.DIAGNOSIS_CONTEXT));
|
||||
input.put("planner_mode", DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.PLANNER_MODE) == null
|
||||
? PlannerMode.NORMAL.name()
|
||||
: DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.PLANNER_MODE));
|
||||
state.value(DiagnosisGraphState.RETRY_CONTEXT)
|
||||
.ifPresent(value -> input.put("retry_context", value));
|
||||
return input;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected NodeResult parseOutput(OverAllState state, String rawOutput) {
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(
|
||||
JsonPayloadSupport.sanitizeObject(rawOutput));
|
||||
if (!root.isObject() || !root.path("plan").isArray()) {
|
||||
return invalidOutput();
|
||||
}
|
||||
Map<String, Object> plan = objectMapper.convertValue(root, MAP_TYPE);
|
||||
return new NodeResult(
|
||||
PlannerStatus.COMPLETED.name(),
|
||||
"planner_completed",
|
||||
Map.of(
|
||||
DiagnosisGraphState.PLANNER_STATUS,
|
||||
PlannerStatus.COMPLETED.name(),
|
||||
DiagnosisGraphState.PLANNER_PLAN,
|
||||
plan));
|
||||
} catch (Exception ignored) {
|
||||
return invalidOutput();
|
||||
}
|
||||
}
|
||||
|
||||
private NodeResult invalidOutput() {
|
||||
return new NodeResult(
|
||||
PlannerStatus.INVALID_OUTPUT.name(),
|
||||
"planner_invalid_output",
|
||||
Map.of(
|
||||
DiagnosisGraphState.PLANNER_STATUS,
|
||||
PlannerStatus.INVALID_OUTPUT.name(),
|
||||
DiagnosisGraphState.FAILURE_REASON,
|
||||
"planner_invalid_output"));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureStatus() {
|
||||
return PlannerStatus.RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureStatus() {
|
||||
return PlannerStatus.NON_RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureReason() {
|
||||
return "planner_retryable_failed";
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureReason() {
|
||||
return "planner_non_retryable_failed";
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.agent.ReactAgent;
|
||||
import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
|
||||
import java.util.Objects;
|
||||
|
||||
public final class ReactAgentDiagnosisInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final ReactAgent agent;
|
||||
|
||||
public ReactAgentDiagnosisInvoker(ReactAgent agent) {
|
||||
this.agent = Objects.requireNonNull(agent, "agent");
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) throws Exception {
|
||||
AssistantMessage message = agent.call(input, config);
|
||||
return Objects.requireNonNull(message, "assistant message").getText();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.action.AsyncNodeActionWithConfig;
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
public final class VerifiedInputNode implements AsyncNodeActionWithConfig {
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Map<String, Object>> apply(
|
||||
OverAllState state,
|
||||
RunnableConfig config) {
|
||||
Map<String, Object> executorOutput = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.EXECUTOR_OUTPUT);
|
||||
Map<String, Object> gatekeeperResult = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.GATEKEEPER_RESULT);
|
||||
Map<BindingKey, Map<String, Object>> passed = passedBindings(gatekeeperResult);
|
||||
|
||||
List<Map<String, Object>> verifiedClaims = new ArrayList<>();
|
||||
List<Map<String, Object>> verifiedEvidence = new ArrayList<>();
|
||||
Object claimsValue = executorOutput.get("claims");
|
||||
if (claimsValue instanceof List<?> claims) {
|
||||
for (Object claimValue : claims) {
|
||||
if (claimValue instanceof Map<?, ?> claim) {
|
||||
projectClaim(claim, passed, verifiedClaims, verifiedEvidence);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Map<String, Object> verifiedOutput = new LinkedHashMap<>();
|
||||
verifiedOutput.put("answer_version", executorOutput.getOrDefault(
|
||||
"answer_version", "executor_evidence_v2"));
|
||||
verifiedOutput.put("claims", verifiedClaims);
|
||||
|
||||
int attempt = DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1;
|
||||
Map<String, Object> update = new LinkedHashMap<>();
|
||||
update.put(DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT, verifiedOutput);
|
||||
update.put(DiagnosisGraphState.VERIFIED_EVIDENCE, verifiedEvidence);
|
||||
update.put(DiagnosisGraphState.VERIFIED_BINDING_COUNT,
|
||||
verifiedEvidence.size());
|
||||
update.put(DiagnosisGraphState.ORCHESTRATION_EVENTS,
|
||||
List.of(new OrchestrationEvent(
|
||||
DiagnosisGraphTopology.Node.VERIFIED_INPUT,
|
||||
"COMPLETED",
|
||||
"verified_input_completed",
|
||||
attempt)));
|
||||
return CompletableFuture.completedFuture(update);
|
||||
}
|
||||
|
||||
private Map<BindingKey, Map<String, Object>> passedBindings(
|
||||
Map<String, Object> gatekeeperResult) {
|
||||
Map<BindingKey, Map<String, Object>> result = new LinkedHashMap<>();
|
||||
Object bindingsValue = gatekeeperResult.get("checked_bindings");
|
||||
if (!(bindingsValue instanceof List<?> bindings)) {
|
||||
return result;
|
||||
}
|
||||
for (Object value : bindings) {
|
||||
if (!(value instanceof Map<?, ?> binding)
|
||||
|| !ExecutorGatekeeperService.STATUS_PASS.equals(
|
||||
text(binding.get("status")))
|
||||
|| text(binding.get("matched_text")).isBlank()) {
|
||||
continue;
|
||||
}
|
||||
BindingKey key = key(
|
||||
binding.get("claim_id"),
|
||||
binding.get("source_invocation_id"),
|
||||
binding.get("tool_name"),
|
||||
binding.get("raw_path"));
|
||||
result.putIfAbsent(key, copy(binding));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private void projectClaim(
|
||||
Map<?, ?> claim,
|
||||
Map<BindingKey, Map<String, Object>> passed,
|
||||
List<Map<String, Object>> verifiedClaims,
|
||||
List<Map<String, Object>> verifiedEvidence) {
|
||||
String claimId = text(claim.get("claim_id"));
|
||||
Object bindingsValue = claim.get("evidence_bindings");
|
||||
if (!(bindingsValue instanceof List<?> bindings)) {
|
||||
return;
|
||||
}
|
||||
List<Map<String, Object>> filteredBindings = new ArrayList<>();
|
||||
for (Object value : bindings) {
|
||||
if (!(value instanceof Map<?, ?> binding)) {
|
||||
continue;
|
||||
}
|
||||
BindingKey key = key(
|
||||
claimId,
|
||||
binding.get("source_invocation_id"),
|
||||
binding.get("tool_name"),
|
||||
binding.get("raw_path"));
|
||||
Map<String, Object> checked = passed.get(key);
|
||||
if (checked == null) {
|
||||
continue;
|
||||
}
|
||||
filteredBindings.add(copy(binding));
|
||||
Map<String, Object> evidence = new LinkedHashMap<>();
|
||||
evidence.put("claim_id", claimId);
|
||||
evidence.put("source_invocation_id", checked.get("source_invocation_id"));
|
||||
evidence.put("tool_name", checked.get("tool_name"));
|
||||
evidence.put("raw_path", checked.get("raw_path"));
|
||||
evidence.put("matched_text", checked.get("matched_text"));
|
||||
verifiedEvidence.add(evidence);
|
||||
}
|
||||
if (filteredBindings.isEmpty()) {
|
||||
return;
|
||||
}
|
||||
Map<String, Object> filteredClaim = new LinkedHashMap<>();
|
||||
filteredClaim.put("claim_id", claimId);
|
||||
filteredClaim.put("claim_type", text(claim.get("claim_type")));
|
||||
filteredClaim.put("claim_text", text(claim.get("claim_text")));
|
||||
filteredClaim.put("support_level", text(claim.get("support_level")));
|
||||
filteredClaim.put("evidence_bindings", filteredBindings);
|
||||
verifiedClaims.add(filteredClaim);
|
||||
}
|
||||
|
||||
private BindingKey key(
|
||||
Object claimId,
|
||||
Object invocationId,
|
||||
Object toolName,
|
||||
Object rawPath) {
|
||||
return new BindingKey(
|
||||
text(claimId),
|
||||
text(invocationId),
|
||||
text(toolName),
|
||||
text(rawPath));
|
||||
}
|
||||
|
||||
private Map<String, Object> copy(Map<?, ?> source) {
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
source.forEach((key, value) -> result.put(String.valueOf(key), value));
|
||||
return result;
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
return value == null ? "" : String.valueOf(value).trim();
|
||||
}
|
||||
|
||||
private record BindingKey(
|
||||
String claimId,
|
||||
String invocationId,
|
||||
String toolName,
|
||||
String rawPath) {
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.superbiz.agent.diagnosis.protocol.VerifierDecision;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
final class VerifierDecisionStateMapper {
|
||||
|
||||
VerifierDecision decision(OverAllState state) {
|
||||
Map<String, Object> output = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIER_OUTPUT);
|
||||
String verdict = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.EFFECTIVE_VERDICT);
|
||||
if (verdict == null) {
|
||||
verdict = text(output.getOrDefault("verdict", "LOW_CONFID"));
|
||||
}
|
||||
return new VerifierDecision(
|
||||
verdict,
|
||||
doubleValue(output.get("groundedness_score")),
|
||||
intValue(output.get("critical_fact_count")),
|
||||
maps(output.get("claim_checks")),
|
||||
maps(output.get("facts_checked")),
|
||||
text(output.get("rationale")),
|
||||
DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1);
|
||||
}
|
||||
|
||||
String originalQuery(OverAllState state) {
|
||||
Map<String, Object> context = DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.DIAGNOSIS_CONTEXT);
|
||||
String query = text(context.get("query"));
|
||||
return query.isBlank() ? text(context.get("original_query")) : query;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> maps(Object value) {
|
||||
if (!(value instanceof List<?> items)) {
|
||||
return List.of();
|
||||
}
|
||||
List<Map<String, Object>> result = new ArrayList<>();
|
||||
for (Object item : items) {
|
||||
if (item instanceof Map<?, ?> map) {
|
||||
Map<String, Object> copy = new LinkedHashMap<>();
|
||||
map.forEach((key, nested) -> copy.put(String.valueOf(key), nested));
|
||||
result.add(copy);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private double doubleValue(Object value) {
|
||||
if (value instanceof Number number) {
|
||||
return number.doubleValue();
|
||||
}
|
||||
try {
|
||||
return value == null ? 0.0 : Double.parseDouble(String.valueOf(value));
|
||||
} catch (NumberFormatException ignored) {
|
||||
return 0.0;
|
||||
}
|
||||
}
|
||||
|
||||
private int intValue(Object value) {
|
||||
if (value instanceof Number number) {
|
||||
return number.intValue();
|
||||
}
|
||||
try {
|
||||
return value == null ? 0 : Integer.parseInt(String.valueOf(value));
|
||||
} catch (NumberFormatException ignored) {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
private String text(Object value) {
|
||||
return value == null ? "" : String.valueOf(value);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,134 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.diagnosis.protocol.JsonPayloadSupport;
|
||||
import com.superbiz.agent.diagnosis.protocol.VerifierDecision;
|
||||
import com.superbiz.agent.diagnosis.protocol.VerifierOutputParser;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.Verdict;
|
||||
import com.superbiz.agent.graph.diagnosis.DiagnosisGraphStatus.VerifierStatus;
|
||||
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.Map;
|
||||
|
||||
public final class VerifierNodeAdapter extends AbstractDiagnosisAgentNodeAdapter {
|
||||
|
||||
private static final TypeReference<Map<String, Object>> MAP_TYPE =
|
||||
new TypeReference<>() {
|
||||
};
|
||||
|
||||
private final VerifierOutputParser outputParser;
|
||||
|
||||
public VerifierNodeAdapter(DiagnosisAgentInvoker invoker) {
|
||||
this(invoker, new ObjectMapper(), DiagnosisNodeFailureClassifier.defaults());
|
||||
}
|
||||
|
||||
public VerifierNodeAdapter(
|
||||
DiagnosisAgentInvoker invoker,
|
||||
ObjectMapper objectMapper,
|
||||
DiagnosisNodeFailureClassifier failureClassifier) {
|
||||
super(
|
||||
DiagnosisGraphTopology.Node.VERIFIER,
|
||||
DiagnosisGraphState.VERIFIER_RETRY_COUNT,
|
||||
invoker,
|
||||
objectMapper,
|
||||
failureClassifier);
|
||||
this.outputParser = new VerifierOutputParser(objectMapper);
|
||||
}
|
||||
|
||||
@Override
|
||||
protected Map<String, Object> projectInput(OverAllState state) {
|
||||
Map<String, Object> input = new LinkedHashMap<>();
|
||||
input.put("diagnosis_context", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.DIAGNOSIS_CONTEXT));
|
||||
input.put("verified_executor_output", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT));
|
||||
input.put("verified_evidence", DiagnosisGraphState.listValue(
|
||||
state, DiagnosisGraphState.VERIFIED_EVIDENCE));
|
||||
input.put("gatekeeper_audit", DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.GATEKEEPER_RESULT));
|
||||
input.put("verdict_ceiling", DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.VERIFIER_VERDICT_CEILING));
|
||||
state.value(DiagnosisGraphState.RETRY_CONTEXT)
|
||||
.ifPresent(value -> input.put("retry_context", value));
|
||||
return input;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected NodeResult parseOutput(OverAllState state, String rawOutput) {
|
||||
int round = DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT) + 1;
|
||||
VerifierDecision modelDecision = outputParser.parse(rawOutput, round);
|
||||
if (modelDecision == null) {
|
||||
return invalidOutput();
|
||||
}
|
||||
Map<String, Object> structured;
|
||||
try {
|
||||
structured = objectMapper.readValue(
|
||||
JsonPayloadSupport.sanitizeObject(rawOutput), MAP_TYPE);
|
||||
} catch (Exception ignored) {
|
||||
return invalidOutput();
|
||||
}
|
||||
structured.put("claim_checks", modelDecision.claimChecks());
|
||||
structured.put("facts_checked", modelDecision.factsChecked());
|
||||
structured.put("verdict", modelDecision.verdict());
|
||||
|
||||
VerifierDecision effectiveDecision = outputParser.applyCeiling(
|
||||
modelDecision,
|
||||
"valid",
|
||||
DiagnosisGraphState.mapValue(
|
||||
state, DiagnosisGraphState.GATEKEEPER_RESULT));
|
||||
String ceiling = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.VERIFIER_VERDICT_CEILING);
|
||||
if (Verdict.LOW_CONFID.name().equals(ceiling)
|
||||
&& Verdict.PASS.name().equals(effectiveDecision.verdict())) {
|
||||
effectiveDecision = effectiveDecision.withVerdict(
|
||||
Verdict.LOW_CONFID.name());
|
||||
}
|
||||
|
||||
Map<String, Object> values = new LinkedHashMap<>();
|
||||
values.put(DiagnosisGraphState.VERIFIER_STATUS,
|
||||
VerifierStatus.COMPLETED.name());
|
||||
values.put(DiagnosisGraphState.VERIFIER_OUTPUT, structured);
|
||||
values.put(DiagnosisGraphState.VERIFIER_MODEL_VERDICT,
|
||||
modelDecision.verdict());
|
||||
values.put(DiagnosisGraphState.EFFECTIVE_VERDICT,
|
||||
effectiveDecision.verdict());
|
||||
return new NodeResult(
|
||||
VerifierStatus.COMPLETED.name(),
|
||||
"verifier_completed_" + effectiveDecision.verdict().toLowerCase(),
|
||||
values);
|
||||
}
|
||||
|
||||
private NodeResult invalidOutput() {
|
||||
return new NodeResult(
|
||||
VerifierStatus.INVALID_OUTPUT.name(),
|
||||
"verifier_invalid_output",
|
||||
Map.of(
|
||||
DiagnosisGraphState.VERIFIER_STATUS,
|
||||
VerifierStatus.INVALID_OUTPUT.name(),
|
||||
DiagnosisGraphState.FAILURE_REASON,
|
||||
"verifier_invalid_output"));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureStatus() {
|
||||
return VerifierStatus.RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureStatus() {
|
||||
return VerifierStatus.NON_RETRYABLE_FAILED.name();
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String retryableFailureReason() {
|
||||
return "verifier_retryable_failed";
|
||||
}
|
||||
|
||||
@Override
|
||||
protected String nonRetryableFailureReason() {
|
||||
return "verifier_non_retryable_failed";
|
||||
}
|
||||
}
|
||||
@@ -5,9 +5,8 @@ import com.alibaba.cloud.ai.graph.agent.hook.HookPosition;
|
||||
import com.alibaba.cloud.ai.graph.agent.hook.HookPositions;
|
||||
import com.alibaba.cloud.ai.graph.agent.hook.messages.AgentCommand;
|
||||
import com.alibaba.cloud.ai.graph.agent.hook.messages.MessagesModelHook;
|
||||
import com.fasterxml.jackson.core.type.TypeReference;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.diagnosis.protocol.ExecutorEvidenceParser;
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
import com.superbiz.agent.service.GatekeeperRuleCatalog;
|
||||
import com.superbiz.agent.service.ToolTraceSummaryService;
|
||||
@@ -18,12 +17,9 @@ import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
import org.springframework.ai.chat.messages.Message;
|
||||
import org.springframework.ai.chat.messages.UserMessage;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.LinkedHashSet;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Set;
|
||||
|
||||
/**
|
||||
* Replaces verifier history with an explicit structured payload.
|
||||
@@ -34,9 +30,8 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
|
||||
private final ToolTraceSummaryService toolTraceSummaryService;
|
||||
private final ExecutorGatekeeperService executorGatekeeperService;
|
||||
private final ExecutorEvidenceParser executorEvidenceParser;
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private static final TypeReference<Map<String, Object>> MAP_TYPE = new TypeReference<>() {
|
||||
};
|
||||
|
||||
public VerifierInputHook(ToolTraceSummaryService toolTraceSummaryService) {
|
||||
this(toolTraceSummaryService, null);
|
||||
@@ -44,8 +39,16 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
|
||||
public VerifierInputHook(ToolTraceSummaryService toolTraceSummaryService,
|
||||
ExecutorGatekeeperService executorGatekeeperService) {
|
||||
this(toolTraceSummaryService, executorGatekeeperService,
|
||||
new ExecutorEvidenceParser());
|
||||
}
|
||||
|
||||
VerifierInputHook(ToolTraceSummaryService toolTraceSummaryService,
|
||||
ExecutorGatekeeperService executorGatekeeperService,
|
||||
ExecutorEvidenceParser executorEvidenceParser) {
|
||||
this.toolTraceSummaryService = toolTraceSummaryService;
|
||||
this.executorGatekeeperService = executorGatekeeperService;
|
||||
this.executorEvidenceParser = executorEvidenceParser;
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -72,11 +75,10 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
: toolTraceSummaryService.buildVerifierTraceSummaryForRun(runId, executorFinalAnswer);
|
||||
VerifierContextHolder.setToolTraceSummary(toolTraceSummary);
|
||||
|
||||
ExecutorOutputParseResult parseResult = parseExecutorOutput(executorFinalAnswer);
|
||||
parseResult = new ExecutorOutputParseResult(
|
||||
enrichExecutorStructuredOutput(parseResult.structuredOutput(), toolTraceSummary),
|
||||
parseResult.status()
|
||||
);
|
||||
ExecutorEvidenceParser.ParseResult parseResult =
|
||||
executorEvidenceParser.enrich(
|
||||
executorEvidenceParser.parse(executorFinalAnswer),
|
||||
toolTraceSummary);
|
||||
VerifierContextHolder.setExecutorStructuredOutput(parseResult.structuredOutput());
|
||||
VerifierContextHolder.setExecutorOutputParseStatus(parseResult.status());
|
||||
|
||||
@@ -100,7 +102,8 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, Object> runGatekeeper(String sessionId, String runId, ExecutorOutputParseResult parseResult) {
|
||||
private Map<String, Object> runGatekeeper(String sessionId, String runId,
|
||||
ExecutorEvidenceParser.ParseResult parseResult) {
|
||||
if (executorGatekeeperService == null) {
|
||||
return passGatekeeperResult();
|
||||
}
|
||||
@@ -131,182 +134,6 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
return result;
|
||||
}
|
||||
|
||||
private ExecutorOutputParseResult parseExecutorOutput(String executorFinalAnswer) {
|
||||
if (executorFinalAnswer == null || executorFinalAnswer.isBlank()) {
|
||||
return new ExecutorOutputParseResult(null, status("missing", "executor_final_answer is blank"));
|
||||
}
|
||||
|
||||
String sanitized = sanitizeJsonPayload(executorFinalAnswer);
|
||||
if (!looksJsonLike(sanitized)) {
|
||||
return new ExecutorOutputParseResult(null, status("missing", "executor output is not JSON"));
|
||||
}
|
||||
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(sanitized);
|
||||
if (!root.isObject() || !root.path("claims").isArray()) {
|
||||
return new ExecutorOutputParseResult(null, status("malformed",
|
||||
"executor output JSON does not match evidence-attribution contract"));
|
||||
}
|
||||
Map<String, Object> structuredOutput = objectMapper.convertValue(root, MAP_TYPE);
|
||||
return new ExecutorOutputParseResult(structuredOutput, status("valid", "parsed executor evidence contract"));
|
||||
} catch (Exception e) {
|
||||
log.debug("Failed to parse executor structured output", e);
|
||||
return new ExecutorOutputParseResult(null, status("malformed", e.getMessage()));
|
||||
}
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
private Map<String, Object> enrichExecutorStructuredOutput(Map<String, Object> structuredOutput,
|
||||
List<Map<String, Object>> toolTraceSummary) {
|
||||
if (structuredOutput == null) {
|
||||
return null;
|
||||
}
|
||||
Map<String, List<Long>> invocationIdsByTool = invocationIdsByTool(toolTraceSummary);
|
||||
List<Map<String, Object>> warnings = new ArrayList<>();
|
||||
enrichEvidenceBindingsInSection(structuredOutput.get("claims"), invocationIdsByTool, warnings);
|
||||
enrichEvidenceBindingsInSection(structuredOutput.get("recommended_actions"), invocationIdsByTool, warnings);
|
||||
if (!warnings.isEmpty()) {
|
||||
structuredOutput.put("_gatekeeper_warnings", warnings);
|
||||
}
|
||||
return structuredOutput;
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
private void enrichEvidenceBindingsInSection(Object sectionValue,
|
||||
Map<String, List<Long>> invocationIdsByTool,
|
||||
List<Map<String, Object>> warnings) {
|
||||
if (!(sectionValue instanceof List<?> items)) {
|
||||
return;
|
||||
}
|
||||
for (Object itemValue : items) {
|
||||
if (!(itemValue instanceof Map<?, ?> item)) {
|
||||
continue;
|
||||
}
|
||||
Object bindingsValue = item.get("evidence_bindings");
|
||||
if (!(bindingsValue instanceof List<?> bindings)) {
|
||||
continue;
|
||||
}
|
||||
for (Object bindingValue : bindings) {
|
||||
if (!(bindingValue instanceof Map<?, ?> rawBinding)) {
|
||||
continue;
|
||||
}
|
||||
Map<String, Object> binding = (Map<String, Object>) rawBinding;
|
||||
String normalizedToolName = normalizeToolName(binding.get("tool_name"));
|
||||
if (!normalizedToolName.isBlank()) {
|
||||
binding.put("tool_name", normalizedToolName);
|
||||
}
|
||||
if (!hasInvocationId(binding)) {
|
||||
List<Long> ids = invocationIdsByTool.getOrDefault(normalizedToolName, List.of());
|
||||
if (ids.size() == 1) {
|
||||
binding.put("source_invocation_id", ids.get(0));
|
||||
warnings.add(Map.of(
|
||||
"rule", "evidence.invocation_auto_backfill",
|
||||
"message", "source_invocation_id was auto-filled from the unique tool invocation candidate; raw_path remains missing if Executor did not provide it",
|
||||
"tool_name", normalizedToolName,
|
||||
"source_invocation_id", ids.get(0)
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private Map<String, List<Long>> invocationIdsByTool(List<Map<String, Object>> toolTraceSummary) {
|
||||
Map<String, Set<Long>> idsByTool = new LinkedHashMap<>();
|
||||
for (Map<String, Object> summary : toolTraceSummary == null ? List.<Map<String, Object>>of() : toolTraceSummary) {
|
||||
String toolName = normalizeToolName(summary.get("tool_name"));
|
||||
if (toolName.isBlank()) {
|
||||
continue;
|
||||
}
|
||||
List<Long> ids = toLongList(summary.get("source_invocation_ids"));
|
||||
if (ids.isEmpty()) {
|
||||
continue;
|
||||
}
|
||||
idsByTool.computeIfAbsent(toolName, ignored -> new LinkedHashSet<>()).addAll(ids);
|
||||
}
|
||||
|
||||
Map<String, List<Long>> result = new LinkedHashMap<>();
|
||||
for (Map.Entry<String, Set<Long>> entry : idsByTool.entrySet()) {
|
||||
result.put(entry.getKey(), new ArrayList<>(entry.getValue()));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private boolean hasInvocationId(Map<String, Object> binding) {
|
||||
if (asLong(binding.get("source_invocation_id")) != null) {
|
||||
return true;
|
||||
}
|
||||
return toLongList(binding.get("source_invocation_ids")).size() == 1;
|
||||
}
|
||||
|
||||
private List<Long> toLongList(Object value) {
|
||||
if (!(value instanceof List<?> values)) {
|
||||
return List.of();
|
||||
}
|
||||
List<Long> ids = new ArrayList<>();
|
||||
for (Object item : values) {
|
||||
Long id = asLong(item);
|
||||
if (id != null) {
|
||||
ids.add(id);
|
||||
}
|
||||
}
|
||||
return ids;
|
||||
}
|
||||
|
||||
private Long asLong(Object value) {
|
||||
if (value instanceof Number number) {
|
||||
return number.longValue();
|
||||
}
|
||||
if (value instanceof String text) {
|
||||
try {
|
||||
return Long.parseLong(text);
|
||||
} catch (NumberFormatException ignored) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private String normalizeToolName(Object value) {
|
||||
String toolName = value == null ? "" : String.valueOf(value);
|
||||
return switch (toolName) {
|
||||
case "lookupKnowledge" -> "lookup_knowledge";
|
||||
case "queryLogs" -> "query_logs";
|
||||
case "queryPrometheusAlerts" -> "query_metrics";
|
||||
case "getAvailableLogTopics" -> "get_available_log_topics";
|
||||
default -> toolName;
|
||||
};
|
||||
}
|
||||
|
||||
private String sanitizeJsonPayload(String raw) {
|
||||
String trimmed = raw.trim();
|
||||
int fenceStart = trimmed.indexOf("```");
|
||||
if (fenceStart >= 0) {
|
||||
int firstNewline = trimmed.indexOf('\n', fenceStart);
|
||||
int lastFence = trimmed.indexOf("```", firstNewline + 1);
|
||||
if (firstNewline >= 0 && lastFence > firstNewline) {
|
||||
return trimmed.substring(firstNewline + 1, lastFence).trim();
|
||||
}
|
||||
}
|
||||
int objectStart = trimmed.indexOf('{');
|
||||
int objectEnd = trimmed.lastIndexOf('}');
|
||||
if (objectStart >= 0 && objectEnd > objectStart) {
|
||||
return trimmed.substring(objectStart, objectEnd + 1).trim();
|
||||
}
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
private boolean looksJsonLike(String text) {
|
||||
return text.startsWith("{") && text.endsWith("}");
|
||||
}
|
||||
|
||||
private Map<String, Object> status(String status, String detail) {
|
||||
Map<String, Object> result = new LinkedHashMap<>();
|
||||
result.put("status", status);
|
||||
result.put("detail", detail == null ? "" : detail);
|
||||
return result;
|
||||
}
|
||||
|
||||
private String extractLastAssistantText(List<Message> previousMessages) {
|
||||
for (int i = previousMessages.size() - 1; i >= 0; i--) {
|
||||
if (previousMessages.get(i) instanceof AssistantMessage assistantMessage) {
|
||||
@@ -344,9 +171,4 @@ public class VerifierInputHook extends MessagesModelHook {
|
||||
return message.toString();
|
||||
}
|
||||
|
||||
private record ExecutorOutputParseResult(
|
||||
Map<String, Object> structuredOutput,
|
||||
Map<String, Object> status
|
||||
) {
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,12 +8,16 @@ import com.alibaba.cloud.ai.graph.agent.hook.Hook;
|
||||
import com.alibaba.cloud.ai.graph.agent.hook.skills.SkillsAgentHook;
|
||||
import com.alibaba.cloud.ai.graph.exception.GraphRunnerException;
|
||||
import com.alibaba.cloud.ai.graph.skills.registry.SkillRegistry;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.superbiz.agent.agent.tool.DateTimeTools;
|
||||
import com.superbiz.agent.agent.tool.InternalDocsTools;
|
||||
import com.superbiz.agent.agent.tool.QueryLogsTools;
|
||||
import com.superbiz.agent.agent.tool.QueryMetricsTools;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerOutputParser;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerRenderResult;
|
||||
import com.superbiz.agent.diagnosis.protocol.ComposerSafeInputBuilder;
|
||||
import com.superbiz.agent.diagnosis.protocol.VerifierDecision;
|
||||
import com.superbiz.agent.diagnosis.protocol.VerifierOutputParser;
|
||||
import com.superbiz.agent.domain.entity.ChatSession;
|
||||
import com.superbiz.agent.domain.entity.DiagnosisRun;
|
||||
import com.superbiz.agent.hook.AgentLoggingHook;
|
||||
@@ -61,8 +65,6 @@ import java.util.UUID;
|
||||
public class ChatService {
|
||||
|
||||
private static final Logger logger = LoggerFactory.getLogger(ChatService.class);
|
||||
private static final String LOW_CONFID_DISCLAIMER = "以下结论基于当前已获取证据，仍存在部分证据缺口，请谨慎参考。";
|
||||
private static final String DEGRADED_PREFIX = "当前无法基于已获取证据生成可靠结论，建议人工介入。";
|
||||
private static final String CHAT_PROMPT_AUDIT_VERSION = "chat-prompts-v1";
|
||||
|
||||
/** 封装 answer + 后端生成的 sessionId/runId，用于 feedback 关联 */
|
||||
@@ -134,6 +136,12 @@ public class ChatService {
|
||||
private String chatVerifierPrompt;
|
||||
private String chatComposerPrompt;
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private final VerifierOutputParser verifierOutputParser =
|
||||
new VerifierOutputParser(objectMapper);
|
||||
private final ComposerSafeInputBuilder composerSafeInputBuilder =
|
||||
new ComposerSafeInputBuilder();
|
||||
private final ComposerOutputParser composerOutputParser =
|
||||
new ComposerOutputParser();
|
||||
|
||||
@PostConstruct
|
||||
public void init() {
|
||||
@@ -697,197 +705,25 @@ public class ChatService {
|
||||
}
|
||||
|
||||
private VerifierDecision parseVerifierDecision(String verifierOutput, int round) {
|
||||
if (verifierOutput == null || verifierOutput.isBlank()) {
|
||||
VerifierDecision modelDecision = verifierOutputParser.parse(
|
||||
verifierOutput, round);
|
||||
if (modelDecision == null) {
|
||||
logger.error("解析 verifier_output 失败: {}", verifierOutput);
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(sanitizeJsonPayload(verifierOutput));
|
||||
List<Map<String, Object>> claimChecks = parseClaimChecks(root.path("claim_checks"));
|
||||
List<Map<String, Object>> factsChecked = claimChecks.isEmpty()
|
||||
? parseFactsChecked(root.path("facts_checked"))
|
||||
: mapClaimChecksToFactsChecked(claimChecks);
|
||||
String verdict = effectiveVerifierVerdict(root.path("verdict").asText("LOW_CONFID"));
|
||||
int criticalFactCount = root.path("critical_fact_count").asInt(countCriticalFacts(factsChecked));
|
||||
|
||||
return new VerifierDecision(
|
||||
verdict,
|
||||
root.path("groundedness_score").asDouble(0.0),
|
||||
criticalFactCount,
|
||||
claimChecks,
|
||||
factsChecked,
|
||||
root.path("rationale").asText(""),
|
||||
round
|
||||
);
|
||||
} catch (Exception e) {
|
||||
logger.error("解析 verifier_output 失败: {}", verifierOutput, e);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
private String sanitizeJsonPayload(String raw) {
|
||||
String trimmed = raw.trim();
|
||||
if (trimmed.startsWith("```")) {
|
||||
int firstNewline = trimmed.indexOf('\n');
|
||||
int lastFence = trimmed.lastIndexOf("```");
|
||||
if (firstNewline >= 0 && lastFence > firstNewline) {
|
||||
return trimmed.substring(firstNewline + 1, lastFence).trim();
|
||||
}
|
||||
}
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
private String effectiveVerifierVerdict(String modelVerdict) {
|
||||
String verdict = normalizeVerdict(modelVerdict);
|
||||
Map<String, Object> parseStatus = VerifierContextHolder.getExecutorOutputParseStatus();
|
||||
String parseState = parseStatus == null ? "" : String.valueOf(parseStatus.getOrDefault("status", ""));
|
||||
if (("missing".equals(parseState) || "malformed".equals(parseState)) && "PASS".equals(verdict)) {
|
||||
return "LOW_CONFID";
|
||||
}
|
||||
|
||||
Map<String, Object> gatekeeperResult = VerifierContextHolder.getGatekeeperResult();
|
||||
if (gatekeeperResult == null || !"fail".equals(String.valueOf(gatekeeperResult.get("status")))) {
|
||||
return verdict;
|
||||
}
|
||||
String severity = String.valueOf(gatekeeperResult.getOrDefault("severity", ""));
|
||||
if (ExecutorGatekeeperService.SEVERITY_REJECT.equals(severity)) {
|
||||
return "REJECT";
|
||||
}
|
||||
if (ExecutorGatekeeperService.SEVERITY_LOW_CONFID.equals(severity)) {
|
||||
return "PASS".equals(verdict) ? "LOW_CONFID" : verdict;
|
||||
}
|
||||
if (containsRule(gatekeeperResult.get("failed_rules"), ExecutorGatekeeperService.RULE_INVOCATION_REF)) {
|
||||
return "REJECT";
|
||||
}
|
||||
return "PASS".equals(verdict) ? "LOW_CONFID" : verdict;
|
||||
}
|
||||
|
||||
private String normalizeVerdict(String verdict) {
|
||||
if ("PASS".equals(verdict) || "LOW_CONFID".equals(verdict) || "REJECT".equals(verdict)) {
|
||||
return verdict;
|
||||
}
|
||||
return "LOW_CONFID";
|
||||
}
|
||||
|
||||
private boolean containsRule(Object rulesValue, String ruleId) {
|
||||
if (!(rulesValue instanceof List<?> rules)) {
|
||||
return false;
|
||||
}
|
||||
return rules.stream().anyMatch(rule -> ruleId.equals(String.valueOf(rule)));
|
||||
}
|
||||
|
||||
private int countCriticalFacts(List<Map<String, Object>> factsChecked) {
|
||||
return (int) factsChecked.stream()
|
||||
.filter(fact -> Boolean.TRUE.equals(fact.get("is_critical")))
|
||||
.count();
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseClaimChecks(JsonNode claimChecksNode) {
|
||||
List<Map<String, Object>> claimChecks = new ArrayList<>();
|
||||
if (!claimChecksNode.isArray()) {
|
||||
return claimChecks;
|
||||
}
|
||||
for (JsonNode claimNode : claimChecksNode) {
|
||||
Map<String, Object> claimCheck = new LinkedHashMap<>();
|
||||
claimCheck.put("claim_id", claimNode.path("claim_id").asText(""));
|
||||
claimCheck.put("claim_text", claimNode.path("claim_text").asText(""));
|
||||
claimCheck.put("claim_type", claimNode.path("claim_type").asText(""));
|
||||
claimCheck.put("verification", normalizeClaimVerification(claimNode.path("verification").asText("unsupported")));
|
||||
claimCheck.put("detail", claimNode.path("detail").asText(""));
|
||||
claimCheck.put("evidence_refs", parseEvidenceRefs(claimNode.path("evidence_refs")));
|
||||
claimChecks.add(claimCheck);
|
||||
}
|
||||
return claimChecks;
|
||||
}
|
||||
|
||||
private String normalizeClaimVerification(String verification) {
|
||||
return switch (verification) {
|
||||
case "direct_observation", "reasonable_inference", "overstated", "unsupported",
|
||||
"external_unknown", "contradicted" -> verification;
|
||||
default -> "unsupported";
|
||||
};
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> mapClaimChecksToFactsChecked(List<Map<String, Object>> claimChecks) {
|
||||
List<Map<String, Object>> factsChecked = new ArrayList<>();
|
||||
for (Map<String, Object> claimCheck : claimChecks) {
|
||||
String claimId = String.valueOf(claimCheck.getOrDefault("claim_id", ""));
|
||||
String claimText = String.valueOf(claimCheck.getOrDefault("claim_text", ""));
|
||||
String claimType = String.valueOf(claimCheck.getOrDefault("claim_type", ""));
|
||||
Map<String, Object> fact = new LinkedHashMap<>();
|
||||
fact.put("fact", claimId.isBlank() ? claimText : claimId + ": " + claimText);
|
||||
fact.put("is_critical", isCriticalClaimType(claimType));
|
||||
fact.put("verification", mapClaimVerificationToFactVerification(
|
||||
String.valueOf(claimCheck.getOrDefault("verification", "unsupported"))));
|
||||
fact.put("detail", claimCheck.getOrDefault("detail", ""));
|
||||
fact.put("evidence_refs", claimCheck.getOrDefault("evidence_refs", List.of()));
|
||||
factsChecked.add(fact);
|
||||
}
|
||||
return factsChecked;
|
||||
}
|
||||
|
||||
private boolean isCriticalClaimType(String claimType) {
|
||||
return "root_cause".equals(claimType)
|
||||
|| "symptom".equals(claimType)
|
||||
|| "impact".equals(claimType)
|
||||
|| "risk".equals(claimType);
|
||||
}
|
||||
|
||||
private String mapClaimVerificationToFactVerification(String verification) {
|
||||
return switch (verification) {
|
||||
case "direct_observation" -> "direct_evidence";
|
||||
case "reasonable_inference", "overstated" -> "indirect_support";
|
||||
case "contradicted" -> "contradicted";
|
||||
default -> "no_evidence";
|
||||
};
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseFactsChecked(JsonNode factsNode) {
|
||||
List<Map<String, Object>> factsChecked = new ArrayList<>();
|
||||
if (!factsNode.isArray()) {
|
||||
return factsChecked;
|
||||
}
|
||||
for (JsonNode factNode : factsNode) {
|
||||
Map<String, Object> fact = new LinkedHashMap<>();
|
||||
fact.put("fact", factNode.path("fact").asText(""));
|
||||
fact.put("is_critical", factNode.path("is_critical").asBoolean(false));
|
||||
fact.put("verification", factNode.path("verification").asText(""));
|
||||
fact.put("detail", factNode.path("detail").asText(""));
|
||||
fact.put("evidence_refs", parseEvidenceRefs(factNode.path("evidence_refs")));
|
||||
factsChecked.add(fact);
|
||||
}
|
||||
return factsChecked;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseEvidenceRefs(JsonNode evidenceRefsNode) {
|
||||
List<Map<String, Object>> evidenceRefs = new ArrayList<>();
|
||||
if (!evidenceRefsNode.isArray()) {
|
||||
return evidenceRefs;
|
||||
}
|
||||
for (JsonNode refNode : evidenceRefsNode) {
|
||||
Map<String, Object> evidenceRef = new LinkedHashMap<>();
|
||||
evidenceRef.put("trace_ref", refNode.path("trace_ref").asText(""));
|
||||
evidenceRef.put("tool_name", refNode.path("tool_name").asText(""));
|
||||
evidenceRef.put("topic_domain", refNode.path("topic_domain").asText(""));
|
||||
evidenceRef.put("note", refNode.path("note").asText(""));
|
||||
|
||||
List<Long> sourceInvocationIds = new ArrayList<>();
|
||||
JsonNode idsNode = refNode.path("source_invocation_ids");
|
||||
if (idsNode.isArray()) {
|
||||
for (JsonNode idNode : idsNode) {
|
||||
if (idNode.canConvertToLong()) {
|
||||
sourceInvocationIds.add(idNode.asLong());
|
||||
}
|
||||
}
|
||||
}
|
||||
evidenceRef.put("source_invocation_ids", sourceInvocationIds);
|
||||
evidenceRefs.add(evidenceRef);
|
||||
}
|
||||
return evidenceRefs;
|
||||
Map<String, Object> parseStatus =
|
||||
VerifierContextHolder.getExecutorOutputParseStatus();
|
||||
String parseState = parseStatus == null
|
||||
? ""
|
||||
: String.valueOf(parseStatus.getOrDefault("status", ""));
|
||||
return verifierOutputParser.applyCeiling(
|
||||
modelDecision,
|
||||
parseState,
|
||||
VerifierContextHolder.getGatekeeperResult());
|
||||
}
|
||||
|
||||
private VerifierDecision buildVerifierFallbackDecision(int round, String rationale) {
|
||||
return new VerifierDecision("LOW_CONFID", 0.0, 0, List.of(), List.of(), rationale, round);
|
||||
return verifierOutputParser.fallback(round, rationale);
|
||||
}
|
||||
|
||||
private String extractStateText(Optional<OverAllState> stateOptional, String key) {
|
||||
@@ -993,290 +829,24 @@ public class ChatService {
|
||||
}
|
||||
|
||||
private Map<String, Object> buildComposerInput(String originalQuery, VerifierDecision decision) {
|
||||
Map<String, Object> input = new LinkedHashMap<>();
|
||||
input.put("original_query", originalQuery);
|
||||
input.put("verdict", decision.verdict());
|
||||
|
||||
Map<String, Map<String, Object>> claimsById = indexExecutorClaims();
|
||||
List<Map<String, Object>> allowedClaims = new ArrayList<>();
|
||||
List<Map<String, Object>> allowedHypotheses = new ArrayList<>();
|
||||
List<String> missingInfo = extractStructuredMissingInfo();
|
||||
List<Map<String, Object>> recommendedActions = extractStructuredRecommendedActions();
|
||||
|
||||
if (decision.claimChecks().isEmpty()) {
|
||||
addLegacyFactsToComposerInput(decision, allowedClaims, allowedHypotheses, missingInfo);
|
||||
} else {
|
||||
for (Map<String, Object> check : decision.claimChecks()) {
|
||||
String verification = String.valueOf(check.getOrDefault("verification", "unsupported"));
|
||||
String claimId = String.valueOf(check.getOrDefault("claim_id", ""));
|
||||
String claimText = String.valueOf(check.getOrDefault("claim_text", ""));
|
||||
String detail = String.valueOf(check.getOrDefault("detail", ""));
|
||||
Map<String, Object> claim = buildComposerClaim(claimsById.get(claimId), check);
|
||||
|
||||
switch (verification) {
|
||||
case "direct_observation", "reasonable_inference" -> allowedClaims.add(claim);
|
||||
case "overstated" -> allowedHypotheses.add(Map.of(
|
||||
"hypothesis_text", claimText,
|
||||
"basis", detail.isBlank() ? "当前证据只能支持部分判断，不能作为确认结论" : detail
|
||||
));
|
||||
case "unsupported", "external_unknown", "contradicted" ->
|
||||
addMissingInfo(missingInfo, claimText, detail);
|
||||
default -> addMissingInfo(missingInfo, claimText, detail);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ("REJECT".equals(decision.verdict())) {
|
||||
allowedHypotheses = List.of();
|
||||
}
|
||||
|
||||
input.put("allowed_claims", allowedClaims);
|
||||
input.put("allowed_hypotheses", allowedHypotheses);
|
||||
input.put("missing_info", missingInfo);
|
||||
input.put("recommended_actions", recommendedActions);
|
||||
input.put("rationale", decision.rationale());
|
||||
return input;
|
||||
}
|
||||
|
||||
private Map<String, Map<String, Object>> indexExecutorClaims() {
|
||||
Map<String, Map<String, Object>> claimsById = new LinkedHashMap<>();
|
||||
Map<String, Object> structuredOutput = VerifierContextHolder.getExecutorStructuredOutput();
|
||||
Object claims = structuredOutput == null ? null : structuredOutput.get("claims");
|
||||
if (claims instanceof List<?> claimList) {
|
||||
for (Object item : claimList) {
|
||||
if (item instanceof Map<?, ?> rawClaim) {
|
||||
Map<String, Object> claim = new LinkedHashMap<>();
|
||||
rawClaim.forEach((key, value) -> claim.put(String.valueOf(key), value));
|
||||
String claimId = String.valueOf(claim.getOrDefault("claim_id", ""));
|
||||
if (!claimId.isBlank()) {
|
||||
claimsById.put(claimId, claim);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return claimsById;
|
||||
}
|
||||
|
||||
private Map<String, Object> buildComposerClaim(Map<String, Object> executorClaim, Map<String, Object> claimCheck) {
|
||||
Map<String, Object> claim = new LinkedHashMap<>();
|
||||
claim.put("claim_id", valueFrom(executorClaim, claimCheck, "claim_id"));
|
||||
claim.put("claim_type", valueFrom(executorClaim, claimCheck, "claim_type"));
|
||||
claim.put("claim_text", valueFrom(executorClaim, claimCheck, "claim_text"));
|
||||
claim.put("support_level", executorClaim == null ? "" : String.valueOf(executorClaim.getOrDefault("support_level", "")));
|
||||
claim.put("verification", String.valueOf(claimCheck.getOrDefault("verification", "")));
|
||||
claim.put("detail", String.valueOf(claimCheck.getOrDefault("detail", "")));
|
||||
return claim;
|
||||
}
|
||||
|
||||
private String valueFrom(Map<String, Object> primary, Map<String, Object> fallback, String key) {
|
||||
Object value = primary == null ? null : primary.get(key);
|
||||
if (value == null || String.valueOf(value).isBlank()) {
|
||||
value = fallback.get(key);
|
||||
}
|
||||
return value == null ? "" : String.valueOf(value);
|
||||
}
|
||||
|
||||
private List<String> extractStructuredMissingInfo() {
|
||||
List<String> missingInfo = new ArrayList<>();
|
||||
Map<String, Object> structuredOutput = VerifierContextHolder.getExecutorStructuredOutput();
|
||||
Object missing = structuredOutput == null ? null : structuredOutput.get("missing_info");
|
||||
if (missing instanceof List<?> missingList) {
|
||||
for (Object item : missingList) {
|
||||
String text = String.valueOf(item);
|
||||
if (!text.isBlank() && !missingInfo.contains(text)) {
|
||||
missingInfo.add(text);
|
||||
}
|
||||
}
|
||||
}
|
||||
return missingInfo;
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> extractStructuredRecommendedActions() {
|
||||
List<Map<String, Object>> actions = new ArrayList<>();
|
||||
Map<String, Object> structuredOutput = VerifierContextHolder.getExecutorStructuredOutput();
|
||||
Object recommendedActions = structuredOutput == null ? null : structuredOutput.get("recommended_actions");
|
||||
if (recommendedActions instanceof List<?> actionList) {
|
||||
for (Object item : actionList) {
|
||||
if (item instanceof Map<?, ?> rawAction) {
|
||||
Map<String, Object> action = new LinkedHashMap<>();
|
||||
action.put("action_text", textValue(rawAction.get("action_text")));
|
||||
action.put("reason", textValue(rawAction.get("reason")));
|
||||
if (!String.valueOf(action.get("action_text")).isBlank()) {
|
||||
actions.add(action);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return actions;
|
||||
}
|
||||
|
||||
private void addLegacyFactsToComposerInput(VerifierDecision decision, List<Map<String, Object>> allowedClaims,
|
||||
List<Map<String, Object>> allowedHypotheses,
|
||||
List<String> missingInfo) {
|
||||
for (Map<String, Object> fact : decision.factsChecked()) {
|
||||
String verification = String.valueOf(fact.getOrDefault("verification", ""));
|
||||
String factText = String.valueOf(fact.getOrDefault("fact", ""));
|
||||
String detail = String.valueOf(fact.getOrDefault("detail", ""));
|
||||
if ("direct_evidence".equals(verification)) {
|
||||
allowedClaims.add(Map.of(
|
||||
"claim_id", "",
|
||||
"claim_type", "",
|
||||
"claim_text", factText,
|
||||
"support_level", "direct",
|
||||
"verification", verification,
|
||||
"detail", detail
|
||||
));
|
||||
} else if ("indirect_support".equals(verification)) {
|
||||
allowedHypotheses.add(Map.of(
|
||||
"hypothesis_text", factText,
|
||||
"basis", detail.isBlank() ? "当前仅有间接支持，不能作为确认结论" : detail
|
||||
));
|
||||
} else {
|
||||
addMissingInfo(missingInfo, factText, detail);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void addMissingInfo(List<String> missingInfo, String text, String detail) {
|
||||
if (text == null || text.isBlank()) {
|
||||
return;
|
||||
}
|
||||
String value = detail == null || detail.isBlank() ? text : text + "：" + detail;
|
||||
if (!missingInfo.contains(value)) {
|
||||
missingInfo.add(value);
|
||||
}
|
||||
return composerSafeInputBuilder.build(
|
||||
originalQuery,
|
||||
decision,
|
||||
VerifierContextHolder.getExecutorStructuredOutput());
|
||||
}
|
||||
|
||||
private ComposerRenderResult parseComposerOutput(String composerOutput, Map<String, Object> composerInput) {
|
||||
try {
|
||||
JsonNode root = objectMapper.readTree(sanitizeJsonPayload(composerOutput));
|
||||
String answerSummary = root.path("answer_summary").asText("");
|
||||
String userFacingAnswer = root.path("user_facing_answer").asText("");
|
||||
if (answerSummary.isBlank() || userFacingAnswer.isBlank() || !root.path("recommended_actions").isArray()) {
|
||||
return buildFixedFallbackAnswer(composerInput, "composer_schema_invalid");
|
||||
}
|
||||
|
||||
Map<String, Object> audit = new LinkedHashMap<>();
|
||||
audit.put("status", "valid");
|
||||
audit.put("answer_summary", answerSummary);
|
||||
audit.put("recommended_actions", parseComposerActions(root.path("recommended_actions")));
|
||||
audit.put("user_facing_answer", userFacingAnswer);
|
||||
return new ComposerRenderResult(userFacingAnswer, audit);
|
||||
} catch (Exception e) {
|
||||
logger.warn("解析 composer_output 失败，使用安全降级模板");
|
||||
return buildFixedFallbackAnswer(composerInput, "composer_malformed");
|
||||
}
|
||||
}
|
||||
|
||||
private List<Map<String, Object>> parseComposerActions(JsonNode actionsNode) {
|
||||
List<Map<String, Object>> actions = new ArrayList<>();
|
||||
if (!actionsNode.isArray()) {
|
||||
return actions;
|
||||
}
|
||||
for (JsonNode actionNode : actionsNode) {
|
||||
Map<String, Object> action = new LinkedHashMap<>();
|
||||
action.put("action_text", actionNode.path("action_text").asText(""));
|
||||
action.put("reason", actionNode.path("reason").asText(""));
|
||||
actions.add(action);
|
||||
}
|
||||
return actions;
|
||||
return composerOutputParser.parse(
|
||||
composerOutput,
|
||||
composerInput,
|
||||
buildNextStepSuggestionsFromTrace());
|
||||
}
|
||||
|
||||
private ComposerRenderResult buildFixedFallbackAnswer(Map<String, Object> composerInput, String status) {
|
||||
String answer = renderSafeFallback(composerInput);
|
||||
Map<String, Object> audit = new LinkedHashMap<>();
|
||||
audit.put("status", status);
|
||||
audit.put("detail", "used safe fallback rendering");
|
||||
audit.put("answer_summary", firstSentence(answer));
|
||||
audit.put("recommended_actions", composerInput.getOrDefault("recommended_actions", List.of()));
|
||||
audit.put("user_facing_answer", answer);
|
||||
return new ComposerRenderResult(answer, audit);
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
private String renderSafeFallback(Map<String, Object> composerInput) {
|
||||
String verdict = String.valueOf(composerInput.getOrDefault("verdict", "LOW_CONFID"));
|
||||
List<Map<String, Object>> allowedClaims =
|
||||
(List<Map<String, Object>>) composerInput.getOrDefault("allowed_claims", List.of());
|
||||
List<Map<String, Object>> allowedHypotheses =
|
||||
(List<Map<String, Object>>) composerInput.getOrDefault("allowed_hypotheses", List.of());
|
||||
List<String> missingInfo = (List<String>) composerInput.getOrDefault("missing_info", List.of());
|
||||
List<Map<String, Object>> recommendedActions =
|
||||
(List<Map<String, Object>>) composerInput.getOrDefault("recommended_actions", List.of());
|
||||
|
||||
StringBuilder output = new StringBuilder();
|
||||
if ("REJECT".equals(verdict)) {
|
||||
output.append(DEGRADED_PREFIX);
|
||||
} else if ("LOW_CONFID".equals(verdict)) {
|
||||
output.append(LOW_CONFID_DISCLAIMER);
|
||||
}
|
||||
|
||||
output.append("\n\n已确认信息：");
|
||||
if (allowedClaims.isEmpty()) {
|
||||
output.append("\n- 暂无可稳定确认的信息");
|
||||
} else {
|
||||
for (Map<String, Object> claim : allowedClaims) {
|
||||
String text = String.valueOf(claim.getOrDefault("claim_text", ""));
|
||||
if (!text.isBlank()) {
|
||||
output.append("\n- ").append(text);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!"REJECT".equals(verdict) && !allowedHypotheses.isEmpty()) {
|
||||
output.append("\n\n可能方向：");
|
||||
for (Map<String, Object> hypothesis : allowedHypotheses) {
|
||||
output.append("\n- ").append(hypothesis.getOrDefault("hypothesis_text", ""));
|
||||
String basis = String.valueOf(hypothesis.getOrDefault("basis", ""));
|
||||
if (!basis.isBlank()) {
|
||||
output.append("（").append(basis).append("）");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
output.append("\n\n").append("REJECT".equals(verdict) ? "证据缺口：" : "当前缺口：");
|
||||
if (missingInfo.isEmpty()) {
|
||||
output.append("\n- 当前缺少足够的直接证据支撑核心结论");
|
||||
} else {
|
||||
for (String gap : missingInfo) {
|
||||
output.append("\n- ").append(gap);
|
||||
}
|
||||
}
|
||||
|
||||
output.append("\n\n建议下一步：");
|
||||
if (recommendedActions.isEmpty()) {
|
||||
for (String suggestion : buildNextStepSuggestionsFromTrace()) {
|
||||
output.append("\n- ").append(suggestion);
|
||||
}
|
||||
} else {
|
||||
for (Map<String, Object> action : recommendedActions) {
|
||||
String text = String.valueOf(action.getOrDefault("action_text", ""));
|
||||
if (!text.isBlank()) {
|
||||
output.append("\n- ").append(text);
|
||||
String reason = String.valueOf(action.getOrDefault("reason", ""));
|
||||
if (!reason.isBlank()) {
|
||||
output.append("：").append(reason);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return output.toString().trim();
|
||||
}
|
||||
|
||||
private String firstSentence(String text) {
|
||||
if (text == null || text.isBlank()) {
|
||||
return "";
|
||||
}
|
||||
int end = text.indexOf('\n');
|
||||
return end < 0 ? text : text.substring(0, end);
|
||||
}
|
||||
|
||||
private String textValue(Object value) {
|
||||
if (value == null) {
|
||||
return "";
|
||||
}
|
||||
String text = String.valueOf(value);
|
||||
return "null".equals(text) ? "" : text;
|
||||
return composerOutputParser.fallback(
|
||||
composerInput,
|
||||
buildNextStepSuggestionsFromTrace(),
|
||||
status);
|
||||
}
|
||||
|
||||
private List<String> buildNextStepSuggestionsFromTrace() {
|
||||
@@ -1335,20 +905,6 @@ public class ChatService {
|
||||
return gaps;
|
||||
}
|
||||
|
||||
private record VerifierDecision(
|
||||
String verdict,
|
||||
double groundednessScore,
|
||||
int criticalFactCount,
|
||||
List<Map<String, Object>> claimChecks,
|
||||
List<Map<String, Object>> factsChecked,
|
||||
String rationale,
|
||||
int round
|
||||
) {
|
||||
}
|
||||
|
||||
private record ComposerRenderResult(String answer, Map<String, Object> audit) {
|
||||
}
|
||||
|
||||
/** 从 agent_step 和 tool_invocation 汇总指标回填 diagnosis_run */
|
||||
private void backfillRunMetrics(DiagnosisRun run) {
|
||||
try {
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class ComposerOutputParserTest {
|
||||
|
||||
private final ComposerOutputParser parser = new ComposerOutputParser();
|
||||
|
||||
@Test
|
||||
void malformedOutputUsesSafeAllowedMaterialFallback() {
|
||||
ComposerRenderResult result = parser.parse(
|
||||
"not-json",
|
||||
Map.of(
|
||||
"verdict", "LOW_CONFID",
|
||||
"allowed_claims", List.of(Map.of(
|
||||
"claim_text", "连接数达到上限")),
|
||||
"allowed_hypotheses", List.of(),
|
||||
"missing_info", List.of("缺少泄漏检测日志"),
|
||||
"recommended_actions", List.of(),
|
||||
"raw_executor_output", "未验证的秘密根因"),
|
||||
List.of("人工查看 Trace"));
|
||||
|
||||
assertEquals("composer_malformed", result.audit().get("status"));
|
||||
assertTrue(result.answer().contains("连接数达到上限"));
|
||||
assertFalse(result.answer().contains("未验证的秘密根因"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
|
||||
class ComposerSafeInputBuilderTest {
|
||||
|
||||
private final ComposerSafeInputBuilder builder = new ComposerSafeInputBuilder();
|
||||
|
||||
@Test
|
||||
@SuppressWarnings("unchecked")
|
||||
void rejectInputContainsOnlyVerifierAllowedClaimsAndMissingInformation() {
|
||||
Map<String, Object> executorOutput = Map.of(
|
||||
"answer_version", "executor_evidence_v2",
|
||||
"claims", List.of(
|
||||
Map.of("claim_id", "c1", "claim_type", "symptom",
|
||||
"claim_text", "连接数达到上限", "support_level", "direct"),
|
||||
Map.of("claim_id", "c2", "claim_type", "root_cause",
|
||||
"claim_text", "存在连接泄漏", "support_level", "inference")),
|
||||
"hypotheses", List.of(Map.of("hypothesis_text", "raw hypothesis")),
|
||||
"missing_info", List.of("缺少泄漏检测日志"),
|
||||
"recommended_actions", List.of(Map.of(
|
||||
"action_text", "开启泄漏检测", "reason", "补齐证据")));
|
||||
VerifierDecision decision = new VerifierDecision(
|
||||
"REJECT", 0.4, 2,
|
||||
List.of(
|
||||
Map.of("claim_id", "c1", "claim_text", "连接数达到上限",
|
||||
"claim_type", "symptom", "verification", "direct_observation",
|
||||
"detail", "指标直接支持"),
|
||||
Map.of("claim_id", "c2", "claim_text", "存在连接泄漏",
|
||||
"claim_type", "root_cause", "verification", "unsupported",
|
||||
"detail", "没有泄漏检测日志")),
|
||||
List.of(), "核心原因未证实", 1);
|
||||
|
||||
Map<String, Object> input = builder.build("分析连接池", decision, executorOutput);
|
||||
|
||||
List<Map<String, Object>> allowed =
|
||||
(List<Map<String, Object>>) input.get("allowed_claims");
|
||||
assertEquals(List.of("连接数达到上限"),
|
||||
allowed.stream().map(claim -> claim.get("claim_text")).toList());
|
||||
assertEquals(List.of(), input.get("allowed_hypotheses"));
|
||||
assertEquals(List.of("缺少泄漏检测日志", "存在连接泄漏：没有泄漏检测日志"),
|
||||
input.get("missing_info"));
|
||||
assertFalse(input.containsKey("executor_output"));
|
||||
assertFalse(input.toString().contains("raw hypothesis"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
|
||||
class ExecutorEvidenceParserTest {
|
||||
|
||||
private final ExecutorEvidenceParser parser = new ExecutorEvidenceParser();
|
||||
|
||||
@Test
|
||||
void parsesLegalNoEvidenceSnapshotAsValid() {
|
||||
ExecutorEvidenceParser.ParseResult result = parser.parse("""
|
||||
{
|
||||
"answer_version": "executor_evidence_v2",
|
||||
"claims": [],
|
||||
"hypotheses": [],
|
||||
"recommended_actions": [],
|
||||
"missing_info": ["当前查询没有返回可引用证据"]
|
||||
}
|
||||
""");
|
||||
|
||||
assertEquals("valid", result.status().get("status"));
|
||||
assertEquals("executor_evidence_v2", result.structuredOutput().get("answer_version"));
|
||||
assertEquals(List.of(), result.structuredOutput().get("claims"));
|
||||
}
|
||||
|
||||
@Test
|
||||
@SuppressWarnings("unchecked")
|
||||
void enrichesUniqueInvocationWithoutChangingClaimText() {
|
||||
ExecutorEvidenceParser.ParseResult parsed = parser.parse("""
|
||||
{
|
||||
"answer_version": "executor_evidence_v2",
|
||||
"claims": [{
|
||||
"claim_id": "claim-1",
|
||||
"claim_text": "连接数达到上限",
|
||||
"evidence_bindings": [{
|
||||
"tool_name": "queryPrometheusAlerts",
|
||||
"raw_path": "$.alerts[0]",
|
||||
"evidence_excerpt": "active=50 max=50"
|
||||
}]
|
||||
}]
|
||||
}
|
||||
""");
|
||||
|
||||
ExecutorEvidenceParser.ParseResult enriched = parser.enrich(parsed, List.of(Map.of(
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_ids", List.of(17L))));
|
||||
|
||||
Map<String, Object> claim = ((List<Map<String, Object>>)
|
||||
enriched.structuredOutput().get("claims")).get(0);
|
||||
Map<String, Object> binding = ((List<Map<String, Object>>)
|
||||
claim.get("evidence_bindings")).get(0);
|
||||
assertEquals("连接数达到上限", claim.get("claim_text"));
|
||||
assertEquals("query_metrics", binding.get("tool_name"));
|
||||
assertEquals(17L, binding.get("source_invocation_id"));
|
||||
assertEquals(1, ((List<?>) enriched.structuredOutput()
|
||||
.get("_gatekeeper_warnings")).size());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class SafeFallbackRendererTest {
|
||||
|
||||
private final SafeFallbackRenderer renderer = new SafeFallbackRenderer();
|
||||
|
||||
@Test
|
||||
void rendersOnlyAllowedMaterial() {
|
||||
String answer = renderer.render(Map.of(
|
||||
"verdict", "REJECT",
|
||||
"allowed_claims", List.of(Map.of(
|
||||
"claim_text", "连接数达到上限")),
|
||||
"allowed_hypotheses", List.of(),
|
||||
"missing_info", List.of("缺少泄漏检测日志"),
|
||||
"recommended_actions", List.of(Map.of(
|
||||
"action_text", "开启泄漏检测", "reason", "补齐证据")),
|
||||
"raw_executor_output", "秘密的未验证根因"),
|
||||
List.of("人工查看 Trace"));
|
||||
|
||||
assertTrue(answer.contains("当前无法基于已获取证据生成可靠结论"));
|
||||
assertTrue(answer.contains("连接数达到上限"));
|
||||
assertTrue(answer.contains("缺少泄漏检测日志"));
|
||||
assertTrue(answer.contains("开启泄漏检测：补齐证据"));
|
||||
assertFalse(answer.contains("秘密的未验证根因"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
package com.superbiz.agent.diagnosis.protocol;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
|
||||
class VerifierOutputParserTest {
|
||||
|
||||
private final VerifierOutputParser parser = new VerifierOutputParser();
|
||||
|
||||
@Test
|
||||
void appliesLowConfidenceCeilingWithoutChangingModelVerdict() {
|
||||
VerifierDecision modelDecision = parser.parse("""
|
||||
{
|
||||
"verdict": "PASS",
|
||||
"groundedness_score": 0.92,
|
||||
"critical_fact_count": 1,
|
||||
"facts_checked": [{
|
||||
"fact": "连接数达到上限",
|
||||
"is_critical": true,
|
||||
"verification": "direct_evidence",
|
||||
"detail": "active=50 max=50",
|
||||
"evidence_refs": []
|
||||
}],
|
||||
"rationale": "证据充分"
|
||||
}
|
||||
""", 1);
|
||||
|
||||
VerifierDecision effectiveDecision = parser.applyCeiling(
|
||||
modelDecision,
|
||||
"valid",
|
||||
Map.of("status", "fail", "severity", "low_confid"));
|
||||
|
||||
assertEquals("PASS", modelDecision.verdict());
|
||||
assertEquals("LOW_CONFID", effectiveDecision.verdict());
|
||||
assertEquals(1, effectiveDecision.criticalFactCount());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
|
||||
class ComposerNodeAdapterTest {
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
@Test
|
||||
void usesStableAllowedMaterialInputAcrossTechnicalRetry() throws Exception {
|
||||
CapturingInvoker invoker = new CapturingInvoker("""
|
||||
{
|
||||
"answer_summary": "已确认连接数达到上限",
|
||||
"recommended_actions": [],
|
||||
"user_facing_answer": "已确认连接数达到上限。"
|
||||
}
|
||||
""");
|
||||
ComposerNodeAdapter adapter = new ComposerNodeAdapter(invoker);
|
||||
Map<String, Object> common = new LinkedHashMap<>();
|
||||
common.put(DiagnosisGraphState.DIAGNOSIS_CONTEXT,
|
||||
Map.of("query", "分析连接池"));
|
||||
common.put(DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT, Map.of(
|
||||
"answer_version", "executor_evidence_v2",
|
||||
"claims", List.of(Map.of(
|
||||
"claim_id", "c1", "claim_type", "symptom",
|
||||
"claim_text", "连接数达到上限", "support_level", "direct"))));
|
||||
common.put(DiagnosisGraphState.VERIFIER_OUTPUT, Map.of(
|
||||
"verdict", "PASS",
|
||||
"groundedness_score", 1.0,
|
||||
"critical_fact_count", 1,
|
||||
"claim_checks", List.of(Map.of(
|
||||
"claim_id", "c1", "claim_type", "symptom",
|
||||
"claim_text", "连接数达到上限",
|
||||
"verification", "direct_observation", "detail", "direct")),
|
||||
"facts_checked", List.of(),
|
||||
"rationale", "证据充分"));
|
||||
common.put(DiagnosisGraphState.EFFECTIVE_VERDICT, "PASS");
|
||||
common.put(DiagnosisGraphState.EXECUTOR_OUTPUT, "must-not-leak");
|
||||
|
||||
Map<String, Object> first = adapter.apply(
|
||||
new OverAllState(common),
|
||||
RunnableConfig.builder().threadId("run-composer").build()).get();
|
||||
Map<String, Object> retryState = new LinkedHashMap<>(common);
|
||||
retryState.put(DiagnosisGraphState.COMPOSER_RETRY_COUNT, 1);
|
||||
adapter.apply(new OverAllState(retryState),
|
||||
RunnableConfig.builder().threadId("run-composer").build()).get();
|
||||
|
||||
JsonNode input = objectMapper.readTree(invoker.inputs.get(0));
|
||||
assertEquals(invoker.inputs.get(0), invoker.inputs.get(1));
|
||||
assertFalse(input.has("executor_output"));
|
||||
assertFalse(input.toString().contains("must-not-leak"));
|
||||
assertEquals("COMPLETED", first.get(DiagnosisGraphState.COMPOSER_STATUS));
|
||||
assertEquals("已确认连接数达到上限。",
|
||||
first.get(DiagnosisGraphState.FINAL_ANSWER));
|
||||
}
|
||||
|
||||
private static final class CapturingInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final String output;
|
||||
private final List<String> inputs = new ArrayList<>();
|
||||
|
||||
private CapturingInvoker(String output) {
|
||||
this.output = output;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) {
|
||||
inputs.add(input);
|
||||
return output;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -325,6 +325,25 @@ class DiagnosisGraphRoutingTest {
|
||||
assertEquals(1, script.calls(DiagnosisGraphTopology.Node.COMPOSER));
|
||||
}
|
||||
|
||||
@Test
|
||||
void verifierLowConfidenceWithOnlyNonCriticalGapDoesNotRetryEvidence()
|
||||
throws Exception {
|
||||
ScriptedDiagnosisGraphActions script = new ScriptedDiagnosisGraphActions();
|
||||
appendThroughVerifiedInput(script);
|
||||
verifier(script, VerifierStatus.COMPLETED, Verdict.LOW_CONFID,
|
||||
Verdict.PASS, Map.of("facts_checked", List.of(
|
||||
Map.of(
|
||||
"fact", "optional detail remains unverified",
|
||||
"is_critical", false,
|
||||
"verification", "no_evidence"))));
|
||||
composerCompleted(script);
|
||||
|
||||
run(script);
|
||||
|
||||
assertEquals(0, script.calls(DiagnosisGraphTopology.Node.EVIDENCE_RETRY));
|
||||
assertEquals(1, script.calls(DiagnosisGraphTopology.Node.COMPOSER));
|
||||
}
|
||||
|
||||
@Test
|
||||
void verifierRejectStillRunsComposerWithSafeMaterial() throws Exception {
|
||||
ScriptedDiagnosisGraphActions script = new ScriptedDiagnosisGraphActions();
|
||||
@@ -583,6 +602,7 @@ class DiagnosisGraphRoutingTest {
|
||||
return Map.of("facts_checked", List.of(
|
||||
Map.of(
|
||||
"fact", "database saturation remains unverified",
|
||||
"is_critical", true,
|
||||
"verification", "no_evidence")));
|
||||
}
|
||||
|
||||
|
||||
+24
@@ -0,0 +1,24 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.util.concurrent.TimeoutException;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
|
||||
class DiagnosisNodeFailureClassifierTest {
|
||||
|
||||
private final DiagnosisNodeFailureClassifier classifier =
|
||||
DiagnosisNodeFailureClassifier.defaults();
|
||||
|
||||
@Test
|
||||
void classifiesOnlyRecognizedTransientFailuresAsRetryable() {
|
||||
assertEquals(DiagnosisNodeFailureClassifier.FailureKind.RETRYABLE,
|
||||
classifier.classify(new TimeoutException("model timed out")));
|
||||
assertEquals(DiagnosisNodeFailureClassifier.FailureKind.RETRYABLE,
|
||||
classifier.classify(new RuntimeException(new IOException("connection reset"))));
|
||||
assertEquals(DiagnosisNodeFailureClassifier.FailureKind.NON_RETRYABLE,
|
||||
classifier.classify(new IllegalStateException("unknown failure")));
|
||||
}
|
||||
}
|
||||
+399
@@ -0,0 +1,399 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.CompiledGraph;
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Deque;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.mockito.ArgumentMatchers.anyMap;
|
||||
import static org.mockito.ArgumentMatchers.eq;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.times;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
class DiagnosisRealGraphIntegrationTest {
|
||||
|
||||
@Test
|
||||
void legalNoEvidenceWithVerifiedBindingContinuesUnderLowConfidenceCeiling()
|
||||
throws Exception {
|
||||
ExecutorGatekeeperService gatekeeper = mock(ExecutorGatekeeperService.class);
|
||||
when(gatekeeper.validateRun(eq("run-no-evidence"), anyMap(), anyMap()))
|
||||
.thenReturn(Map.of(
|
||||
"status", "fail",
|
||||
"severity", "low_confid",
|
||||
"checked_bindings", List.of(Map.of(
|
||||
"claim_id", "c-no", "status", "pass",
|
||||
"tool_name", "query_logs",
|
||||
"source_invocation_id", 31L,
|
||||
"raw_path", "$.no_evidence",
|
||||
"matched_text", "query_logs total=0 evidence_status=no_evidence")),
|
||||
"failed_rules", List.of("evidence.missing")));
|
||||
String noEvidenceOutput = """
|
||||
{"answer_version":"executor_evidence_v2","claims":[{
|
||||
"claim_id":"c-no","claim_type":"negative_observation",
|
||||
"claim_text":"当前查询未检索到连接泄漏日志","support_level":"direct",
|
||||
"evidence_bindings":[{"tool_name":"query_logs",
|
||||
"source_invocation_id":31,"raw_path":"$.no_evidence",
|
||||
"evidence_excerpt":"query_logs total=0 evidence_status=no_evidence"}]}],
|
||||
"hypotheses":[],"recommended_actions":[],"missing_info":[]}
|
||||
""";
|
||||
String verifier = """
|
||||
{"verdict":"PASS","groundedness_score":0.8,
|
||||
"critical_fact_count":1,"claim_checks":[{
|
||||
"claim_id":"c-no","claim_text":"当前查询未检索到连接泄漏日志",
|
||||
"claim_type":"negative_observation","verification":"direct_observation",
|
||||
"detail":"no-evidence binding verified","evidence_refs":[]}],
|
||||
"facts_checked":[],"rationale":"查询结果可信但证据有限"}
|
||||
""";
|
||||
DiagnosisGraphActions actions = new DiagnosisRealGraphActionsFactory().create(
|
||||
constant(plannerOutput()), constant(noEvidenceOutput), constant(verifier),
|
||||
constant(composerOutput()), gatekeeper);
|
||||
|
||||
OverAllState state = new DiagnosisGraphFactory().compile(actions)
|
||||
.invoke(initialState(), config("run-no-evidence"))
|
||||
.orElseThrow();
|
||||
|
||||
assertEquals("LOW_CONFID", DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.EFFECTIVE_VERDICT));
|
||||
assertEquals(List.of("planner", "executor", "gatekeeper", "verified_input",
|
||||
"verifier", "composer"), eventNodes(state));
|
||||
}
|
||||
|
||||
@Test
|
||||
void criticalGapPerformsOneIncrementalRoundAndRevalidatesCompleteSnapshot()
|
||||
throws Exception {
|
||||
ExecutorGatekeeperService gatekeeper = mock(ExecutorGatekeeperService.class);
|
||||
when(gatekeeper.validateRun(eq("run-critical-retry"), anyMap(), anyMap()))
|
||||
.thenReturn(passResult())
|
||||
.thenReturn(passResultWithSecondClaim());
|
||||
QueueInvoker planner = new QueueInvoker(plannerOutput(), """
|
||||
{"selected_skill":null,"selection_reason":"evidence gap",
|
||||
"plan":["只补查泄漏检测日志"],"reasoning":"增量补证据"}
|
||||
""");
|
||||
QueueInvoker executor = new QueueInvoker(executorOutput(), executorFullSnapshot());
|
||||
QueueInvoker verifier = new QueueInvoker("""
|
||||
{"verdict":"LOW_CONFID","groundedness_score":0.5,
|
||||
"critical_fact_count":1,"claim_checks":[],
|
||||
"facts_checked":[{"fact":"c2: 是否存在连接泄漏",
|
||||
"is_critical":true,"verification":"no_evidence",
|
||||
"detail":"缺少泄漏检测日志","evidence_refs":[]}],
|
||||
"rationale":"需补证据"}
|
||||
""", verifierSecondPassOutput());
|
||||
QueueInvoker composer = new QueueInvoker(composerOutput());
|
||||
DiagnosisGraphActions actions = new DiagnosisRealGraphActionsFactory().create(
|
||||
planner, executor, verifier, composer, gatekeeper);
|
||||
|
||||
OverAllState state = new DiagnosisGraphFactory().compile(actions)
|
||||
.invoke(initialState(), config("run-critical-retry"))
|
||||
.orElseThrow();
|
||||
|
||||
assertEquals(2, planner.inputs.size());
|
||||
assertEquals(2, executor.inputs.size());
|
||||
assertEquals(2, verifier.inputs.size());
|
||||
assertEquals(1, composer.inputs.size());
|
||||
assertEquals(1, DiagnosisGraphState.intValue(
|
||||
state, DiagnosisGraphState.EVIDENCE_RETRY_COUNT));
|
||||
assertEquals(true, new com.fasterxml.jackson.databind.ObjectMapper()
|
||||
.readTree(executor.inputs.get(1))
|
||||
.path("execution_constraints")
|
||||
.path("only_execute_incremental_queries").asBoolean());
|
||||
assertEquals("executor_evidence_v2",
|
||||
new com.fasterxml.jackson.databind.ObjectMapper()
|
||||
.readTree(executor.inputs.get(1))
|
||||
.path("retry_context")
|
||||
.path("prior_verified_executor_output")
|
||||
.path("answer_version").asText());
|
||||
assertEquals(List.of("planner", "executor", "gatekeeper", "verified_input",
|
||||
"verifier", "evidence_retry", "planner", "executor",
|
||||
"gatekeeper", "verified_input", "verifier", "composer"),
|
||||
eventNodes(state));
|
||||
verify(gatekeeper, times(2)).validateRun(
|
||||
eq("run-critical-retry"), anyMap(), anyMap());
|
||||
}
|
||||
|
||||
@Test
|
||||
void gatekeeperRejectSkipsVerifierAndUsesPreVerificationFallback() throws Exception {
|
||||
ExecutorGatekeeperService gatekeeper = mock(ExecutorGatekeeperService.class);
|
||||
when(gatekeeper.validateRun(eq("run-real-reject"), anyMap(), anyMap()))
|
||||
.thenReturn(Map.of(
|
||||
"status", "fail",
|
||||
"severity", "reject",
|
||||
"checked_bindings", List.of(),
|
||||
"failed_rules", List.of("evidence.raw_path")));
|
||||
DiagnosisGraphActions actions = new DiagnosisRealGraphActionsFactory().create(
|
||||
constant(plannerOutput()),
|
||||
constant(executorOutput()),
|
||||
(input, config) -> { throw new AssertionError("verifier must not run"); },
|
||||
(input, config) -> { throw new AssertionError("composer must not run"); },
|
||||
gatekeeper);
|
||||
|
||||
OverAllState state = new DiagnosisGraphFactory().compile(actions)
|
||||
.invoke(initialState(), config("run-real-reject"))
|
||||
.orElseThrow();
|
||||
|
||||
String answer = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.FINAL_ANSWER);
|
||||
assertFalse(answer.contains("连接数达到上限"));
|
||||
assertEquals(List.of("planner", "executor", "gatekeeper", "fallback"),
|
||||
eventNodes(state));
|
||||
}
|
||||
|
||||
@Test
|
||||
void exhaustedComposerRetryUsesVerifiedMaterialWithoutRerunningPredecessors()
|
||||
throws Exception {
|
||||
ExecutorGatekeeperService gatekeeper = mock(ExecutorGatekeeperService.class);
|
||||
when(gatekeeper.validateRun(eq("run-composer-fallback"), anyMap(), anyMap()))
|
||||
.thenReturn(passResult());
|
||||
QueueInvoker planner = new QueueInvoker(plannerOutput());
|
||||
QueueInvoker executor = new QueueInvoker(executorOutput());
|
||||
QueueInvoker verifier = new QueueInvoker(verifierPassOutput());
|
||||
QueueInvoker composer = new QueueInvoker("not-json", "still-not-json");
|
||||
DiagnosisGraphActions actions = new DiagnosisRealGraphActionsFactory().create(
|
||||
planner, executor, verifier, composer, gatekeeper);
|
||||
|
||||
OverAllState state = new DiagnosisGraphFactory().compile(actions)
|
||||
.invoke(initialState(), config("run-composer-fallback"))
|
||||
.orElseThrow();
|
||||
|
||||
String answer = DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.FINAL_ANSWER);
|
||||
assertEquals(1, planner.inputs.size());
|
||||
assertEquals(1, executor.inputs.size());
|
||||
assertEquals(1, verifier.inputs.size());
|
||||
assertEquals(2, composer.inputs.size());
|
||||
assertEquals(composer.inputs.get(0), composer.inputs.get(1));
|
||||
assertFalse(answer.contains("秘密的未验证根因"));
|
||||
assertEquals(List.of("planner", "executor", "gatekeeper", "verified_input",
|
||||
"verifier", "composer", "composer", "fallback"),
|
||||
eventNodes(state));
|
||||
}
|
||||
|
||||
@Test
|
||||
void compiledRealNodeGraphCompletesPassPathInExactOrder() throws Exception {
|
||||
ExecutorGatekeeperService gatekeeper = mock(ExecutorGatekeeperService.class);
|
||||
when(gatekeeper.validateRun(eq("run-real-pass"), anyMap(), anyMap()))
|
||||
.thenReturn(Map.of(
|
||||
"status", "pass",
|
||||
"severity", "none",
|
||||
"checked_bindings", List.of(Map.of(
|
||||
"claim_id", "c1",
|
||||
"status", "pass",
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 17L,
|
||||
"raw_path", "$.alerts[0]",
|
||||
"matched_text", "active=50 max=50")),
|
||||
"failed_rules", List.of()));
|
||||
DiagnosisRealGraphActionsFactory actionFactory =
|
||||
new DiagnosisRealGraphActionsFactory();
|
||||
DiagnosisGraphActions actions = actionFactory.create(
|
||||
constant("""
|
||||
{"selected_skill":null,"selection_reason":"none",
|
||||
"plan":["查询指标"],"reasoning":"确认现象"}
|
||||
"""),
|
||||
constant("""
|
||||
{"answer_version":"executor_evidence_v2","claims":[{
|
||||
"claim_id":"c1","claim_type":"symptom",
|
||||
"claim_text":"连接数达到上限","support_level":"direct",
|
||||
"evidence_bindings":[{"tool_name":"query_metrics",
|
||||
"source_invocation_id":17,"raw_path":"$.alerts[0]",
|
||||
"evidence_excerpt":"active=50 max=50"}]}],
|
||||
"hypotheses":[],"recommended_actions":[],"missing_info":[]}
|
||||
"""),
|
||||
constant("""
|
||||
{"verdict":"PASS","groundedness_score":1.0,
|
||||
"critical_fact_count":1,"claim_checks":[{
|
||||
"claim_id":"c1","claim_text":"连接数达到上限",
|
||||
"claim_type":"symptom","verification":"direct_observation",
|
||||
"detail":"指标直接支持","evidence_refs":[]}],
|
||||
"facts_checked":[],"rationale":"证据充分"}
|
||||
"""),
|
||||
constant("""
|
||||
{"answer_summary":"已确认连接数达到上限",
|
||||
"recommended_actions":[],
|
||||
"user_facing_answer":"已确认连接数达到上限。"}
|
||||
"""),
|
||||
gatekeeper);
|
||||
CompiledGraph graph = new DiagnosisGraphFactory().compile(actions);
|
||||
|
||||
OverAllState state = graph.invoke(Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT,
|
||||
Map.of("query", "分析连接池"),
|
||||
DiagnosisGraphState.PLANNER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.VERIFIER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.COMPOSER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.PLANNER_MODE, "NORMAL"),
|
||||
RunnableConfig.builder()
|
||||
.threadId("run-real-pass")
|
||||
.addMetadata("runId", "run-real-pass")
|
||||
.build()).orElseThrow();
|
||||
|
||||
assertEquals("已确认连接数达到上限。", DiagnosisGraphState.stringValue(
|
||||
state, DiagnosisGraphState.FINAL_ANSWER));
|
||||
List<?> events = DiagnosisGraphState.listValue(
|
||||
state, DiagnosisGraphState.ORCHESTRATION_EVENTS);
|
||||
assertEquals(List.of("planner", "executor", "gatekeeper", "verified_input",
|
||||
"verifier", "composer"),
|
||||
events.stream().map(event -> ((OrchestrationEvent) event).node()).toList());
|
||||
verify(gatekeeper, times(1)).validateRun(
|
||||
eq("run-real-pass"), anyMap(), anyMap());
|
||||
}
|
||||
|
||||
private DiagnosisAgentInvoker constant(String output) {
|
||||
return (input, config) -> output;
|
||||
}
|
||||
|
||||
private Map<String, Object> initialState() {
|
||||
return Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT,
|
||||
Map.of("query", "分析连接池"),
|
||||
DiagnosisGraphState.PLANNER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.VERIFIER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.COMPOSER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.PLANNER_MODE, "NORMAL");
|
||||
}
|
||||
|
||||
private RunnableConfig config(String runId) {
|
||||
return RunnableConfig.builder()
|
||||
.threadId(runId)
|
||||
.addMetadata("runId", runId)
|
||||
.build();
|
||||
}
|
||||
|
||||
private List<String> eventNodes(OverAllState state) {
|
||||
return DiagnosisGraphState.listValue(state, DiagnosisGraphState.ORCHESTRATION_EVENTS)
|
||||
.stream()
|
||||
.map(event -> ((OrchestrationEvent) event).node())
|
||||
.toList();
|
||||
}
|
||||
|
||||
private String plannerOutput() {
|
||||
return """
|
||||
{"selected_skill":null,"selection_reason":"none",
|
||||
"plan":["查询指标"],"reasoning":"确认现象"}
|
||||
""";
|
||||
}
|
||||
|
||||
private String executorOutput() {
|
||||
return """
|
||||
{"answer_version":"executor_evidence_v2","claims":[{
|
||||
"claim_id":"c1","claim_type":"symptom",
|
||||
"claim_text":"连接数达到上限","support_level":"direct",
|
||||
"evidence_bindings":[{"tool_name":"query_metrics",
|
||||
"source_invocation_id":17,"raw_path":"$.alerts[0]",
|
||||
"evidence_excerpt":"active=50 max=50"}]}],
|
||||
"hypotheses":[{"hypothesis_text":"秘密的未验证根因"}],
|
||||
"recommended_actions":[],"missing_info":[]}
|
||||
""";
|
||||
}
|
||||
|
||||
private String verifierPassOutput() {
|
||||
return """
|
||||
{"verdict":"PASS","groundedness_score":1.0,
|
||||
"critical_fact_count":1,"claim_checks":[{
|
||||
"claim_id":"c1","claim_text":"连接数达到上限",
|
||||
"claim_type":"symptom","verification":"direct_observation",
|
||||
"detail":"指标直接支持","evidence_refs":[]}],
|
||||
"facts_checked":[],"rationale":"证据充分"}
|
||||
""";
|
||||
}
|
||||
|
||||
private String verifierSecondPassOutput() {
|
||||
return """
|
||||
{"verdict":"PASS","groundedness_score":1.0,
|
||||
"critical_fact_count":2,"claim_checks":[
|
||||
{"claim_id":"c1","claim_text":"连接数达到上限",
|
||||
"claim_type":"symptom","verification":"direct_observation",
|
||||
"detail":"指标直接支持","evidence_refs":[]},
|
||||
{"claim_id":"c2","claim_text":"泄漏日志存在",
|
||||
"claim_type":"root_cause","verification":"direct_observation",
|
||||
"detail":"日志直接支持","evidence_refs":[]}],
|
||||
"facts_checked":[],"rationale":"补证据完成"}
|
||||
""";
|
||||
}
|
||||
|
||||
private String composerOutput() {
|
||||
return """
|
||||
{"answer_summary":"诊断完成","recommended_actions":[],
|
||||
"user_facing_answer":"诊断完成。"}
|
||||
""";
|
||||
}
|
||||
|
||||
private String executorFullSnapshot() {
|
||||
return """
|
||||
{"answer_version":"executor_evidence_v2","claims":[
|
||||
{"claim_id":"c1","claim_type":"symptom",
|
||||
"claim_text":"连接数达到上限","support_level":"direct",
|
||||
"evidence_bindings":[{"tool_name":"query_metrics",
|
||||
"source_invocation_id":17,"raw_path":"$.alerts[0]",
|
||||
"evidence_excerpt":"active=50 max=50"}]},
|
||||
{"claim_id":"c2","claim_type":"root_cause",
|
||||
"claim_text":"泄漏日志存在","support_level":"direct",
|
||||
"evidence_bindings":[{"tool_name":"query_logs",
|
||||
"source_invocation_id":18,"raw_path":"$.logs[0]",
|
||||
"evidence_excerpt":"connection leak detected"}]}],
|
||||
"hypotheses":[],"recommended_actions":[],"missing_info":[]}
|
||||
""";
|
||||
}
|
||||
|
||||
private Map<String, Object> passResult() {
|
||||
return Map.of(
|
||||
"status", "pass",
|
||||
"severity", "none",
|
||||
"checked_bindings", List.of(Map.of(
|
||||
"claim_id", "c1", "status", "pass",
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 17L,
|
||||
"raw_path", "$.alerts[0]",
|
||||
"matched_text", "active=50 max=50")),
|
||||
"failed_rules", List.of());
|
||||
}
|
||||
|
||||
private Map<String, Object> passResultWithSecondClaim() {
|
||||
return Map.of(
|
||||
"status", "pass",
|
||||
"severity", "none",
|
||||
"checked_bindings", List.of(
|
||||
Map.of("claim_id", "c1", "status", "pass",
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 17L,
|
||||
"raw_path", "$.alerts[0]",
|
||||
"matched_text", "active=50 max=50"),
|
||||
Map.of("claim_id", "c2", "status", "pass",
|
||||
"tool_name", "query_logs",
|
||||
"source_invocation_id", 18L,
|
||||
"raw_path", "$.logs[0]",
|
||||
"matched_text", "connection leak detected")),
|
||||
"failed_rules", List.of());
|
||||
}
|
||||
|
||||
private static final class QueueInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final Deque<String> outputs = new ArrayDeque<>();
|
||||
private final List<String> inputs = new ArrayList<>();
|
||||
|
||||
private QueueInvoker(String... outputs) {
|
||||
this.outputs.addAll(List.of(outputs));
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) {
|
||||
inputs.add(input);
|
||||
if (outputs.isEmpty()) {
|
||||
throw new IllegalStateException("unexpected invoker call");
|
||||
}
|
||||
return outputs.removeFirst();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertSame;
|
||||
|
||||
class EvidenceRetryPrepareNodeTest {
|
||||
|
||||
@Test
|
||||
@SuppressWarnings("unchecked")
|
||||
void buildsBoundedContextFromCriticalGapsAndCompletedQueryRefs() throws Exception {
|
||||
Map<String, Object> priorOutput = Map.of(
|
||||
"answer_version", "executor_evidence_v2",
|
||||
"claims", List.of(Map.of("claim_id", "c1", "claim_text", "已确认现象")));
|
||||
List<Map<String, Object>> priorEvidence = List.of(
|
||||
Map.of("claim_id", "c1", "tool_name", "query_metrics",
|
||||
"source_invocation_id", 17L, "raw_path", "$.alerts[0]",
|
||||
"matched_text", "active=50 max=50"),
|
||||
Map.of("claim_id", "c2", "tool_name", "query_metrics",
|
||||
"source_invocation_id", 17L, "raw_path", "$.alerts[0]",
|
||||
"matched_text", "duplicate ref"));
|
||||
Map<String, Object> verifierOutput = Map.of(
|
||||
"facts_checked", List.of(
|
||||
Map.of("fact", "c2: 是否泄漏",
|
||||
"is_critical", true,
|
||||
"verification", "no_evidence",
|
||||
"detail", "缺少泄漏日志"),
|
||||
Map.of("fact", "可选细节",
|
||||
"is_critical", false,
|
||||
"verification", "no_evidence")));
|
||||
EvidenceRetryPrepareNode node = new EvidenceRetryPrepareNode();
|
||||
|
||||
Map<String, Object> update = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT, priorOutput,
|
||||
DiagnosisGraphState.VERIFIED_EVIDENCE, priorEvidence,
|
||||
DiagnosisGraphState.VERIFIER_OUTPUT, verifierOutput,
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 0)),
|
||||
RunnableConfig.builder().threadId("run-retry").build()).get();
|
||||
|
||||
Map<String, Object> context = (Map<String, Object>) update.get(
|
||||
DiagnosisGraphState.RETRY_CONTEXT);
|
||||
assertSame(priorOutput, context.get("prior_verified_executor_output"));
|
||||
assertEquals(priorEvidence, context.get("prior_verified_evidence"));
|
||||
assertEquals(List.of("c2"), ((List<Map<String, Object>>) context.get(
|
||||
"evidence_gaps")).stream().map(gap -> gap.get("claim_id")).toList());
|
||||
assertEquals(1, ((List<?>) context.get("completed_query_refs")).size());
|
||||
Map<String, Object> constraints =
|
||||
(Map<String, Object>) context.get("constraints");
|
||||
assertEquals(1, constraints.get("max_retry"));
|
||||
assertEquals(true, constraints.get("only_execute_incremental_queries"));
|
||||
assertEquals(true, constraints.get("preserve_prior_verified_claims"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class ExecutorNodeAdapterTest {
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
@Test
|
||||
void evidenceRetryRequestsIncrementalQueriesAndCompleteSnapshot() throws Exception {
|
||||
CapturingInvoker invoker = new CapturingInvoker("""
|
||||
{
|
||||
"answer_version": "executor_evidence_v2",
|
||||
"claims": [],
|
||||
"hypotheses": [],
|
||||
"recommended_actions": [],
|
||||
"missing_info": ["增量查询仍无证据"]
|
||||
}
|
||||
""");
|
||||
ExecutorNodeAdapter adapter = new ExecutorNodeAdapter(invoker);
|
||||
OverAllState state = new OverAllState(Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT, Map.of(
|
||||
"query", "分析连接池耗尽"),
|
||||
DiagnosisGraphState.PLANNER_PLAN, Map.of(
|
||||
"plan", List.of("只补查泄漏检测日志")),
|
||||
DiagnosisGraphState.PLANNER_MODE, "EVIDENCE_GAP_ONLY",
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 1,
|
||||
DiagnosisGraphState.RETRY_CONTEXT, Map.of(
|
||||
"completed_query_refs", List.of("query_metrics:17:$.alerts[0]"))));
|
||||
|
||||
Map<String, Object> update = adapter.apply(
|
||||
state, RunnableConfig.builder().threadId("run-executor").build()).get();
|
||||
|
||||
JsonNode input = objectMapper.readTree(invoker.input);
|
||||
assertEquals("EVIDENCE_GAP_ONLY", input.path("planner_mode").asText());
|
||||
assertTrue(input.path("execution_constraints")
|
||||
.path("only_execute_incremental_queries").asBoolean());
|
||||
assertTrue(input.path("execution_constraints")
|
||||
.path("return_complete_executor_evidence_v2_snapshot").asBoolean());
|
||||
assertFalse(input.has("verified_evidence"));
|
||||
assertEquals("COMPLETED", update.get(DiagnosisGraphState.EXECUTOR_STATUS));
|
||||
assertEquals(List.of(), ((Map<?, ?>) update.get(
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT)).get("claims"));
|
||||
OrchestrationEvent event = (OrchestrationEvent) ((List<?>) update.get(
|
||||
DiagnosisGraphState.ORCHESTRATION_EVENTS)).get(0);
|
||||
assertEquals(2, event.attempt());
|
||||
}
|
||||
|
||||
private static final class CapturingInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final String output;
|
||||
private String input;
|
||||
|
||||
private CapturingInvoker(String output) {
|
||||
this.output = output;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) {
|
||||
this.input = input;
|
||||
return output;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertTrue;
|
||||
|
||||
class FallbackNodeTest {
|
||||
|
||||
@Test
|
||||
void preVerificationFallbackNeverRendersExecutorClaims() throws Exception {
|
||||
FallbackNode node = new FallbackNode();
|
||||
Map<String, Object> update = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.FAILURE_REASON, "gatekeeper_reject",
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, Map.of(
|
||||
"claims", List.of(Map.of(
|
||||
"claim_text", "秘密的未验证根因"))),
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 0)),
|
||||
RunnableConfig.builder().threadId("run-fallback").build()).get();
|
||||
|
||||
String answer = String.valueOf(update.get(DiagnosisGraphState.FINAL_ANSWER));
|
||||
assertTrue(answer.contains("当前无法基于已获取证据生成可靠结论"));
|
||||
assertTrue(answer.contains("查看当前 Run Trace"));
|
||||
assertFalse(answer.contains("秘密的未验证根因"));
|
||||
}
|
||||
|
||||
@Test
|
||||
void postComposerFallbackRendersOnlyVerifierAllowedMaterial() throws Exception {
|
||||
FallbackNode node = new FallbackNode();
|
||||
Map<String, Object> update = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT,
|
||||
Map.of("query", "分析连接池"),
|
||||
DiagnosisGraphState.COMPOSER_STATUS,
|
||||
"NON_RETRYABLE_FAILED",
|
||||
DiagnosisGraphState.EFFECTIVE_VERDICT,
|
||||
"LOW_CONFID",
|
||||
DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT,
|
||||
Map.of("answer_version", "executor_evidence_v2",
|
||||
"claims", List.of(Map.of(
|
||||
"claim_id", "c1",
|
||||
"claim_type", "symptom",
|
||||
"claim_text", "连接数达到上限",
|
||||
"support_level", "direct"))),
|
||||
DiagnosisGraphState.VERIFIER_OUTPUT,
|
||||
Map.of("verdict", "LOW_CONFID",
|
||||
"claim_checks", List.of(Map.of(
|
||||
"claim_id", "c1",
|
||||
"claim_type", "symptom",
|
||||
"claim_text", "连接数达到上限",
|
||||
"verification", "direct_observation",
|
||||
"detail", "指标直接支持")),
|
||||
"facts_checked", List.of(),
|
||||
"rationale", "仍缺少根因证据"),
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT,
|
||||
Map.of("claims", List.of(Map.of(
|
||||
"claim_text", "秘密的未验证根因"))))),
|
||||
RunnableConfig.builder().threadId("run-post-fallback").build()).get();
|
||||
|
||||
String answer = String.valueOf(update.get(DiagnosisGraphState.FINAL_ANSWER));
|
||||
assertTrue(answer.contains("连接数达到上限"));
|
||||
assertFalse(answer.contains("秘密的未验证根因"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.superbiz.agent.service.ExecutorGatekeeperService;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertSame;
|
||||
import static org.mockito.ArgumentMatchers.anyMap;
|
||||
import static org.mockito.ArgumentMatchers.anyString;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.times;
|
||||
import static org.mockito.Mockito.verify;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
class GatekeeperNodeTest {
|
||||
|
||||
@Test
|
||||
void normalizesPassAndUnknownResultsDeterministically() throws Exception {
|
||||
ExecutorGatekeeperService service = mock(ExecutorGatekeeperService.class);
|
||||
GatekeeperNode node = new GatekeeperNode(service);
|
||||
Map<String, Object> executorOutput = Map.of(
|
||||
"answer_version", "executor_evidence_v2", "claims", List.of());
|
||||
RunnableConfig config = RunnableConfig.builder()
|
||||
.addMetadata("runId", "run-normalization")
|
||||
.build();
|
||||
|
||||
when(service.validateRun(anyString(), anyMap(), anyMap()))
|
||||
.thenReturn(Map.of(
|
||||
"status", "pass",
|
||||
"severity", "none",
|
||||
"checked_bindings", List.of()))
|
||||
.thenReturn(Map.of(
|
||||
"status", "mystery",
|
||||
"severity", "unknown",
|
||||
"checked_bindings", List.of()));
|
||||
|
||||
Map<String, Object> pass = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, executorOutput)), config).get();
|
||||
Map<String, Object> unknown = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, executorOutput)), config).get();
|
||||
|
||||
assertEquals("PASS", pass.get(DiagnosisGraphState.GATEKEEPER_STATUS));
|
||||
assertEquals("PASS", pass.get(
|
||||
DiagnosisGraphState.VERIFIER_VERDICT_CEILING));
|
||||
assertEquals("REJECT", unknown.get(
|
||||
DiagnosisGraphState.GATEKEEPER_STATUS));
|
||||
}
|
||||
|
||||
@Test
|
||||
void preservesRawLowConfidenceResultAndCountsPassedBindings() throws Exception {
|
||||
ExecutorGatekeeperService service = mock(ExecutorGatekeeperService.class);
|
||||
Map<String, Object> executorOutput = Map.of(
|
||||
"answer_version", "executor_evidence_v2", "claims", List.of());
|
||||
Map<String, Object> rawResult = Map.of(
|
||||
"status", "fail",
|
||||
"severity", "low_confid",
|
||||
"checked_bindings", List.of(
|
||||
Map.of("claim_id", "c1", "status", "pass"),
|
||||
Map.of("claim_id", "c2", "status", "fail")),
|
||||
"failed_rules", List.of("evidence.missing"));
|
||||
when(service.validateRun("run-gatekeeper", executorOutput, Map.of(
|
||||
"status", "valid", "detail", "graph executor completed")))
|
||||
.thenReturn(rawResult);
|
||||
GatekeeperNode node = new GatekeeperNode(service);
|
||||
|
||||
Map<String, Object> update = node.apply(
|
||||
new OverAllState(Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, executorOutput)),
|
||||
RunnableConfig.builder()
|
||||
.addMetadata("runId", "run-gatekeeper")
|
||||
.build()).get();
|
||||
|
||||
assertSame(rawResult, update.get(DiagnosisGraphState.GATEKEEPER_RESULT));
|
||||
assertEquals("LOW_CONFID", update.get(
|
||||
DiagnosisGraphState.GATEKEEPER_STATUS));
|
||||
assertEquals(1, update.get(DiagnosisGraphState.VERIFIED_BINDING_COUNT));
|
||||
assertEquals("LOW_CONFID", update.get(
|
||||
DiagnosisGraphState.VERIFIER_VERDICT_CEILING));
|
||||
verify(service, times(1)).validateRun(
|
||||
"run-gatekeeper", executorOutput, Map.of(
|
||||
"status", "valid", "detail", "graph executor completed"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
import static org.junit.jupiter.api.Assertions.assertSame;
|
||||
|
||||
class PlannerNodeAdapterTest {
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
@Test
|
||||
void projectsOnlyPlannerInputAndParsesCompletedPlan() throws Exception {
|
||||
CapturingInvoker invoker = new CapturingInvoker("""
|
||||
{
|
||||
"selected_skill": null,
|
||||
"selection_reason": "无需 skill",
|
||||
"plan": ["查询连接池指标"],
|
||||
"reasoning": "先确认现象"
|
||||
}
|
||||
""");
|
||||
PlannerNodeAdapter adapter = new PlannerNodeAdapter(invoker);
|
||||
RunnableConfig config = RunnableConfig.builder()
|
||||
.threadId("run-planner")
|
||||
.addMetadata("runId", "run-planner")
|
||||
.build();
|
||||
OverAllState state = new OverAllState(Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT, Map.of(
|
||||
"query", "分析连接池耗尽",
|
||||
"history", List.of()),
|
||||
DiagnosisGraphState.PLANNER_MODE, "NORMAL",
|
||||
DiagnosisGraphState.PLANNER_RETRY_COUNT, 0,
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, "must-not-leak"));
|
||||
|
||||
Map<String, Object> update = adapter.apply(state, config).get();
|
||||
|
||||
JsonNode input = objectMapper.readTree(invoker.input);
|
||||
assertEquals("分析连接池耗尽",
|
||||
input.path("diagnosis_context").path("query").asText());
|
||||
assertEquals("NORMAL", input.path("planner_mode").asText());
|
||||
assertFalse(input.has("executor_output"));
|
||||
assertSame(config, invoker.config);
|
||||
assertEquals("COMPLETED", update.get(DiagnosisGraphState.PLANNER_STATUS));
|
||||
assertEquals(List.of("查询连接池指标"),
|
||||
((Map<?, ?>) update.get(DiagnosisGraphState.PLANNER_PLAN)).get("plan"));
|
||||
assertEquals(1, ((List<?>) update.get(
|
||||
DiagnosisGraphState.ORCHESTRATION_EVENTS)).size());
|
||||
}
|
||||
|
||||
private static final class CapturingInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final String output;
|
||||
private String input;
|
||||
private RunnableConfig config;
|
||||
|
||||
private CapturingInvoker(String output) {
|
||||
this.output = output;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) {
|
||||
this.input = input;
|
||||
this.config = config;
|
||||
return output;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.alibaba.cloud.ai.graph.agent.ReactAgent;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.ai.chat.messages.AssistantMessage;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.mockito.ArgumentMatchers.eq;
|
||||
import static org.mockito.ArgumentMatchers.same;
|
||||
import static org.mockito.Mockito.mock;
|
||||
import static org.mockito.Mockito.when;
|
||||
|
||||
class ReactAgentDiagnosisInvokerTest {
|
||||
|
||||
@Test
|
||||
void forwardsExactInputAndRunnableConfig() throws Exception {
|
||||
ReactAgent agent = mock(ReactAgent.class);
|
||||
RunnableConfig config = RunnableConfig.builder()
|
||||
.threadId("run-invoker")
|
||||
.addMetadata("runId", "run-invoker")
|
||||
.build();
|
||||
when(agent.call(eq("projected input"), same(config)))
|
||||
.thenReturn(new AssistantMessage("agent output"));
|
||||
|
||||
DiagnosisAgentInvoker invoker = new ReactAgentDiagnosisInvoker(agent);
|
||||
|
||||
assertEquals("agent output", invoker.invoke("projected input", config));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
|
||||
class VerifiedInputNodeTest {
|
||||
|
||||
@Test
|
||||
@SuppressWarnings("unchecked")
|
||||
void projectsOnlyClaimsWithExactlyMatchedPassedBindings() throws Exception {
|
||||
Map<String, Object> passedBinding = Map.of(
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 11L,
|
||||
"raw_path", "$.alerts[0]",
|
||||
"evidence_excerpt", "active=50 max=50");
|
||||
Map<String, Object> failedBinding = Map.of(
|
||||
"tool_name", "query_logs",
|
||||
"source_invocation_id", 12L,
|
||||
"raw_path", "$.logs[0]",
|
||||
"evidence_excerpt", "suspected leak");
|
||||
Map<String, Object> executorOutput = Map.of(
|
||||
"answer_version", "executor_evidence_v2",
|
||||
"claims", List.of(
|
||||
Map.of("claim_id", "c1", "claim_text", "连接数达到上限",
|
||||
"claim_type", "symptom", "support_level", "direct",
|
||||
"evidence_bindings", List.of(passedBinding, failedBinding)),
|
||||
Map.of("claim_id", "c2", "claim_text", "存在连接泄漏",
|
||||
"claim_type", "root_cause", "support_level", "indirect",
|
||||
"evidence_bindings", List.of(failedBinding))),
|
||||
"hypotheses", List.of(Map.of("hypothesis_text", "must-not-leak")));
|
||||
Map<String, Object> gatekeeper = Map.of(
|
||||
"status", "fail",
|
||||
"severity", "low_confid",
|
||||
"checked_bindings", List.of(
|
||||
Map.of("claim_id", "c1", "status", "pass",
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 11L,
|
||||
"raw_path", "$.alerts[0]",
|
||||
"matched_text", "active=50 max=50"),
|
||||
Map.of("claim_id", "c1", "status", "fail",
|
||||
"tool_name", "query_logs",
|
||||
"source_invocation_id", 12L,
|
||||
"raw_path", "$.logs[0]"),
|
||||
Map.of("claim_id", "c3", "status", "pass",
|
||||
"tool_name", "query_metrics",
|
||||
"source_invocation_id", 99L,
|
||||
"raw_path", "$.alerts[9]",
|
||||
"matched_text", "unreferenced")));
|
||||
VerifiedInputNode node = new VerifiedInputNode();
|
||||
|
||||
Map<String, Object> update = node.apply(new OverAllState(Map.of(
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, executorOutput,
|
||||
DiagnosisGraphState.GATEKEEPER_RESULT, gatekeeper,
|
||||
DiagnosisGraphState.EVIDENCE_RETRY_COUNT, 0)),
|
||||
RunnableConfig.builder().threadId("run-verified").build()).get();
|
||||
|
||||
Map<String, Object> verifiedOutput = (Map<String, Object>) update.get(
|
||||
DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT);
|
||||
List<Map<String, Object>> claims =
|
||||
(List<Map<String, Object>>) verifiedOutput.get("claims");
|
||||
assertEquals(List.of("c1"),
|
||||
claims.stream().map(claim -> claim.get("claim_id")).toList());
|
||||
assertEquals(List.of(passedBinding), claims.get(0).get("evidence_bindings"));
|
||||
assertFalse(verifiedOutput.containsKey("hypotheses"));
|
||||
List<Map<String, Object>> evidence =
|
||||
(List<Map<String, Object>>) update.get(
|
||||
DiagnosisGraphState.VERIFIED_EVIDENCE);
|
||||
assertEquals(List.of("active=50 max=50"),
|
||||
evidence.stream().map(item -> item.get("matched_text")).toList());
|
||||
assertEquals(1, update.get(DiagnosisGraphState.VERIFIED_BINDING_COUNT));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,83 @@
|
||||
package com.superbiz.agent.graph.diagnosis;
|
||||
|
||||
import com.alibaba.cloud.ai.graph.OverAllState;
|
||||
import com.alibaba.cloud.ai.graph.RunnableConfig;
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.assertEquals;
|
||||
import static org.junit.jupiter.api.Assertions.assertFalse;
|
||||
|
||||
class VerifierNodeAdapterTest {
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
|
||||
@Test
|
||||
void separatesModelAndEffectiveVerdictWithStableWhitelistedInput() throws Exception {
|
||||
CapturingInvoker invoker = new CapturingInvoker("""
|
||||
{
|
||||
"verdict": "PASS",
|
||||
"groundedness_score": 0.9,
|
||||
"critical_fact_count": 1,
|
||||
"facts_checked": [{
|
||||
"fact": "c1: 连接数达到上限",
|
||||
"is_critical": true,
|
||||
"verification": "direct_evidence",
|
||||
"detail": "指标直接支持",
|
||||
"evidence_refs": []
|
||||
}],
|
||||
"rationale": "证据充分"
|
||||
}
|
||||
""");
|
||||
VerifierNodeAdapter adapter = new VerifierNodeAdapter(invoker);
|
||||
Map<String, Object> common = Map.of(
|
||||
DiagnosisGraphState.DIAGNOSIS_CONTEXT, Map.of("query", "分析连接池"),
|
||||
DiagnosisGraphState.VERIFIED_EXECUTOR_OUTPUT, Map.of(
|
||||
"answer_version", "executor_evidence_v2", "claims", List.of()),
|
||||
DiagnosisGraphState.VERIFIED_EVIDENCE, List.of(Map.of(
|
||||
"claim_id", "c1", "matched_text", "active=50 max=50")),
|
||||
DiagnosisGraphState.GATEKEEPER_RESULT, Map.of(
|
||||
"status", "fail", "severity", "low_confid"),
|
||||
DiagnosisGraphState.VERIFIER_VERDICT_CEILING, "LOW_CONFID",
|
||||
DiagnosisGraphState.EXECUTOR_OUTPUT, "must-not-leak");
|
||||
|
||||
Map<String, Object> first = adapter.apply(
|
||||
new OverAllState(new java.util.LinkedHashMap<>(common)),
|
||||
RunnableConfig.builder().threadId("run-verifier").build()).get();
|
||||
Map<String, Object> retryState = new java.util.LinkedHashMap<>(common);
|
||||
retryState.put(DiagnosisGraphState.VERIFIER_RETRY_COUNT, 1);
|
||||
Map<String, Object> second = adapter.apply(
|
||||
new OverAllState(retryState),
|
||||
RunnableConfig.builder().threadId("run-verifier").build()).get();
|
||||
|
||||
JsonNode input = objectMapper.readTree(invoker.inputs.get(0));
|
||||
assertFalse(input.has("executor_output"));
|
||||
assertFalse(input.has("tool_trace_summary"));
|
||||
assertEquals(invoker.inputs.get(0), invoker.inputs.get(1));
|
||||
assertEquals("COMPLETED", first.get(DiagnosisGraphState.VERIFIER_STATUS));
|
||||
assertEquals("PASS", first.get(DiagnosisGraphState.VERIFIER_MODEL_VERDICT));
|
||||
assertEquals("LOW_CONFID", first.get(DiagnosisGraphState.EFFECTIVE_VERDICT));
|
||||
assertEquals("COMPLETED", second.get(DiagnosisGraphState.VERIFIER_STATUS));
|
||||
}
|
||||
|
||||
private static final class CapturingInvoker implements DiagnosisAgentInvoker {
|
||||
|
||||
private final String output;
|
||||
private final List<String> inputs = new ArrayList<>();
|
||||
|
||||
private CapturingInvoker(String output) {
|
||||
this.output = output;
|
||||
}
|
||||
|
||||
@Override
|
||||
public String invoke(String input, RunnableConfig config) {
|
||||
inputs.add(input);
|
||||
return output;
|
||||
}
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user