431 lines
14 KiB
Markdown
431 lines
14 KiB
Markdown
# 当前 MVP 架构
|
||
|
||
**更新日期**：2026-07-08
|
||
**状态**：当前可运行架构
|
||
**适用范围**：Demo、面试讲解、后续迭代规划
|
||
|
||
## 1. 系统定位
|
||
|
||
SuperBizAgent MVP 不是通用 Chatbot，而是面向故障诊断的 Agent 工程项目。
|
||
|
||
核心目标：
|
||
|
||
- 支持用户主动发起的 Chat 诊断。
|
||
- 支持 AIOps 告警触发的自动诊断。
|
||
- 保留 Agent 的规划、执行、验证过程。
|
||
- 工具调用必须显式、可追踪、可回放。
|
||
- RAG 检索必须通过 `lookup_knowledge` 暴露证据链。
|
||
- 每次诊断都沉淀 session、step、tool invocation 和 self evaluation。
|
||
|
||
## 2. 总体分层
|
||
|
||
```mermaid
|
||
flowchart TB
|
||
subgraph API["API Layer"]
|
||
ChatController["ChatController"]
|
||
TraceController["DiagnosisTraceController"]
|
||
SearchController["SearchController"]
|
||
DocumentController["DocumentController"]
|
||
end
|
||
|
||
subgraph App["Application Service"]
|
||
ChatService["ChatService"]
|
||
AiOpsService["AiOpsService"]
|
||
TraceService["DiagnosisTraceService"]
|
||
end
|
||
|
||
subgraph Agent["Agent Orchestration"]
|
||
Supervisor["Supervisor"]
|
||
Planner["Planner"]
|
||
Executor["Executor"]
|
||
Gatekeeper["Gatekeeper"]
|
||
Verifier["Verifier"]
|
||
Composer["Composer"]
|
||
end
|
||
|
||
subgraph Tools["Evidence Tools"]
|
||
KnowledgeTool["lookup_knowledge"]
|
||
LogsTool["query_logs"]
|
||
MetricsTool["query_metrics"]
|
||
AlertsTool["queryPrometheusAlerts"]
|
||
end
|
||
|
||
subgraph Skills["Skill / Playbook"]
|
||
SkillRegistry["SkillRegistry"]
|
||
PlannerSkillHook["PlannerSkillMetadataHook"]
|
||
SkillsHook["SkillsAgentHook"]
|
||
ReadSkill["read_skill"]
|
||
end
|
||
|
||
subgraph RAG["RAG Retrieval"]
|
||
L0["KnowledgeIndexService"]
|
||
VectorSearch["VectorSearchService"]
|
||
VectorStore["Spring AI VectorStore"]
|
||
SdkFallback["Milvus SDK fallback"]
|
||
end
|
||
|
||
subgraph Store["Persistence and Trace"]
|
||
Session["diagnosis_session"]
|
||
Step["agent_step"]
|
||
Invocation["tool_invocation"]
|
||
ApiDoc["api_document"]
|
||
Milvus["Milvus/Zilliz"]
|
||
end
|
||
|
||
API --> App
|
||
ChatService --> Agent
|
||
AiOpsService --> Agent
|
||
SkillRegistry --> PlannerSkillHook
|
||
PlannerSkillHook --> Planner
|
||
SkillRegistry --> SkillsHook
|
||
SkillsHook --> Executor
|
||
Executor --> ReadSkill
|
||
Agent --> Tools
|
||
KnowledgeTool --> RAG
|
||
RAG --> Store
|
||
Tools --> Invocation
|
||
Agent --> Step
|
||
App --> Session
|
||
TraceService --> Session
|
||
TraceService --> Step
|
||
TraceService --> Invocation
|
||
```
|
||
|
||
```text
|
||
API Layer
|
||
-> ChatController
|
||
-> DiagnosisTraceController
|
||
-> SearchController
|
||
-> DocumentController
|
||
|
||
Application Service
|
||
-> ChatService
|
||
-> AiOpsService
|
||
-> DiagnosisTraceService
|
||
|
||
Agent Orchestration
|
||
-> Supervisor
|
||
-> Planner
|
||
-> Executor
|
||
-> Gatekeeper
|
||
-> Verifier
|
||
-> Composer
|
||
|
||
Evidence Tools
|
||
-> lookup_knowledge
|
||
-> query_logs
|
||
-> query_metrics
|
||
-> queryPrometheusAlerts
|
||
|
||
Skill / Playbook
|
||
-> SkillRegistry
|
||
-> PlannerSkillMetadataHook gives Planner name/description only
|
||
-> SkillsAgentHook gives Executor read_skill
|
||
-> Verifier is isolated from skills
|
||
|
||
RAG Retrieval
|
||
-> KnowledgeIndexService
|
||
-> VectorSearchService
|
||
-> Spring AI VectorStore
|
||
-> Milvus SDK fallback
|
||
|
||
Persistence
|
||
-> diagnosis_session
|
||
-> agent_step
|
||
-> tool_invocation
|
||
-> api_document
|
||
-> Milvus/Zilliz collection
|
||
|
||
Quality Gates
|
||
-> executor gatekeeper
|
||
-> chat verifier
|
||
-> AIOps rule evaluation
|
||
-> diagnosis eval baseline
|
||
-> RAG retrieval baseline
|
||
```
|
||
|
||
## 3. Chat 诊断链路
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
autonumber
|
||
actor User as 用户
|
||
participant API as POST /api/chat
|
||
participant Chat as ChatService
|
||
participant Planner as Planner Agent
|
||
participant Executor as Executor Agent
|
||
participant Tool as Evidence Tools
|
||
participant Gatekeeper as Gatekeeper Hook
|
||
participant Verifier as Verifier Agent
|
||
participant Composer as Composer Agent
|
||
participant DB as Trace Tables
|
||
participant Trace as Trace API
|
||
|
||
User->>API: 提交诊断问题
|
||
API->>Chat: execute chat strategy
|
||
Chat->>Planner: 复杂问题进入规划
|
||
Planner->>DB: 写入 agent_step
|
||
Planner->>Executor: 下发排查方向
|
||
Executor->>Tool: lookup_knowledge / logs / metrics
|
||
Tool->>DB: 写入 tool_invocation
|
||
Tool-->>Executor: 返回证据
|
||
Executor->>Gatekeeper: 输出 executor_evidence_v2
|
||
Gatekeeper->>DB: 读取 tool_invocation.evidence_refs 并校验引用
|
||
Gatekeeper->>Verifier: 传入已验真的 claims / excerpts
|
||
Verifier->>DB: 合并 self_evaluation.verifier_evaluation
|
||
Verifier->>Composer: 传入 allowed_claims / missing_info / actions
|
||
Composer->>Chat: 生成最终用户答复
|
||
Chat->>DB: 保存 diagnosis_session.answer
|
||
User->>Trace: GET /api/diagnosis/{sessionId}/trace
|
||
Trace->>DB: 聚合 session / step / tool
|
||
Trace-->>User: 返回可回放诊断链路
|
||
```
|
||
|
||
```text
|
||
POST /api/chat
|
||
-> ChatService
|
||
-> 简单问题：轻量回答
|
||
-> 复杂诊断：Agent 编排
|
||
-> Planner 制定排查方向
|
||
-> Executor 调用证据工具
|
||
-> lookup_knowledge
|
||
-> query_logs
|
||
-> query_metrics
|
||
-> Gatekeeper 校验 Executor 证据引用真实性
|
||
-> Verifier 判断 claim 是否能由已核验证据推出
|
||
-> Composer 生成最终用户答复
|
||
-> 保存 diagnosis_session
|
||
-> 保存 agent_step
|
||
-> 保存 tool_invocation
|
||
-> 合并 self_evaluation.verifier_evaluation
|
||
```
|
||
|
||
Chat 链路的质量门禁由三段组成：Gatekeeper 先做代码级引用验真，Verifier 再做 LLM 可推导性判断，Composer 最后控制对用户的表达边界。Gatekeeper、Verifier、Composer 的输出合并到 `diagnosis_session.self_evaluation.verifier_evaluation`，Trace API 会展示该验证结果。
|
||
|
||
Agent 编排细节见 [agent-orchestration.md](agent-orchestration.md)。
|
||
|
||
关键代码：
|
||
|
||
- `src/main/java/com/superbiz/agent/controller/ChatController.java`
|
||
- `src/main/java/com/superbiz/agent/service/ChatService.java`
|
||
- `src/main/java/com/superbiz/agent/service/SelfEvaluationMergeService.java`
|
||
- `src/main/java/com/superbiz/agent/service/DiagnosisTraceService.java`
|
||
|
||
## 4. AIOps 诊断链路
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Request["POST /api/ai_ops"] --> Payload{"包含告警 payload?"}
|
||
Payload -->|是| Targeted["PAYLOAD_TARGETED"]
|
||
Payload -->|否| Discovery["AUTO_DISCOVERY"]
|
||
|
||
Targeted --> BuildPrompt["构造聚焦 payload 的诊断 prompt"]
|
||
Targeted --> QueryAug["生成 recommended lookup_knowledge query"]
|
||
Discovery --> DiscoverAlert["通过 queryPrometheusAlerts 发现活跃告警"]
|
||
|
||
BuildPrompt --> Plan["Planner 规划排查"]
|
||
QueryAug --> Plan
|
||
DiscoverAlert --> Plan
|
||
|
||
Plan --> Execute["Executor 收集证据"]
|
||
Execute --> Knowledge["lookup_knowledge"]
|
||
Execute --> Metrics["query_metrics / Prometheus"]
|
||
Execute --> Logs["query_logs"]
|
||
|
||
Knowledge --> Report["告警分析报告"]
|
||
Metrics --> Report
|
||
Logs --> Report
|
||
|
||
Report --> RuleEval["AiOpsRuleEvaluationService"]
|
||
RuleEval --> SelfEval["self_evaluation.aiops_rule_evaluation"]
|
||
Report --> Trace["DiagnosisTraceService"]
|
||
SelfEval --> Trace
|
||
```
|
||
|
||
```text
|
||
POST /api/ai_ops
|
||
-> AiOpsService
|
||
-> 判断是否有告警 payload
|
||
-> PAYLOAD_TARGETED
|
||
-> AUTO_DISCOVERY
|
||
-> 构造 AIOps 诊断 prompt
|
||
-> payload 模式补充 recommended lookup_knowledge query
|
||
-> Agent 编排
|
||
-> Planner / Executor
|
||
-> Prometheus / logs / knowledge tools
|
||
-> 生成告警分析报告
|
||
-> AiOpsRuleEvaluationService
|
||
-> 合并 self_evaluation.aiops_rule_evaluation
|
||
-> Trace API 可查看全链路
|
||
```
|
||
|
||
AIOps 保留两种模式：
|
||
|
||
| 模式 | 触发条件 | 行为 |
|
||
|---|---|---|
|
||
| `PAYLOAD_TARGETED` | 请求包含 alertName、service、severity、description、timeRange 等字段 | 以 payload 为唯一主诊断对象，并生成推荐知识库 query |
|
||
| `AUTO_DISCOVERY` | 请求没有明确告警 payload | 先查询当前活跃告警，再选择目标排查 |
|
||
|
||
AIOps 当前使用轻量规则验证器，重点检查：
|
||
|
||
- 最终报告是否存在。
|
||
- payload 模式是否聚焦输入告警。
|
||
- 是否使用关键证据工具，例如 `lookup_knowledge`、日志、指标。
|
||
|
||
关键代码：
|
||
|
||
- `src/main/java/com/superbiz/agent/service/AiOpsService.java`
|
||
- `src/main/java/com/superbiz/agent/service/AiOpsRuleEvaluationService.java`
|
||
|
||
## 5. RAG 位置
|
||
|
||
RAG 不是隐藏在 Chat Advisor 里的隐式能力，而是 Executor 可以显式调用的工具：
|
||
|
||
```mermaid
|
||
flowchart LR
|
||
Executor["Executor Agent"] --> Tool["lookup_knowledge Tool"]
|
||
Tool --> L0["L0 domain/entity hint"]
|
||
Tool --> Search["VectorSearchService"]
|
||
L0 --> Search
|
||
Search --> VectorStore["Spring AI VectorStore"]
|
||
Search --> Fallback["Milvus SDK fallback"]
|
||
VectorStore --> Normalize["score/rawScore/scoreLabel"]
|
||
Fallback --> Normalize
|
||
Normalize --> Evidence["evidence output"]
|
||
Evidence --> Invocation["tool_invocation"]
|
||
Evidence --> Executor
|
||
```
|
||
|
||
```text
|
||
Executor
|
||
-> lookup_knowledge(query)
|
||
-> L0 domain/entity hint
|
||
-> VectorSearchService
|
||
-> Spring AI VectorStore
|
||
-> Milvus SDK fallback
|
||
-> evidence shaping
|
||
-> tool_invocation
|
||
```
|
||
|
||
保留显式工具的原因：
|
||
|
||
- Agent 何时检索、检索什么、证据是什么，必须能在 trace 中解释。
|
||
- AIOps payload 到 query 的业务映射需要项目内控制。
|
||
- `tool_invocation` 是后续评测、回放和面试讲解的核心材料。
|
||
|
||
RAG 总体设计见 [rag-architecture.md](rag-architecture.md)，检索运行细节见 [retrieval-observability.md](retrieval-observability.md)。
|
||
|
||
## 6. 持久化模型
|
||
|
||
当前诊断持久化以三张表为核心：
|
||
|
||
```text
|
||
diagnosis_session
|
||
-> 一次诊断会话的主记录
|
||
-> query / status / agent_flow / answer
|
||
-> self_evaluation
|
||
-> step_count / tool_call_count / duration
|
||
|
||
agent_step
|
||
-> Agent 模型调用步骤
|
||
-> step_index / agent_name
|
||
-> model_input / model_output / thought
|
||
-> duration / token_count
|
||
|
||
tool_invocation
|
||
-> 工具调用事实
|
||
-> tool_name / input_params / output_preview
|
||
-> retrieval_layer / retrieval_details
|
||
-> retrieval_details.evidence_refs
|
||
-> relevance_level / dedup_reason
|
||
-> duration / success
|
||
```
|
||
|
||
说明：
|
||
|
||
- 旧的 `diagnosis_record` 已不是当前主模型，迁移脚本中已经由 `diagnosis_session + agent_step + tool_invocation` 取代。
|
||
- `api_document` 仍用于文档元数据管理。
|
||
- 文档向量内容存放在 Milvus/Zilliz collection 中。
|
||
|
||
会话和 Trace 生命周期见 [session-trace-lifecycle.md](session-trace-lifecycle.md)，完整数据关系见 [data-model.md](data-model.md)。
|
||
|
||
## 7. Trace API
|
||
|
||
```text
|
||
GET /api/diagnosis/{sessionId}/trace
|
||
```
|
||
|
||
Trace API 聚合：
|
||
|
||
- 会话状态和最终报告。
|
||
- Agent step 序列。
|
||
- 工具调用和检索细节。
|
||
- Chat Gatekeeper / Verifier / Composer 结果。
|
||
- AIOps rule evaluation 结果。
|
||
|
||
Trace 是本项目区别于普通问答系统的关键：答案不是孤立文本，而是可以追溯到 Agent 决策、工具调用和证据来源。
|
||
|
||
Prompt、Hook、Gatekeeper、Verifier、Composer 和评测门禁的完整说明见 [harness-quality-gates.md](harness-quality-gates.md)，用户反馈与 `self_evaluation` 闭环见 [feedback-architecture.md](feedback-architecture.md)。
|
||
|
||
## 8. 质量门禁
|
||
|
||
当前质量门禁分层如下：
|
||
|
||
| 门禁 | 位置 | 作用 |
|
||
|---|---|---|
|
||
| Executor Gatekeeper | `VerifierInputHook` / `ExecutorGatekeeperService` | 校验 Executor 引用的 invocation、`raw_path`、`evidence_excerpt` 是否真实 |
|
||
| Chat Verifier | `ChatService` | 判断已验真证据是否能推出 Executor claims |
|
||
| Chat Composer | `ChatService` | 只表达 Verifier 允许输出的内容，避免把 no-evidence 说成已排除 |
|
||
| AIOps Rule Evaluation | `AiOpsRuleEvaluationService` | 校验告警诊断是否聚焦 payload 并使用证据 |
|
||
| Diagnosis Eval Baseline | `mvp/eval/` | 固化诊断 trace 和报告行为 |
|
||
| RAG Retrieval Baseline | `eval/rag-retrieval/` | 固化检索召回行为，避免 RAG 重构回退 |
|
||
| Live RAG Acceptance | `scripts/eval_rag_live_acceptance.py` | 在运行环境中验证重建索引后的真实检索 |
|
||
|
||
## 9. 当前完成状态
|
||
|
||
已经完成：
|
||
|
||
- Chat 和 AIOps 两条入口链路。
|
||
- 显式 `lookup_knowledge` Agent Tool。
|
||
- L0 从最终决策降级为 domain/entity hint。
|
||
- `VectorSearchService` 作为稳定检索门面。
|
||
- Spring AI VectorStore 读取路径。
|
||
- Milvus SDK fallback。
|
||
- `score` / `rawScore` / `scoreLabel` 分数语义拆分。
|
||
- `title`、`breadcrumb`、`content` 参与 embedding 文本。
|
||
- `tool_invocation` 记录检索层、relevance level、dedup reason。
|
||
- Chat verifier 和 AIOps rule evaluation 合并进 `self_evaluation`。
|
||
- Chat Executor 结构化输出 `executor_evidence_v2`，不再直接承担最终用户答复。
|
||
- `tool_invocation.retrieval_details.evidence_refs` 支持 `raw_path` 精确引用和 `$.no_evidence` 负向证据。
|
||
- Gatekeeper 对 Executor 引用做代码级验真，Verifier 只判断可推导性。
|
||
- Composer 在 Verifier 之后生成最终用户表达，并限制 negative observation 过度表述。
|
||
- RAG offline baseline 和 live acceptance 脚本。
|
||
|
||
暂不作为当前已完成能力声明：
|
||
|
||
- 完整 QueryTransformer / MultiQuery。
|
||
- BM25、RRF、cross-encoder rerank。
|
||
- 完整邻居 chunk / section context expansion。
|
||
- VectorStore 写入路径全面迁移。
|
||
- 完整 LLM-based AIOps verifier。
|
||
|
||
后续 Agent 拆分、Skill/Playbook、MCP 工具协议化和进程隔离等方向见 [evolution-roadmap.md](evolution-roadmap.md)。
|
||
|
||
## 10. 关键代码索引
|
||
|
||
| 能力 | 代码 |
|
||
|---|---|
|
||
| Chat 入口与编排 | `ChatController`, `ChatService` |
|
||
| AIOps 入口与编排 | `ChatController.aiOps`, `AiOpsService` |
|
||
| AIOps 规则验证 | `AiOpsRuleEvaluationService` |
|
||
| 知识库工具 | `LookupKnowledgeTool` |
|
||
| L0 hint | `KnowledgeIndexService` |
|
||
| 向量检索门面 | `VectorSearchService` |
|
||
| 文档切片 | `DocumentChunkService` |
|
||
| 向量写入 | `VectorIndexService` |
|
||
| Spring AI VectorStore 配置辅助 | `SpringAiVectorStoreSidecarService` |
|
||
| Trace 聚合 | `DiagnosisTraceService` |
|
||
| 工具调用记录 | `ToolInvocationRecorder` |
|
||
| Executor 引用验真 | `ExecutorGatekeeperService`, `VerifierInputHook` |
|
||
| self_evaluation 合并 | `SelfEvaluationMergeService` |
|