docs: reorganize MVP interview documentation

interview/aiops-lightweight-verifier.md

@@ -1,38 +1,37 @@
-# AIOps Lightweight Verifier
+# AIOps 轻量规则验证器
 
-## What Changed
+## 1. 改动是什么
 
-AIOps now has a deterministic post-run quality gate.
+AIOps 现在有一个确定性的后置质量门禁。最终告警报告持久化后，`AiOpsRuleEvaluationService` 会检查：
 
-After the final AIOps report is persisted, the service evaluates:
+- 最终报告是否存在，且不是明显过短。
+- payload 模式下，报告是否提到输入的告警和服务。
+- 是否有证据工具调用，例如 `lookup_knowledge`、`query_metrics`、`query_logs`。
 
-- whether the final report exists and is not trivially short
-- whether a payload-targeted report mentions the supplied alert and service
-- whether evidence tools such as `lookup_knowledge`, `query_metrics`, or `query_logs` were persisted
-
-The result is stored under:
+结果写入：
 
 ```text
 diagnosis_session.self_evaluation.aiops_rule_evaluation
 ```
 
-The trace API returns this payload through the existing session self-evaluation field.
+Trace API 会通过 session self-evaluation 展示这个结果。
 
-## Why Rule-Based First
+## 2. 为什么先做规则型
 
-This is not a full LLM verifier yet.
+这还不是完整 LLM Verifier。
 
-The first AIOps quality risks are concrete and easy to check with rules:
+AIOps 第一阶段质量风险比较具体，适合先用规则：
 
-- Did the report stay focused on the payload?
-- Did the run use evidence tools?
-- Did the system produce a usable final report?
+- 报告有没有生成。
+- 报告有没有聚焦 payload。
+- 有没有使用证据工具。
+- 有没有把无关告警展开成主诊断对象。
 
-Rule evaluation is stable, cheap, and easy to explain. It also avoids adding another hidden model call to the AIOps flow before the current trace contract is mature.
+规则验证稳定、便宜、容易解释，也不会在当前链路里额外引入一次隐藏模型调用。
 
-## Verdicts
+## 3. 判定结果
 
-The evaluator emits:
+当前评估器输出：
 
 ```text
 PASS
@@ -40,14 +39,36 @@ WARN
 FAIL
 ```
 
-`FAIL` is reserved for critical issues such as a missing or too-short report. Missing payload focus terms or missing evidence tools currently produce `WARN`, because valid reports may use slightly different wording or evidence may be unavailable in a mock/demo environment.
+含义：
 
-## Interview Answer
+- `PASS`：核心检查通过。
+- `WARN`：报告存在，但可能缺少 payload 关键词或证据工具。
+- `FAIL`：缺少最终报告、报告过短等关键问题。
 
-If asked why AIOps has a verifier now:
+缺少 payload 关键词或证据工具先给 `WARN`，因为 demo/mock 环境下证据可能不可用，且报告措辞可能与 payload 字段不完全一致。
 
-> Chat already has an LLM verifier because the user questions are open-ended. For AIOps, I started with a lighter rule-based verifier because the first quality checks are very concrete: payload focus, evidence coverage, and report completeness. The evaluation is persisted into `self_evaluation`, so the trace can show not only what the Agent did, but also whether the output passed basic quality gates.
+## 4. 面试回答
 
-If asked why not use the Chat verifier directly:
+如果被问：为什么 AIOps 也需要验证器？
+
+```text
+Chat 已经有 LLM Verifier，因为用户问题开放度高。
+AIOps 的第一阶段质量风险更明确：报告是否聚焦输入告警、是否使用证据工具、报告是否完整。
+所以我先做了轻量规则验证器，把结果写入 self_evaluation，让 Trace 不只展示 Agent 做了什么，也展示输出是否通过基础质量门。
+```
+
+如果被问：为什么不直接复用 Chat Verifier？
+
+```text
+AIOps 验证语义和 Chat 不一样。
+它要检查 alert scope、payload focus、证据工具覆盖，以及是否过度展开无关 active alerts。
+直接复用 Chat Verifier 会混淆这些语义。
+规则评估先提供稳定质量门，后续 AIOps LLM Verifier 可以基于同一套 trace contract 扩展。
+```
+
+## 5. 后续增强
+
+- 引入 AIOps LLM Verifier，逐条校验根因和建议是否有 evidence refs。
+- 把 rule evaluation 的 checks 在 Trace API 中结构化展示。
+- 将 payload scope violation 沉淀为 bad case。
 
-> AIOps verification is different from Chat verification. It needs to check alert scope, evidence tool coverage, and whether unrelated active alerts were over-expanded. Reusing the Chat verifier directly would blur those semantics. The rule-based evaluator gives us a stable first quality gate; a later AIOps LLM verifier can build on the same trace contract.