feat(harness): complete protocol repair stop and archive ISS-016

Add repairable INVALID_PROGRESS_PROTOCOL observations, independent PROGRESS_PROTOCOL_VIOLATED saturation, and controlled release paths. Archive the OpenSpec change after syncing main specs and devflow.
2026-07-27 19:10:07 +08:00
parent 5c369f3b6c
commit 38f781b157
44 changed files with 977 additions and 80 deletions
@@ -40,6 +40,8 @@ Agent-facing Schema 使用三个具体输入类型：`RagToolCall`、`QueryLogsT

 首个 Tool Call 以及上一结果已由 Harness 确定性评价时允许省略 `previous_observation`。存在待评价调用时，新 Tool Call 必须携带完全匹配的上一 Tool Call ID 和二值评价；缺失、错序或跨 Run 引用返回有界协议错误且不执行 Tool。

+协议错误返回可修正 Model Observation，而不是只返回稳定错误码。Observation 可包含 `repair_required=true`、`violation_type`、`missing_field`、`expected_previous_tool_call_id` 和允许的 `information_gain` 值；这些字段均来自 Harness 控制状态和安全 Tool Call ID，不包含业务参数、scope、预算、Prompt、raw response 或内部异常。
+
 ### 3. 先应用上一轮评价，再决定本轮是否放行

 interceptor 的固定顺序为：
@@ -78,6 +80,12 @@ tracker 记录控制指令已交付。下一轮模型仍发起 Tool Call时，in

 替代方案是无限返回 STOP_REQUIRED。拒绝，因为模型无视指令时仍会消耗模型预算并重现原问题。

+### 6.1 连续协议错误也进入受控停止
+
+`INVALID_PROGRESS_PROTOCOL` 表示模型没有遵守 Tool Call Envelope 进展协议，不表示 Tool 内容无信息增益。因此它不累计 `NO_GAIN`，而是单独维护连续协议错误计数。首次或未达阈值的协议错误返回可修正 observation，给模型一次按字段修复的机会；达到阈值后 tracker 进入 `SATURATED`，`stop_reason=PROGRESS_PROTOCOL_VIOLATED`，当前请求收到一次 `STOP_REQUIRED`。
+
+如果模型在 `STOP_REQUIRED` 后继续请求 Tool，沿用既有 `DiagnosisCollectionStoppedException` 受控停止路径。Release 仅在 ProgressSnapshot 有当前 Run 已验真 observed facts 时发布 `INSUFFICIENT_EVIDENCE`；没有安全进展时继续 fail closed，不伪造用户可见事实。
+
 ### 7. Agent 执行返回 Draft 与停止投影，而不是只返回 Draft

 `DiagnosisAgentUseCase` 返回内部 `DiagnosisAgentExecution`：可选 Draft、`ProgressSnapshot` 和可选 `DiagnosisStopReason`。正常 Draft、强制饱和停止和预算异常都在 Diagnosis executor 边界形成 Release 输入。
@@ -109,6 +117,8 @@ Prompt 不写 Tool 名、Schema、阈值、计数器、Projector、预算或 `ne

 Trace 增加有界事件或字段，记录 Tool Call ID、Tool name、scope 摘要、information gain 的生产者（Harness/Model）、连续计数变化、collection state 和 stop reason。不得记录 Prompt、模型 thought、raw Tool response、完整 SQL 参数、预算余量或模型评价理由。

+协议拒绝 Trace 在稳定 `error_code` 之外记录安全字段：`violation_type`、`repair_prompt_delivered`、`consecutive_protocol_violations` 和达到阈值时的 `stop_reason`。Trace 不记录缺失字段对应的业务值、上一轮观察正文、Tool 参数或异常文本。
+
 ### 11. Run 总账与模型调用明细使用同一份 Provider Usage

 `RunContext` 增加最小线程安全模型调用账本，只维护组件轮次、已审计调用数、Usage 不可用调用数和 Token 合计。每次实际模型调用在预算放行后取得组件轮次；`HarnessModelInterceptor` 负责 Diagnosis Agent，`GuardModelCall` 负责 Router、System Chat、Knowledge Answer、Evidence Repair 和 Semantic Guard。两条入口都从 Spring AI `Usage` 读取同一组 input/output Token，先登记调用明细，再交给现有 `RunBudget` 累加总账。
@@ -18,6 +18,7 @@ Diagnosis Agent 面对知识库未知、日志为空或查询条件不足的问
 - 精简中文 Diagnosis Prompt，明确模型不必须得出根因、允许零次 Tool 调用、正确但对当前推导无用的内容属于 `NO_GAIN`，且合法放弃是成功完成。
 - 增加最小模型调用审计：按组件和轮次记录 input/output/total Token，回填 Diagnosis AgentStep Token，并在 Run 结束时与预算总账对账；不记录 Prompt、模型正文或推理内容。
 - 对进入 Harness 后被协议、饱和或重复 scope 门禁拒绝的 Tool 请求记录安全 Trace，使预算 Tool 计数、实际执行和拒绝决策可区分；不记录 Tool 参数或原始响应。
+- 对 `INVALID_PROGRESS_PROTOCOL` 增加可修正的模型观察：明确缺失/错序的协议字段、期望上一轮 Tool Call ID 和允许的 `information_gain` 值；连续协议错误达到阈值后转为受控停止，避免模型反复空转。

 ## Capabilities

@@ -58,6 +59,7 @@ Diagnosis Agent 面对知识库未知、日志为空或查询条件不足的问
 - `CanonicalInvocationStore` 是当前 Run 完整 Tool 调用真相源；Run 内 tracker 只维护控制状态和完成调用索引，不复制 raw response。
 - `RunContext` 继续使用“结构不可变 + 线程安全可变 handle”的既有模式；阈值在 Run 启动时固定。
 - 现有 `SafeFallback.observed_facts / verified_sources / limitations / next_steps` 和前端渲染能力必须复用。
+- 协议错误不等同于 Tool 内容 `NO_GAIN`；它使用独立 `PROGRESS_PROTOCOL_VIOLATED` stop reason，并且只在有可发布 ProgressSnapshot 时进入安全 Fallback。
 - 当前未提交的预算 Fallback 修改保留用户价值，但业务 Release 决策需要从 `ChatApplicationUseCase` 迁移到 `DiagnosisReleaseUseCase`。
 - 当前环境缺少 `codebase-retrieval` 和 LSP；本次影响核查使用 `rg` 引用搜索、源码和测试阅读降级完成。用户已明确允许忽略 GitNexus。

@@ -73,6 +75,7 @@ Diagnosis Agent 面对知识库未知、日志为空或查询条件不足的问

 - 框架 Tool Schema 生成或 ToolInterceptor 参数处理不符合 Envelope 假设，导致模型无法正确回传或业务 request 未被剥离。
 - STOP_REQUIRED 若未形成受控结束，模型可能继续请求 Tool，或 Agent 调用异常绕过统一 Release。
+- 仅返回通用 `INVALID_PROGRESS_PROTOCOL` 可能不足以让模型自修复；必须返回字段级 repair hint，同时设置连续错误兜底。
 - 最后一轮结果可能没有下一次 Tool Call 来回传模型评价；该情况只能表示模型主动结束，不能伪造 `GAINED / NO_GAIN`。
 - ProgressSnapshot 若读取不完整或混入 raw payload，会造成过程缺失或上下文/隐私边界倒退。
 - `conclusion=null` 与现有 EvidenceGuard 的 `ANALYSIS_MISSING` 规则冲突，需要明确区分“验证引用真实性”和“验证结论完整性”。
@@ -60,7 +60,7 @@ At normal Draft completion, information saturation, or budget termination, the H
 - **THEN** it is not projected as an observed fact and the snapshot records a bounded limitation

 ### Requirement: Information stop reasons SHALL remain distinct from release outcomes
-The Harness SHALL distinguish `INFORMATION_SATURATED` from `BUDGET_LIMIT_REACHED`. Release SHALL continue to expose only `SUCCESS`, `FALLBACK`, `FAILED`, or `CANCELLED`, and SHALL use SafeFallback type to distinguish insufficient evidence from missing required context.
+The Harness SHALL distinguish `INFORMATION_SATURATED`, `BUDGET_LIMIT_REACHED`, and `PROGRESS_PROTOCOL_VIOLATED`. Release SHALL continue to expose only `SUCCESS`, `FALLBACK`, `FAILED`, or `CANCELLED`, and SHALL use SafeFallback type to distinguish insufficient evidence from missing required context.

 #### Scenario: Low gain stops before budget exhaustion
 - **WHEN** consecutive no-gain reaches the configured threshold while hard budget remains
@@ -70,6 +70,10 @@ The Harness SHALL distinguish `INFORMATION_SATURATED` from `BUDGET_LIMIT_REACHED
 - **WHEN** model, Tool, Token, byte, or time protection stops the Diagnosis after at least one safe observation
 - **THEN** Trace retains `BUDGET_LIMIT_REACHED` and Release attempts a deterministic `FALLBACK` from the existing progress without an extra model call

+#### Scenario: Progress protocol violations exceed threshold
+- **WHEN** consecutive invalid progress protocol requests reach the configured threshold
+- **THEN** Trace records `PROGRESS_PROTOCOL_VIOLATED`, the model receives one STOP_REQUIRED observation, and public release remains a normal Fallback only if safe progress exists
+
 ### Requirement: Diagnosis Prompt SHALL license bounded abandonment
 The Chinese Diagnosis Prompt SHALL state that a root cause is not mandatory, `conclusion=null` is valid completion, zero Tool calls are allowed when required query context is missing, and correct but non-advancing content is `NO_GAIN`. It SHALL require the model to stop when no distinct bounded query can produce new diagnostic information and to obey STOP_REQUIRED. It SHALL NOT embed Tool names, Tool schemas, thresholds, counters, `next_action`, or Harness implementation details.

@@ -103,8 +107,25 @@ Every supported Tool request rejected by the Harness before a usable business ob

 #### Scenario: Progress protocol is invalid
 - **WHEN** a supported Tool request omits or misorders a required previous observation
- **THEN** no business Tool executes and Trace records `INVALID_PROGRESS_PROTOCOL` for that Tool request
+- **THEN** no business Tool executes, Trace records `INVALID_PROGRESS_PROTOCOL` with a safe violation type, and the model receives a repairable observation naming the missing or expected protocol field

 #### Scenario: Duplicate or saturated request is blocked
 - **WHEN** a supported Tool request repeats a successful normalized scope or arrives after collection saturation
 - **THEN** Trace records the stable rejection reason while canonical invocation count remains unchanged
+
+### Requirement: Invalid progress protocol SHALL be repairable before bounded stop
+When a supported Tool request violates the Tool Envelope progress protocol, the Harness SHALL return a bounded error observation that helps the model repair the next request. The observation MAY include safe protocol fields such as `repair_required`, `violation_type`, `missing_field`, `expected_previous_tool_call_id`, and allowed `information_gain` values. It SHALL NOT include Tool arguments, normalized scope, raw responses, Prompt, model text, budget values, counters except the bounded consecutive protocol violation count, or internal exception text.
+
+Consecutive invalid progress protocol requests SHALL be counted independently from `NO_GAIN`. Reaching the Run's configured protocol-violation threshold SHALL set stop reason `PROGRESS_PROTOCOL_VIOLATED` and deliver one `STOP_REQUIRED` observation. A later Tool request after that instruction SHALL terminate through the controlled-stop path.
+
+#### Scenario: Missing previous observation is repairable
+- **WHEN** a non-empty Tool observation is pending semantic evaluation and the next Tool request omits `previous_observation`
+- **THEN** the Tool is not executed and the model receives a repairable `INVALID_PROGRESS_PROTOCOL` observation containing `missing_field=previous_observation` and the expected previous Tool Call ID
+
+#### Scenario: Wrong previous observation id is repairable
+- **WHEN** a pending Tool observation exists and the next Tool request references a different `previous_observation.tool_call_id`
+- **THEN** the Tool is not executed and the model receives a repairable `INVALID_PROGRESS_PROTOCOL` observation containing `violation_type=OUT_OF_ORDER_PREVIOUS_OBSERVATION`
+
+#### Scenario: Repeated repair failure stops collection
+- **WHEN** the model repeats invalid progress protocol requests until the configured threshold is reached
+- **THEN** the current Tool is not executed, the model receives `STOP_REQUIRED` with `reason=PROGRESS_PROTOCOL_VIOLATED`, and any further Tool request ends as a controlled stop
@@ -11,6 +11,10 @@ The Diagnosis Agent SHALL expose only the frozen `lookup_knowledge`, `query_logs
 - **WHEN** the last successful Tool result is pending semantic evaluation and the model requests another Tool
 - **THEN** the Envelope must identify that exact prior Tool Call and contain `GAINED` or `NO_GAIN` before the new business Tool can execute

+#### Scenario: Model omits required progress field
+- **WHEN** a prior non-empty Tool observation is pending and the model requests another Tool without `previous_observation`
+- **THEN** the business Tool does not execute and the Agent receives a bounded repair observation explaining the missing `previous_observation` field and expected prior Tool Call ID
+
 #### Scenario: Tool execution fails
 - **WHEN** a registered adapter returns an error result
 - **THEN** the Agent receives `evidence_status=ERROR`, the framework Tool Call ID and a stable error code without automatic Tool retry or raw failure detail
@@ -41,7 +45,7 @@ The internal Agent use case SHALL distinguish a valid Draft, controlled informat

 #### Scenario: Model ignores STOP_REQUIRED
 - **WHEN** the framework surfaces the typed collection-stopped signal after the final completion opportunity
- **THEN** the Agent use case returns no Draft with `INFORMATION_SATURATED` and the current ProgressSnapshot
+- **THEN** the Agent use case returns no Draft with the controlled stop reason and the current ProgressSnapshot

 #### Scenario: Unclassified framework failure
 - **WHEN** Agent execution throws an exception unrelated to controlled stop, cancellation, or budget termination
@@ -49,3 +49,12 @@
 - [x] 7.3 为进展协议错误、重复 scope、信息饱和和观察合同拒绝增加安全 `TOOL_REQUEST_REJECTED` Trace，证明不记录参数或原始响应
 - [x] 7.4 在 Run 完成时持久化 step_count 并记录 Token 总账/明细对账结果，补充 Store 和 Trace 回归测试
 - [x] 7.5 运行 focused、Harness 全量回归、strict validate 和真实 SSE/数据库 E2E，核对 Tool 执行/拒绝数量、Token 对账和 Trace 连续性
+
+## 8. 协议修复反馈与兜底停止
+
+- [x] 8.1 为进展协议错误建模安全 `violation_type`，覆盖缺失 `previous_observation`、错序 Tool Call ID、意外 previous observation、缺失 `input` 和非法 Envelope
+- [x] 8.2 将 `INVALID_PROGRESS_PROTOCOL` observation 改为可修正响应，包含安全 repair hint、期望上一轮 Tool Call ID 和允许的 `information_gain` 值，不泄露业务参数或原始响应
+- [x] 8.3 为 Run tracker 增加连续协议错误计数和 `PROGRESS_PROTOCOL_VIOLATED` stop reason；未达阈值打回修正，达到阈值返回一次 `STOP_REQUIRED`
+- [x] 8.4 Release/Agent 受控停止路径支持 `PROGRESS_PROTOCOL_VIOLATED`：有安全 ProgressSnapshot 时发布 `INSUFFICIENT_EVIDENCE`，无进展时继续 fail closed
+- [x] 8.5 扩展 `TOOL_REQUEST_REJECTED` 审计字段，记录安全 `violation_type`、`repair_prompt_delivered`、连续协议错误次数和 stop reason，并用测试证明不含 Tool 参数、上一轮观察正文或内部异常
+- [x] 8.6 运行 tracker、interceptor、release、agent-loop focused tests 和 strict OpenSpec validate，更新 ISS-015/ISS-016 进度