37 lines
1.7 KiB
Markdown
37 lines
1.7 KiB
Markdown
# Brief: chat-verifier-agent
|
|
|
|
## Background
|
|
|
|
The complex Chat path previously returned Executor answers without a synchronous quality gate. Existing rule scoring was asynchronous and post-hoc, so it could not prevent unsupported answers from reaching users.
|
|
|
|
## Goals
|
|
|
|
1. Add a Verifier Agent after Executor in the complex chat path.
|
|
2. Require structured verifier output with `PASS`, `LOW_CONFID`, or `REJECT`.
|
|
3. Route final user output in code based on verifier verdict.
|
|
4. Persist verifier results under `diagnosis_session.self_evaluation.verifier_evaluation`.
|
|
5. Preserve rule scoring under `rule_evaluation`.
|
|
6. Make verifier decisions traceable to real tool invocations through `evidence_refs` and `source_invocation_ids`.
|
|
|
|
## Scope
|
|
|
|
- `ChatService`: explicit `planner -> executor -> verifier` orchestration, max two rounds, verdict routing, retry context, verifier persistence.
|
|
- `VerifierInputHook`: explicit verifier input payload.
|
|
- `ToolTraceSummaryService`: evidence summary from persisted tool calls.
|
|
- `VerifierContextHolder`: round-local verifier context.
|
|
- `SelfEvaluationMergeService`: safe JSON merge for evaluation channels.
|
|
- `AgentLoggingHook`: concise verifier thought and fuller structured output retention.
|
|
- `chat-verifier-prompt.md`: verifier contract, verdict matrix, and traceability schema.
|
|
|
|
## Non-Goals
|
|
|
|
- Verifier does not call tools.
|
|
- Verifier does not rewrite Executor output.
|
|
- Single-agent chat path remains outside this change.
|
|
- No database schema migration is included.
|
|
- Document-path-level evidence attribution is deferred; current traceability is invocation-level with source document labels.
|
|
|
|
## Related OpenSpec
|
|
|
|
`openspec/changes/archive/2026-07-03-chat-verifier-agent/`
|