feat(harness): add information gain stop and audit

2026-07-27 01:03:34 +08:00
parent de5a5b09d9
commit d0452184ee
92 changed files with 5019 additions and 122 deletions
@@ -0,0 +1,25 @@
+## MODIFIED Requirements
+
+### Requirement: RAG Tool contract SHALL expose only bounded document evidence
+The RAG business Request SHALL contain only `query`. The canonical RAG Result SHALL contain `evidence_status`, `tool_call_id`, `query`, bounded `evidence`, `returned_count`, optional normalized `relevance_level`, and `truncated`; each evidence item SHALL contain only `document_id`, `source`, `title`, `breadcrumb`, and an exact `excerpt`. The RAG projector SHALL accept upstream `relevanceLevel` or `relevance_level` and normalize recognized values without exposing raw relevance scores or retrieval traces.
+
+#### Scenario: RAG evidence is serialized
+- **WHEN** a RAG result contains a matching document excerpt and an upstream relevance level
+- **THEN** its canonical JSON preserves the bounded evidence and normalized `relevance_level` while excluding ContextPack, RetrievalTrace, RerankTrace, raw scores, fallback attempts, metadata, and full document bodies
+
+#### Scenario: RAG query has no evidence
+- **WHEN** RAG executes successfully without a usable document excerpt
+- **THEN** it returns `NO_EVIDENCE`, preserves the original query and framework Tool Call ID, returns an empty evidence list, and does not upgrade relevance into evidence
+
+## ADDED Requirements
+
+### Requirement: Tool results SHALL have separate Harness and model views
+Each successful evidence Tool result SHALL provide a Harness Control View and a bounded Model Observation derived from the same canonical result. The control view MAY contain returned counts, normalized scope, relevance, truncation and duplicate identity. The Model Observation SHALL contain only fields needed to understand and cite the result and SHALL NOT contain raw responses, internal scores, retrieval traces, duplicate fingerprints, counters, thresholds, budgets or store identities.
+
+#### Scenario: Model receives RAG observation
+- **WHEN** a canonical RAG result is READY
+- **THEN** the model receives Tool Call ID, actual query scope, bounded evidence, evidence status, optional coarse relevance and truncation, but not raw scores or Harness counters
+
+#### Scenario: Harness evaluates duplicate scope
+- **WHEN** the same normalized Tool scope is requested again
+- **THEN** the Harness can compare its control view identity without exposing that fingerprint to the model