osiman/SuperBizAgent-java
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(eval): add evidence pipeline acceptance closure

Browse Source
This commit is contained in:
aruo
2026-07-09 00:47:48 +08:00
parent a77c947cd4
commit db0f229285
46 changed files with 1434 additions and 53 deletions
Download Patch File Download Diff File Expand all files Collapse all files
openspec/specs/chat-verifier-agent/spec.md
+26 -1
View File
@@ -1,4 +1,4 @@
# chat-verifier-agent Specification
# chat-verifier-agent Specification
## Purpose
TBD - created by archiving change chat-verifier-agent. Update Purpose after archive.
@@ -531,3 +531,28 @@ The Executor prompt SHALL instruct Executor to keep narrow confirmation question
- **THEN** Executor SHOULD output the minimum necessary claims, normally one and at most two
- **AND** those claims SHALL be `observation` or `negative_observation` unless current-session evidence proves more
- **AND** Executor SHALL NOT emit unrelated root-cause, remediation, or excluded-topic claims as confirmed facts
### Requirement: Gatekeeper SHALL expose rule catalog metadata in audit output
Gatekeeper SHALL include the rule catalog version and enabled rule metadata in its validation result.
#### Scenario: Gatekeeper pass includes rule metadata
- **WHEN** Gatekeeper returns `status=pass`
- **THEN** the result SHALL include `rule_set_version`
- **AND** it SHALL include a rule metadata summary
#### Scenario: Gatekeeper fail includes rule metadata
- **WHEN** Gatekeeper returns `status=fail`
- **THEN** the result SHALL include `rule_set_version`
- **AND** it SHALL include a rule metadata summary
#### Scenario: Gatekeeper fallback pass includes rule metadata
- **WHEN** the Verifier input hook returns a fallback Gatekeeper pass because no Gatekeeper service is available
- **THEN** the result SHOULD still include the default rule set version and an empty or default rule metadata summary
### Requirement: Gatekeeper rule catalog SHALL remain deterministic
The Gatekeeper rule catalog SHALL configure metadata and simple parameters only; validation behavior SHALL remain deterministic Java code.
#### Scenario: Rule metadata is lightweight
- **WHEN** rule metadata is loaded
- **THEN** each enabled rule SHOULD expose an id, description, enabled flag, and default severity or relevant parameter
- **AND** rule metadata SHALL NOT execute dynamic scripts