40 lines
2.9 KiB
Markdown
40 lines
2.9 KiB
Markdown
## MODIFIED Requirements
|
|
|
|
### Requirement: End-to-end MVP acceptance case is documented
|
|
The project SHALL include an end-to-end acceptance case that demonstrates Maven start-up, chat diagnosis, exact trace query, orchestration trace inspection, feedback submission, log inspection, and exact database verification using the same `sessionId + runId`.
|
|
|
|
#### Scenario: Reviewer follows the acceptance case
|
|
- **WHEN** a reviewer follows the documented MVP demo acceptance steps
|
|
- **THEN** they can run the application, submit a diagnosis question, query the exact trace endpoint, inspect `run.orchestrationTrace`, and submit feedback for the same run
|
|
- **AND** they can correlate that run with new logs and exact DiagnosisRun/AgentStep/ToolInvocation database records
|
|
|
|
### Requirement: MVP demo SHALL be reproducible for interviews
|
|
The MVP demo SHALL provide a repeatable way to show a diagnosis answer, exact run trace, StateGraph orchestration summary, verifier evaluation, and feedback.
|
|
|
|
#### Scenario: interview demo check script records an evidence bundle
|
|
- **WHEN** the user runs the interview demo check script against a running `mvp-demo` service
|
|
- **THEN** the script SHALL submit a fixed Chat diagnosis request
|
|
- **AND** it SHALL fetch the trace for the same `sessionId + runId`
|
|
- **AND** it SHALL fail if `run.orchestrationTrace` or its version/final node/termination reason is missing
|
|
- **AND** it SHALL submit useful feedback for that run
|
|
- **AND** it SHALL write chat, trace, feedback, and summary outputs under the configured output directory
|
|
- **AND** summary SHALL include final node, termination reason, degraded, transition count, and evidence retry count
|
|
|
|
#### Scenario: interview demo check fails with actionable readiness output
|
|
- **WHEN** the target service is not reachable
|
|
- **THEN** the script SHALL fail before issuing diagnosis requests
|
|
- **AND** the failure message SHALL name the base URL and the expected startup profile
|
|
|
|
#### Scenario: interview documentation explains audit fields
|
|
- **WHEN** an interviewer asks how Prompt, Gatekeeper, or Graph routing changes are audited
|
|
- **THEN** the demo documentation SHALL point to `prompt_audit.version`, `gatekeeper_result.rule_set_version`, and `run.orchestrationTrace`
|
|
- **AND** it SHALL explain that deterministic eval fixtures and the Graph test suite are the regression source of truth
|
|
|
|
### Requirement: MVP demo SHALL provide a trace inspection checklist
|
|
The MVP demo SHALL document which trace fields to inspect for StateGraph routing, evidence, verifier behavior, and run-level auditability.
|
|
|
|
#### Scenario: Checklist maps fields to interview claims
|
|
- **WHEN** a developer reviews an exact trace response
|
|
- **THEN** the checklist SHALL map `run.orchestrationTrace` version/transitions/final node/termination reason/degraded/evidence retry count to Graph routing claims
|
|
- **AND** it SHALL map AgentStep, ToolInvocation, self-evaluation, Prompt audit, Gatekeeper audit, answer, and feedback paths to their separate responsibilities
|