2.9 KiB
2.9 KiB
MODIFIED Requirements
Requirement: End-to-end MVP acceptance case is documented
The project SHALL include an end-to-end acceptance case that demonstrates Maven start-up, chat diagnosis, exact trace query, orchestration trace inspection, feedback submission, log inspection, and exact database verification using the same sessionId + runId.
Scenario: Reviewer follows the acceptance case
- WHEN a reviewer follows the documented MVP demo acceptance steps
- THEN they can run the application, submit a diagnosis question, query the exact trace endpoint, inspect
run.orchestrationTrace, and submit feedback for the same run - AND they can correlate that run with new logs and exact DiagnosisRun/AgentStep/ToolInvocation database records
Requirement: MVP demo SHALL be reproducible for interviews
The MVP demo SHALL provide a repeatable way to show a diagnosis answer, exact run trace, StateGraph orchestration summary, verifier evaluation, and feedback.
Scenario: interview demo check script records an evidence bundle
- WHEN the user runs the interview demo check script against a running
mvp-demoservice - THEN the script SHALL submit a fixed Chat diagnosis request
- AND it SHALL fetch the trace for the same
sessionId + runId - AND it SHALL fail if
run.orchestrationTraceor its version/final node/termination reason is missing - AND it SHALL submit useful feedback for that run
- AND it SHALL write chat, trace, feedback, and summary outputs under the configured output directory
- AND summary SHALL include final node, termination reason, degraded, transition count, and evidence retry count
Scenario: interview demo check fails with actionable readiness output
- WHEN the target service is not reachable
- THEN the script SHALL fail before issuing diagnosis requests
- AND the failure message SHALL name the base URL and the expected startup profile
Scenario: interview documentation explains audit fields
- WHEN an interviewer asks how Prompt, Gatekeeper, or Graph routing changes are audited
- THEN the demo documentation SHALL point to
prompt_audit.version,gatekeeper_result.rule_set_version, andrun.orchestrationTrace - AND it SHALL explain that deterministic eval fixtures and the Graph test suite are the regression source of truth
Requirement: MVP demo SHALL provide a trace inspection checklist
The MVP demo SHALL document which trace fields to inspect for StateGraph routing, evidence, verifier behavior, and run-level auditability.
Scenario: Checklist maps fields to interview claims
- WHEN a developer reviews an exact trace response
- THEN the checklist SHALL map
run.orchestrationTraceversion/transitions/final node/termination reason/degraded/evidence retry count to Graph routing claims - AND it SHALL map AgentStep, ToolInvocation, self-evaluation, Prompt audit, Gatekeeper audit, answer, and feedback paths to their separate responsibilities