feat: archive mvp demo trace acceptance
This commit is contained in:
zhuyongxin
@@ -0,0 +1,65 @@
|
||||
# MVP Demo Trace Acceptance
|
||||
|
||||
## Result
|
||||
|
||||
Accepted for implementation scope.
|
||||
|
||||
## Verification
|
||||
|
||||
### Static Verification
|
||||
|
||||
- Command: `mvn -q -DskipTests compile`
|
||||
- Result: passed
|
||||
- Notes: New trace controller, service, DTO, profile, verifier fallback, and test sources compile with the project.
|
||||
|
||||
### Script Verification
|
||||
|
||||
- Command: `mvn -q "-Dtest=DiagnosisTraceServiceTest,ChatServiceSupervisorAgentTest" test`
|
||||
- Result: passed
|
||||
- Notes: Covers successful trace aggregation, missing-session 404 path via `SessionNotFoundException`, low-confidence no-retry behavior, method-tool injection, and verifier fallback when Supervisor skips `chat_verifier`.
|
||||
|
||||
### OpenSpec Verification
|
||||
|
||||
- Command: `openspec validate mvp-demo-trace-acceptance --strict`
|
||||
- Result: passed
|
||||
|
||||
### GitNexus Verification
|
||||
|
||||
- Result: skipped by user decision
|
||||
- Notes: User requested subsequent project flow to bypass GitNexus.
|
||||
|
||||
### Manual / Runtime Verification
|
||||
|
||||
- Steps: Follow `mvp/demo/README.md` with `--spring.profiles.active=mvp-demo`.
|
||||
- Result: passed
|
||||
- Notes:
|
||||
- Session `mvp-demo-payment-timeout-20260703-rerun2` completed as `SUCCESS`.
|
||||
- Chat request returned `code=200`, `success=true`, and the same `sessionId`.
|
||||
- Chat duration was `96316 ms`; persisted session duration was `95028 ms`.
|
||||
- Trace API returned `code=200`, `returnedSteps=13`, `returnedTools=12`, `hasVerifier=true`, and `verifierVerdict=LOW_CONFID`.
|
||||
- Trace agents included `planner,executor,verifier`.
|
||||
- Trace tools included `lookup_knowledge,query_logs,query_metrics`.
|
||||
- Feedback submission returned success, and a follow-up trace query showed `feedback=useful`.
|
||||
- MySQL verification confirmed `agent_step` count `13` with agents `executor,planner,verifier`.
|
||||
- MySQL verification confirmed `tool_invocation` count `12` with tools `lookup_knowledge,query_logs,query_metrics`.
|
||||
|
||||
## Completed Scope
|
||||
|
||||
- Added `GET /api/diagnosis/{sessionId}/trace`.
|
||||
- Added read-only trace aggregation from persisted diagnosis tables.
|
||||
- Added `mvp-demo` profile overlay.
|
||||
- Added payment-timeout demo acceptance documentation.
|
||||
- Added MVP note for interview storytelling.
|
||||
- Added verifier fallback so runtime trace remains complete when Supervisor returns without `verifier_output`.
|
||||
|
||||
## Known Limits
|
||||
|
||||
- `mvp-demo` is not a fully offline mock runtime.
|
||||
- Runtime still depends on available MySQL, Redis, Milvus/Zilliz, model, and embedding configuration.
|
||||
- Sensitive configuration cleanup remains intentionally deferred.
|
||||
- Supervisor can still make inefficient routing choices inside a single round; `ChatService` now invokes `chat_verifier` as a fallback when Supervisor returns without `verifier_output`, so trace completeness is preserved for the MVP demo.
|
||||
|
||||
## Handoff
|
||||
|
||||
- Runtime demo passed with current infrastructure.
|
||||
- OpenSpec archive confirmation: requested by user after successful rerun.
|
||||
@@ -0,0 +1,35 @@
|
||||
# MVP Demo Trace Acceptance Brief
|
||||
|
||||
## Background
|
||||
|
||||
- User goal: make the MVP runnable, observable, and explainable for an Agent Engineer interview.
|
||||
- Current problem: the system can execute diagnosis, but reviewers need a simple way to replay one session from final answer back to agent steps and tool evidence.
|
||||
- Associated OpenSpec: `openspec/changes/mvp-demo-trace-acceptance/`
|
||||
- Devflow scale: standard-light.
|
||||
|
||||
## Scope
|
||||
|
||||
- In scope:
|
||||
- `mvp-demo` Spring profile overlay.
|
||||
- `GET /api/diagnosis/{sessionId}/trace` read-only API.
|
||||
- Trace aggregation DTO/service/controller.
|
||||
- Focused service tests.
|
||||
- Demo and acceptance documentation.
|
||||
- Out of scope:
|
||||
- Sensitive configuration cleanup.
|
||||
- Full offline LLM/vector/database mock runtime.
|
||||
- Database schema migration.
|
||||
- Changes to chat execution, verifier routing, upload, or feedback behavior.
|
||||
- Impact area:
|
||||
- `src/main/java/com/superbiz/agent/controller`
|
||||
- `src/main/java/com/superbiz/agent/service`
|
||||
- `src/main/java/com/superbiz/agent/dto`
|
||||
- `src/main/resources/application-mvp-demo.yml`
|
||||
- `mvp/demo`
|
||||
- `mvp/notes`
|
||||
|
||||
## OpenSpec Alignment
|
||||
|
||||
- proposal coverage: covered
|
||||
- specs coverage: covered
|
||||
- tasks coverage: covered
|
||||
@@ -0,0 +1,87 @@
|
||||
# MVP Demo Trace Acceptance Decisions
|
||||
|
||||
## Clarify
|
||||
|
||||
- Entry summary: continue the MVP toward a runnable and explainable demo by adding an `mvp-demo` profile, an end-to-end acceptance case, and a trace query API.
|
||||
- Slug: `mvp-demo-trace-acceptance`
|
||||
- Devflow scale: standard-light. The change adds a public read-only API and documentation, but does not alter core chat execution or persistence schemas.
|
||||
|
||||
## Context
|
||||
|
||||
- `devflow/index.md` was checked. Relevant history includes `session-storage`, `confidence-feedback`, `executor-action-memory-relevance`, and `chat-verifier-agent`.
|
||||
- `mvp/notes/agent-engineering-decisions.md` already recommends the next phase as "可复现 MVP Demo", including `mvp-demo` profile, fixed diagnosis case, one-click request, and `GET /api/diagnosis/{sessionId}/trace`.
|
||||
- `mvp/issues/ISS-003-mvp-design-implementation-review.md` identifies test stability, session traceability, verifier evidence chain, upload path, and SupervisorAgent consistency as recent MVP concerns. Security cleanup is intentionally deferred by user decision.
|
||||
|
||||
## Question Pool
|
||||
|
||||
| # | Dimension | Question | Mode | Status |
|
||||
|---|---|---|---|---|
|
||||
| Q1 | Terminology | Should "trace" mean persisted diagnosis execution evidence instead of transient frontend chat history? | evidence-driven | Resolved |
|
||||
| Q2 | Boundary | Should this change modify chat execution or only expose existing persisted evidence? | evidence-driven | Resolved |
|
||||
| Q3 | Acceptance | What proves the MVP flow is end-to-end enough for demo/interview use? | evidence-driven | Resolved |
|
||||
| Q4 | Interface | What is the API impact level for `GET /api/diagnosis/{sessionId}/trace`? | evidence-driven | Resolved |
|
||||
|
||||
## Evidence-driven
|
||||
|
||||
| Conclusion | Evidence Source | Reported To User |
|
||||
|---|---|---|
|
||||
| Trace should aggregate persisted diagnosis evidence, not Redis-only chat history. | `DiagnosisSession`, `AgentStep`, `ToolInvocation` entities and repositories | Reported in progress update |
|
||||
| Core chat execution does not need to change for this slice. | Existing unified chat path and SupervisorAgent commits; requested scope is demo/profile/trace/acceptance | Reported in progress update |
|
||||
| End-to-end acceptance should cover start -> chat -> trace -> feedback. | `ChatController`, `FeedbackController`, traceable session id decision in MVP notes | Reported in progress update |
|
||||
| Trace API is additive L3 because it is a new HTTP API for frontend/demo consumers. | sm-flow interface impact rules | Recorded in OpenSpec design |
|
||||
|
||||
## User-interview
|
||||
|
||||
| Question | User Words | Confirmation | OpenSpec Writeback |
|
||||
|---|---|---|---|
|
||||
| Should security/sensitive config cleanup be included? | "安全问题先不考虑"; "敏感配置先不做" | Confirmed | Non-goal |
|
||||
| Should this be implemented under sm-flow? | "按照 sm-flow 的流程来实现吧" | Confirmed | This change follows sm-flow artifacts |
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- Decision: Add a new trace API instead of embedding trace details in `/api/chat`.
|
||||
- Reason: Chat execution and observability should stay decoupled.
|
||||
- Impact: Demo can query trace after any successful chat request using the same session id.
|
||||
- Risk accepted: Response shape is new and should be treated as demo-facing contract.
|
||||
|
||||
- Decision: Keep `mvp-demo` profile as configuration overlay, not a fully mocked standalone runtime.
|
||||
- Reason: The current MVP still depends on real DB/Redis/Milvus/LLM for full chat execution; this change avoids inventing a fake runtime that hides integration behavior.
|
||||
- Impact: Demo profile improves repeatability for logs/metrics, while docs remain explicit about required external services.
|
||||
- Risk accepted: End-to-end acceptance may still require valid infrastructure and keys.
|
||||
|
||||
## Cross-Artifact Alignment
|
||||
|
||||
| Upstream -> Downstream | Check | Status |
|
||||
|---|---|---|
|
||||
| brief/prd -> proposal | Goal, scope, non-goals, and acceptance expectation are in proposal | Aligned |
|
||||
| proposal -> design | Scope, constraints, and API impact are in design | Aligned |
|
||||
| design -> specs/tasks | Trace DTO, controller/service, demo profile, and docs are represented | Aligned |
|
||||
| specs -> tasks | Observable behavior is covered by executable tasks | Aligned |
|
||||
|
||||
## Architecture Audit
|
||||
|
||||
- Data path: HTTP trace request -> controller -> trace service -> repositories -> aggregate DTO -> `Result.success`.
|
||||
- The service is read-only and does not mutate diagnosis, step, tool, or feedback state.
|
||||
- No schema change is needed because all required fields already exist in `diagnosis_session`, `agent_step`, and `tool_invocation`.
|
||||
- Main risk is response size for large sessions; MVP mitigates by returning previews already persisted by tools rather than raw external logs.
|
||||
- The additive API is acceptable for MVP because old callers remain unaffected.
|
||||
|
||||
## Pre-apply Research
|
||||
|
||||
- Reference implementations read:
|
||||
- `ChatController` for `/api` controller conventions.
|
||||
- `FeedbackController` for simple API controller shape.
|
||||
- `GlobalExceptionHandler` and `SessionNotFoundException` for 404 handling.
|
||||
- `DiagnosisSessionRepository`, `AgentStepRepository`, `ToolInvocationRepository` for available queries.
|
||||
- `DiagnosisSession`, `AgentStep`, `ToolInvocation` for fields.
|
||||
- Impact analysis:
|
||||
- `DiagnosisSessionRepository`: LOW, direct imports in service/controller paths.
|
||||
- `AgentStepRepository`: HIGH because it participates in chat/AiOps flows. This change only consumes existing query methods and does not modify the repository.
|
||||
- `ToolInvocationRepository`: LOW.
|
||||
|
||||
## Commit Gate
|
||||
|
||||
- OpenSpec proposal/design/specs/tasks exist.
|
||||
- API impact: L3 additive collaboration API, documented in design and spec.
|
||||
- User-confirmed non-goal: sensitive configuration cleanup remains out of scope.
|
||||
- No unresolved user-interview questions remain for this slice.
|
||||
@@ -0,0 +1,25 @@
|
||||
# MVP Demo Trace Acceptance Evidence
|
||||
|
||||
## Evidence
|
||||
|
||||
| Source | Evidence | Conclusion | Reported |
|
||||
|---|---|---|---|
|
||||
| `DiagnosisSessionRepository` | Existing `findBySessionId(String)` query | Trace can locate the session without new repository methods | Yes |
|
||||
| `AgentStepRepository` | Existing `findBySessionIdOrderByStepIndex(String)` query | Agent steps can be returned in execution order | Yes |
|
||||
| `ToolInvocationRepository` | Existing `findBySessionIdOrderByIdAsc(String)` query | Tool evidence can be returned in persisted order | Yes |
|
||||
| `GlobalExceptionHandler` | Handles `SessionNotFoundException` as HTTP 404 with `Result.error(404, ...)` | Missing trace can reuse existing error contract | Yes |
|
||||
| `mvn -q "-Dtest=DiagnosisTraceServiceTest" test` | Command passed | Trace aggregation behavior is covered offline | Yes |
|
||||
| `mvn -q -DskipTests compile` | Command passed | New code compiles with the full project | Yes |
|
||||
| `gitnexus detect-changes --repo SuperBizAgent-java` | Command completed with `No changes detected` and line-ending warnings | Required GitNexus check ran; output likely does not capture newly added files | Yes |
|
||||
|
||||
## Evidence-driven Conclusions
|
||||
|
||||
- Conclusion: No database migration is required.
|
||||
- Evidence: All trace fields are available from existing `diagnosis_session`, `agent_step`, and `tool_invocation` entities.
|
||||
- Risk: Response shape becomes a new API contract.
|
||||
- User confirmation: Not required; additive L3 API recorded in OpenSpec.
|
||||
|
||||
- Conclusion: Trace aggregation can be tested without external infrastructure.
|
||||
- Evidence: `DiagnosisTraceServiceTest` uses mocked repositories and an `ObjectMapper`.
|
||||
- Risk: Runtime integration still depends on configured infrastructure.
|
||||
- User confirmation: Not required; limitation recorded in acceptance docs.
|
||||
Reference in New Issue
Block a user