zhuyongxin
7ae9707a3b
Persist provider reasoning and assistant text separately on agent_reasoning_audit (DeepSeekAssistantMessage path), extract diagnosis_run.conclusion, enrich RAG tool audit (step_id/query/qualityScore), gate empty mysql tools, drop devtools, and align MVP docs after live E2E verification.
1.8 KiB
1.8 KiB
Tasks: rag-eval-hybrid-baseline
1. Generator wiring
- 1.1 Update
scripts/generate_rag_lookup_snapshots.ps1: replaceVectorStoreMode/vector-store.modewithSearchModedefaulthybridand-Dretrieval.search.mode=... - 1.2 Keep
-Dretrieval.kb-scope(defaultrag-eval); document-SearchMode denseoverride - 1.3 Ensure snapshot test picks up
retrieval.search.mode(system property and/or@SpringBootTestproperties)
2. Fixture meta
- 2.1
RagLookupSnapshotGeneratorTestwritessearchModeandkbScope(when set) on each fixture - 2.2 Confirm offline
eval_rag_retrieval.pystill loads fixtures (ignore extra meta)
3. Docs
- 3.1 Rewrite
eval/rag-retrieval/README.mdhybrid-era loop; remove spring vector-store as default generation path - 3.2 Note offline vs live responsibilities; point to qualityScore/hybrid main path briefly
4. Refresh attempt (best-effort)
- 4.1 Attempt
prepare_rag_eval_seed+ hybrid snapshot generate + offline eval when environment allows - 4.2 On success: update fixtures and
reports/baseline.*if needed after diff review - 4.3 On failure: record exact commands, error summary, and “unverified refresh” in change decisions/acceptance notes — do not block wiring delivery
5. Verify
- 5.1 Static: grep shows no required
vector-store.modein snapshot generator path - 5.2 Offline:
python scripts/eval_rag_retrieval.pyruns on committed fixtures (pass or documented baseline drift)
6. Apply-discovered fix (hybrid quality gate)
- 6.1 Hybrid attaches optional
denseDistancewithout overwriting RRF order/scoreLabel - 6.2
toQualityScore(hybrid)prefers dense L2 for absolute gates; rank fallback if no dense - 6.3 Regenerate fixtures; L0 filter fallback case green again; baseline updated