Files

T

zhuyongxin 7ae9707a3b feat(harness,rag): dual LLM audit fields, run conclusion, and hybrid quality

Persist provider reasoning and assistant text separately on agent_reasoning_audit
(DeepSeekAssistantMessage path), extract diagnosis_run.conclusion, enrich RAG
tool audit (step_id/query/qualityScore), gate empty mysql tools, drop devtools,
and align MVP docs after live E2E verification.

2026-07-28 19:43:13 +08:00

1.6 KiB

Raw Blame History

RAG Retrieval Baseline Diff

Generated at: 2026-07-28T06:48:56.439554+00:00

Summary

Metric	Value
Baseline cases	7
Current cases	7
Baseline pass rate	1.0
Current pass rate	0.8571
Baseline recall@K	1.0
Current recall@K	0.8571
Regressions	6
Improvements	0
Changed	3

Items

Type	Scope	Case	Metric	Baseline	Current	Delta	Message
REGRESSION	aggregate		passRate	1.0	0.8571	-0.14290000000000003	aggregate passRate changed
REGRESSION	aggregate		recallAtK	1.0	0.8571	-0.14290000000000003	aggregate recallAtK changed
REGRESSION	aggregate		strongHitRate	1.0	0.8571	-0.14290000000000003	aggregate strongHitRate changed
REGRESSION	case	chat-l0-filter-fallback	passed	True	False	-1.0	chat-l0-filter-fallback passed changed
REGRESSION	case	chat-l0-filter-fallback	hitLevel	strong	weak	-2.0	chat-l0-filter-fallback hitLevel changed
REGRESSION	case	chat-l0-filter-fallback	firstExpectedRank	1	-		chat-l0-filter-fallback firstExpectedRank changed
CHANGED	case	chat-l0-filter-fallback	selectedAttempt	UNFILTERED_VECTOR_RETRY	FILTERED_VECTOR		chat-l0-filter-fallback selectedAttempt changed
CHANGED	case	chat-l0-filter-fallback	fallbackReason	filtered_vector_low_quality	-		chat-l0-filter-fallback fallbackReason changed
CHANGED	case	chat-l0-filter-fallback	rerankTopSource	rag-l0-filter-fallback	rag-l0-filter-decoy		chat-l0-filter-fallback rerankTopSource changed

1.6 KiB Raw Blame History

RAG Retrieval Baseline Diff

Summary

Items

1.6 KiB

Raw Blame History