osiman/SuperBizAgent-java
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(harness,rag): dual LLM audit fields, run conclusion, and hybrid quality

Browse Source 
Persist provider reasoning and assistant text separately on agent_reasoning_audit
(DeepSeekAssistantMessage path), extract diagnosis_run.conclusion, enrich RAG
tool audit (step_id/query/qualityScore), gate empty mysql tools, drop devtools,
and align MVP docs after live E2E verification.
This commit is contained in:
zhuyongxin
2026-07-28 19:43:13 +08:00
parent 2f40536248
commit 7ae9707a3b
116 changed files with 8364 additions and 1141 deletions
Download Patch File Download Diff File Expand all files Collapse all files
eval/rag-retrieval/reports/baseline-diff.json
+107
View File
@@ -0,0 +1,107 @@
{
"generatedAt": "2026-07-28T06:48:56.439554+00:00",
"baselineReport": "eval/rag-retrieval/cases/golden-cases.json",
"currentReport": "eval/rag-retrieval/cases/golden-cases.json",
"baselineCaseCount": 7,
"currentCaseCount": 7,
"baselinePassRate": 1.0,
"currentPassRate": 0.8571,
"baselineRecallAtK": 1.0,
"currentRecallAtK": 0.8571,
"regressionCount": 6,
"improvementCount": 0,
"changedCount": 3,
"hasRegression": true,
"items": [
{
"type": "REGRESSION",
"scope": "aggregate",
"caseId": null,
"metric": "passRate",
"baselineValue": "1.0",
"currentValue": "0.8571",
"delta": -0.14290000000000003,
"message": "aggregate passRate changed"
},
{
"type": "REGRESSION",
"scope": "aggregate",
"caseId": null,
"metric": "recallAtK",
"baselineValue": "1.0",
"currentValue": "0.8571",
"delta": -0.14290000000000003,
"message": "aggregate recallAtK changed"
},
{
"type": "REGRESSION",
"scope": "aggregate",
"caseId": null,
"metric": "strongHitRate",
"baselineValue": "1.0",
"currentValue": "0.8571",
"delta": -0.14290000000000003,
"message": "aggregate strongHitRate changed"
},
{
"type": "REGRESSION",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "passed",
"baselineValue": "True",
"currentValue": "False",
"delta": -1.0,
"message": "chat-l0-filter-fallback passed changed"
},
{
"type": "REGRESSION",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "hitLevel",
"baselineValue": "strong",
"currentValue": "weak",
"delta": -2.0,
"message": "chat-l0-filter-fallback hitLevel changed"
},
{
"type": "REGRESSION",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "firstExpectedRank",
"baselineValue": "1",
"currentValue": "-",
"delta": null,
"message": "chat-l0-filter-fallback firstExpectedRank changed"
},
{
"type": "CHANGED",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "selectedAttempt",
"baselineValue": "UNFILTERED_VECTOR_RETRY",
"currentValue": "FILTERED_VECTOR",
"delta": null,
"message": "chat-l0-filter-fallback selectedAttempt changed"
},
{
"type": "CHANGED",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "fallbackReason",
"baselineValue": "filtered_vector_low_quality",
"currentValue": "-",
"delta": null,
"message": "chat-l0-filter-fallback fallbackReason changed"
},
{
"type": "CHANGED",
"scope": "case",
"caseId": "chat-l0-filter-fallback",
"metric": "rerankTopSource",
"baselineValue": "rag-l0-filter-fallback",
"currentValue": "rag-l0-filter-decoy",
"delta": null,
"message": "chat-l0-filter-fallback rerankTopSource changed"
}
]
}
eval/rag-retrieval/reports/baseline-diff.md
+31
View File
@@ -0,0 +1,31 @@
# RAG Retrieval Baseline Diff
Generated at: `2026-07-28T06:48:56.439554+00:00`
## Summary
| Metric | Value |
|---|---:|
| Baseline cases | 7 |
| Current cases | 7 |
| Baseline pass rate | 1.0 |
| Current pass rate | 0.8571 |
| Baseline recall@K | 1.0 |
| Current recall@K | 0.8571 |
| Regressions | 6 |
| Improvements | 0 |
| Changed | 3 |
## Items
| Type | Scope | Case | Metric | Baseline | Current | Delta | Message |
|---|---|---|---|---|---|---:|---|
| REGRESSION | aggregate | | passRate | 1.0 | 0.8571 | -0.14290000000000003 | aggregate passRate changed |
| REGRESSION | aggregate | | recallAtK | 1.0 | 0.8571 | -0.14290000000000003 | aggregate recallAtK changed |
| REGRESSION | aggregate | | strongHitRate | 1.0 | 0.8571 | -0.14290000000000003 | aggregate strongHitRate changed |
| REGRESSION | case | chat-l0-filter-fallback | passed | True | False | -1.0 | chat-l0-filter-fallback passed changed |
| REGRESSION | case | chat-l0-filter-fallback | hitLevel | strong | weak | -2.0 | chat-l0-filter-fallback hitLevel changed |
| REGRESSION | case | chat-l0-filter-fallback | firstExpectedRank | 1 | - | | chat-l0-filter-fallback firstExpectedRank changed |
| CHANGED | case | chat-l0-filter-fallback | selectedAttempt | UNFILTERED_VECTOR_RETRY | FILTERED_VECTOR | | chat-l0-filter-fallback selectedAttempt changed |
| CHANGED | case | chat-l0-filter-fallback | fallbackReason | filtered_vector_low_quality | - | | chat-l0-filter-fallback fallbackReason changed |
| CHANGED | case | chat-l0-filter-fallback | rerankTopSource | rag-l0-filter-fallback | rag-l0-filter-decoy | | chat-l0-filter-fallback rerankTopSource changed |
eval/rag-retrieval/reports/baseline.json
+38 -14
View File
@@ -1,5 +1,5 @@
{
"generatedAt": "2026-07-06T13:37:59.726351+00:00",
"generatedAt": "2026-07-28T06:55:09.379164+00:00",
"caseFile": "eval/rag-retrieval/cases/golden-cases.json",
"fixtureDir": "eval/rag-retrieval/fixtures",
"aggregate": {
@@ -28,7 +28,7 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:mysql-connection-pool",
"2:incident-diagnosis-flow"
"2:mysql-connection-pool"
],
"matchedKeywords": [
"connection pool",
@@ -41,7 +41,7 @@
"evidenceStatus": "supported",
"includedSources": [
"mysql-connection-pool",
"incident-diagnosis-flow"
"mysql-connection-pool"
],
"omittedSources": [],
"rerankTopSource": "mysql-connection-pool",
@@ -57,7 +57,7 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:incident-diagnosis-flow",
"2:rag-chunk-context-reconstruction"
"2:incident-diagnosis-flow"
],
"matchedKeywords": [
"collect evidence",
@@ -70,7 +70,7 @@
"evidenceStatus": "supported",
"includedSources": [
"incident-diagnosis-flow",
"rag-chunk-context-reconstruction"
"incident-diagnosis-flow"
],
"omittedSources": [],
"rerankTopSource": "incident-diagnosis-flow",
@@ -86,7 +86,9 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:payment-service-latency",
"2:mysql-connection-pool"
"2:aiops-alert-scope-control",
"3:payment-service-latency",
"4:aiops-alert-scope-control"
],
"matchedKeywords": [
"p95 latency",
@@ -99,7 +101,9 @@
"evidenceStatus": "supported",
"includedSources": [
"payment-service-latency",
"mysql-connection-pool"
"aiops-alert-scope-control",
"payment-service-latency",
"aiops-alert-scope-control"
],
"omittedSources": [],
"rerankTopSource": "payment-service-latency",
@@ -114,7 +118,10 @@
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:aiops-alert-scope-control"
"1:aiops-alert-scope-control",
"2:payment-service-latency",
"3:payment-service-latency",
"4:aiops-alert-scope-control"
],
"matchedKeywords": [
"payload",
@@ -126,6 +133,9 @@
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"aiops-alert-scope-control",
"payment-service-latency",
"payment-service-latency",
"aiops-alert-scope-control"
],
"omittedSources": [],
@@ -142,7 +152,9 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:rag-chunk-context-reconstruction",
"2:rag-breadcrumb-embedding-gap"
"2:rag-l0-domain-entity-hint",
"3:rag-chunk-context-reconstruction",
"4:rag-l0-domain-entity-hint"
],
"matchedKeywords": [
"neighbor chunk",
@@ -155,7 +167,9 @@
"evidenceStatus": "supported",
"includedSources": [
"rag-chunk-context-reconstruction",
"rag-breadcrumb-embedding-gap"
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction",
"rag-l0-domain-entity-hint"
],
"omittedSources": [],
"rerankTopSource": "rag-chunk-context-reconstruction",
@@ -171,7 +185,9 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:rag-l0-domain-entity-hint",
"2:rag-l0-l1-fusion-ranking"
"2:rag-chunk-context-reconstruction",
"3:rag-l0-domain-entity-hint",
"4:rag-chunk-context-reconstruction"
],
"matchedKeywords": [
"domain detector",
@@ -184,7 +200,9 @@
"evidenceStatus": "supported",
"includedSources": [
"rag-l0-domain-entity-hint",
"rag-l0-l1-fusion-ranking"
"rag-chunk-context-reconstruction",
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction"
],
"omittedSources": [],
"rerankTopSource": "rag-l0-domain-entity-hint",
@@ -200,7 +218,10 @@
"firstExpectedRank": 1,
"topCandidates": [
"1:rag-l0-filter-fallback",
"2:rag-l0-domain-entity-hint"
"2:rag-l0-filter-decoy",
"3:rag-l0-domain-entity-hint",
"4:rag-l0-domain-entity-hint",
"5:rag-chunk-context-reconstruction"
],
"matchedKeywords": [
"skip the l0 filter",
@@ -213,7 +234,10 @@
"evidenceStatus": "supported",
"includedSources": [
"rag-l0-filter-fallback",
"rag-l0-domain-entity-hint"
"rag-l0-filter-decoy",
"rag-l0-domain-entity-hint",
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction"
],
"omittedSources": [],
"rerankTopSource": "rag-l0-filter-fallback",
eval/rag-retrieval/reports/baseline.md
+8 -8
View File
@@ -1,6 +1,6 @@
# RAG Retrieval Baseline
Generated at: `2026-07-06T13:37:59.726351+00:00`
Generated at: `2026-07-28T06:55:09.379164+00:00`
## Aggregate
@@ -24,10 +24,10 @@ Generated at: `2026-07-06T13:37:59.726351+00:00`
| Case | Scenario | Pass | Hit | Attempt | Fallback | Evidence | First Expected Rank | Top Candidates | Failed Checks |
|---|---|---|---|---|---|---|---:|---|---|
| chat-mysql-connection-pool | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:mysql-connection-pool<br>2:incident-diagnosis-flow | |
| chat-diagnosis-flow | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:incident-diagnosis-flow<br>2:rag-chunk-context-reconstruction | |
| aiops-payment-latency-alert | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:payment-service-latency<br>2:mysql-connection-pool | |
| aiops-prometheus-alert-scope | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:aiops-alert-scope-control | |
| chat-rag-chunk-context | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-chunk-context-reconstruction<br>2:rag-breadcrumb-embedding-gap | |
| chat-l0-domain-hint | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-l0-domain-entity-hint<br>2:rag-l0-l1-fusion-ranking | |
| chat-l0-filter-fallback | chat | true | strong | UNFILTERED_VECTOR_RETRY | filtered_vector_low_quality | supported | 1 | 1:rag-l0-filter-fallback<br>2:rag-l0-domain-entity-hint | |
| chat-mysql-connection-pool | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:mysql-connection-pool<br>2:mysql-connection-pool | |
| chat-diagnosis-flow | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:incident-diagnosis-flow<br>2:incident-diagnosis-flow | |
| aiops-payment-latency-alert | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:payment-service-latency<br>2:aiops-alert-scope-control<br>3:payment-service-latency<br>4:aiops-alert-scope-control | |
| aiops-prometheus-alert-scope | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:aiops-alert-scope-control<br>2:payment-service-latency<br>3:payment-service-latency<br>4:aiops-alert-scope-control | |
| chat-rag-chunk-context | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-chunk-context-reconstruction<br>2:rag-l0-domain-entity-hint<br>3:rag-chunk-context-reconstruction<br>4:rag-l0-domain-entity-hint | |
| chat-l0-domain-hint | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-l0-domain-entity-hint<br>2:rag-chunk-context-reconstruction<br>3:rag-l0-domain-entity-hint<br>4:rag-chunk-context-reconstruction | |
| chat-l0-filter-fallback | chat | true | strong | UNFILTERED_VECTOR_RETRY | filtered_vector_low_quality | supported | 1 | 1:rag-l0-filter-fallback<br>2:rag-l0-filter-decoy<br>3:rag-l0-domain-entity-hint<br>4:rag-l0-domain-entity-hint<br>5:rag-chunk-context-reconstruction | |
eval/rag-retrieval/reports/current.json
+245
View File
@@ -0,0 +1,245 @@
{
"generatedAt": "2026-07-28T06:48:56.421877+00:00",
"caseFile": "eval/rag-retrieval/cases/golden-cases.json",
"fixtureDir": "eval/rag-retrieval/fixtures",
"aggregate": {
"caseCount": 7,
"topK": 5,
"passedCount": 6,
"failedCount": 1,
"passRate": 0.8571,
"lookupResultCaseCount": 7,
"strongHitCount": 6,
"mediumHitCount": 0,
"weakHitCount": 1,
"missCount": 0,
"recallAtK": 0.8571,
"strongHitRate": 0.8571,
"averageFirstHitRank": 1.0
},
"results": [
{
"caseId": "chat-mysql-connection-pool",
"scenario": "chat",
"query": "MySQL connection pool is exhausted. How should I diagnose it?",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:mysql-connection-pool",
"2:mysql-connection-pool"
],
"matchedKeywords": [
"connection pool",
"max_connections",
"hikaricp"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"mysql-connection-pool",
"mysql-connection-pool"
],
"omittedSources": [],
"rerankTopSource": "mysql-connection-pool",
"failedChecks": []
},
{
"caseId": "chat-diagnosis-flow",
"scenario": "chat",
"query": "What is the standard troubleshooting flow for an application incident?",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:incident-diagnosis-flow",
"2:incident-diagnosis-flow"
],
"matchedKeywords": [
"collect evidence",
"verify",
"remediation"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"incident-diagnosis-flow",
"incident-diagnosis-flow"
],
"omittedSources": [],
"rerankTopSource": "incident-diagnosis-flow",
"failedChecks": []
},
{
"caseId": "aiops-payment-latency-alert",
"scenario": "aiops",
"query": "Alert HighLatency on payment-service with p95 latency above threshold",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:payment-service-latency",
"2:aiops-alert-scope-control",
"3:payment-service-latency",
"4:aiops-alert-scope-control"
],
"matchedKeywords": [
"p95 latency",
"payment-service",
"downstream dependency"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"payment-service-latency",
"aiops-alert-scope-control",
"payment-service-latency",
"aiops-alert-scope-control"
],
"omittedSources": [],
"rerankTopSource": "payment-service-latency",
"failedChecks": []
},
{
"caseId": "aiops-prometheus-alert-scope",
"scenario": "aiops",
"query": "When an AIOps request already includes alert payload, should the agent diagnose unrelated active alerts?",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:aiops-alert-scope-control",
"2:payment-service-latency",
"3:payment-service-latency",
"4:aiops-alert-scope-control"
],
"matchedKeywords": [
"payload",
"unrelated active alerts",
"scope"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"aiops-alert-scope-control",
"payment-service-latency",
"payment-service-latency",
"aiops-alert-scope-control"
],
"omittedSources": [],
"rerankTopSource": "aiops-alert-scope-control",
"failedChecks": []
},
{
"caseId": "chat-rag-chunk-context",
"scenario": "chat",
"query": "If a long section is split into multiple chunks, how do we keep retrieval context?",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:rag-chunk-context-reconstruction",
"2:rag-l0-domain-entity-hint",
"3:rag-chunk-context-reconstruction",
"4:rag-l0-domain-entity-hint"
],
"matchedKeywords": [
"neighbor chunk",
"same section",
"breadcrumb"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"rag-chunk-context-reconstruction",
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction",
"rag-l0-domain-entity-hint"
],
"omittedSources": [],
"rerankTopSource": "rag-chunk-context-reconstruction",
"failedChecks": []
},
{
"caseId": "chat-l0-domain-hint",
"scenario": "chat",
"query": "Should L0 keyword matching decide the final retrieval result?",
"dataShape": "lookupResult",
"hitLevel": "strong",
"passed": true,
"firstExpectedRank": 1,
"topCandidates": [
"1:rag-l0-domain-entity-hint",
"2:rag-chunk-context-reconstruction",
"3:rag-l0-domain-entity-hint",
"4:rag-chunk-context-reconstruction"
],
"matchedKeywords": [
"domain detector",
"entity extractor",
"metadata filter"
],
"breadcrumbMatched": true,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction",
"rag-l0-domain-entity-hint",
"rag-chunk-context-reconstruction"
],
"omittedSources": [],
"rerankTopSource": "rag-l0-domain-entity-hint",
"failedChecks": []
},
{
"caseId": "chat-l0-filter-fallback",
"scenario": "chat",
"query": "RAG query was over-filtered by L0 and filtered vector search returned low quality evidence. What should happen?",
"dataShape": "lookupResult",
"hitLevel": "weak",
"passed": false,
"firstExpectedRank": null,
"topCandidates": [
"1:rag-l0-filter-decoy",
"2:rag-l0-filter-decoy"
],
"matchedKeywords": [
"low quality"
],
"breadcrumbMatched": false,
"selectedAttempt": "FILTERED_VECTOR",
"fallbackReason": null,
"evidenceStatus": "supported",
"includedSources": [
"rag-l0-filter-decoy",
"rag-l0-filter-decoy"
],
"omittedSources": [],
"rerankTopSource": "rag-l0-filter-decoy",
"failedChecks": [
"expected document not found",
"selected attempt mismatch: expected UNFILTERED_VECTOR_RETRY, got FILTERED_VECTOR",
"fallback reason mismatch: expected one of [filtered_vector_low_quality, filtered_vector_no_evidence], got <none>",
"rerank top source mismatch: expected rag-l0-filter-fallback, got rag-l0-filter-decoy",
"expected context sources missing: rag-l0-filter-fallback"
]
}
]
}
eval/rag-retrieval/reports/current.md
+33
View File
@@ -0,0 +1,33 @@
# RAG Retrieval Baseline
Generated at: `2026-07-28T06:48:56.421877+00:00`
## Aggregate
| Metric | Value |
|---|---:|
| Cases | 7 |
| Top K | 5 |
| Passed | 6 |
| Failed | 1 |
| Pass rate | 0.8571 |
| LookupResult fixtures | 7 |
| Recall@K | 0.8571 |
| Strong hit rate | 0.8571 |
| Strong hits | 6 |
| Medium hits | 0 |
| Weak hits | 1 |
| Misses | 0 |
| Average first hit rank | 1.0 |
## Cases
| Case | Scenario | Pass | Hit | Attempt | Fallback | Evidence | First Expected Rank | Top Candidates | Failed Checks |
|---|---|---|---|---|---|---|---:|---|---|
| chat-mysql-connection-pool | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:mysql-connection-pool<br>2:mysql-connection-pool | |
| chat-diagnosis-flow | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:incident-diagnosis-flow<br>2:incident-diagnosis-flow | |
| aiops-payment-latency-alert | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:payment-service-latency<br>2:aiops-alert-scope-control<br>3:payment-service-latency<br>4:aiops-alert-scope-control | |
| aiops-prometheus-alert-scope | aiops | true | strong | FILTERED_VECTOR | | supported | 1 | 1:aiops-alert-scope-control<br>2:payment-service-latency<br>3:payment-service-latency<br>4:aiops-alert-scope-control | |
| chat-rag-chunk-context | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-chunk-context-reconstruction<br>2:rag-l0-domain-entity-hint<br>3:rag-chunk-context-reconstruction<br>4:rag-l0-domain-entity-hint | |
| chat-l0-domain-hint | chat | true | strong | FILTERED_VECTOR | | supported | 1 | 1:rag-l0-domain-entity-hint<br>2:rag-chunk-context-reconstruction<br>3:rag-l0-domain-entity-hint<br>4:rag-chunk-context-reconstruction | |
| chat-l0-filter-fallback | chat | false | weak | FILTERED_VECTOR | | supported | | 1:rag-l0-filter-decoy<br>2:rag-l0-filter-decoy | expected document not found<br>selected attempt mismatch: expected UNFILTERED_VECTOR_RETRY, got FILTERED_VECTOR<br>fallback reason mismatch: expected one of [filtered_vector_low_quality, filtered_vector_no_evidence], got <none><br>rerank top source mismatch: expected rag-l0-filter-fallback, got rag-l0-filter-decoy<br>expected context sources missing: rag-l0-filter-fallback |