23 lines
1.2 KiB
Markdown
23 lines
1.2 KiB
Markdown
# Diagnosis Eval Baseline Diff
|
|
|
|
- Baseline pass rate: 100.00%
|
|
- Current pass rate: 80.00%
|
|
- Baseline passed cases: 5/5
|
|
- Current passed cases: 4/5
|
|
- Regressions: 6
|
|
- Improvements: 0
|
|
- Other changes: 2
|
|
|
|
## Diff Items
|
|
|
|
| Type | Scope | Case | Metric | Baseline | Current | Delta | Message |
|
|
| --- | --- | --- | --- | --- | --- | ---: | --- |
|
|
| REGRESSION | aggregate | - | passRate | 1.0 | 0.8 | -0.200 | passRate changed |
|
|
| REGRESSION | aggregate | - | averageToolCallCount | 2.0 | 3.0 | 1.000 | averageToolCallCount changed |
|
|
| CHANGED | aggregate | - | verdictDistribution.LOW_CONFID | 3 | 2 | -1.000 | verdict count changed for LOW_CONFID |
|
|
| CHANGED | aggregate | - | verdictDistribution.REJECT | 0 | 1 | 1.000 | verdict count changed for REJECT |
|
|
| REGRESSION | case | redis-timeout | passed | true | false | - | redis-timeout pass state changed |
|
|
| REGRESSION | case | redis-timeout | verdict | LOW_CONFID | REJECT | -1.000 | redis-timeout verdict changed |
|
|
| REGRESSION | case | redis-timeout | matchedKeywordCount | 2 | 1 | -1.000 | redis-timeout matchedKeywordCount changed |
|
|
| REGRESSION | case | redis-timeout | evidenceCoverage.query_logs | true | false | - | redis-timeout evidence coverage changed for query_logs |
|