feat: treat l0 retrieval as domain hint

This commit is contained in:
aruo
2026-07-05 02:18:40 +08:00
parent 9a2a44d1b5
commit 4a94c14feb
12 changed files with 494 additions and 74 deletions
@@ -0,0 +1,2 @@
schema: spec-driven
created: 2026-07-04
@@ -0,0 +1,62 @@
## Context
`LookupKnowledgeTool` currently performs L0 keyword matching first. If L0 returns exactly one document, the tool treats it as high confidence, skips L1 semantic retrieval, and returns the L0-derived primary result. This was useful for the MVP but conflicts with the RAG refactor direction: L0 should constrain and explain retrieval, not decide final evidence by itself.
The refactor plan keeps L0 and metadata as valuable business signals. This change narrows L0 to a domain/entity hint provider while keeping `lookup_knowledge` as the explicit Agent tool entry point and preserving `tool_invocation` observability.
## Goals / Non-Goals
**Goals:**
- Produce structured L0 hints from current keyword/frontmatter matches.
- Include matched keywords, domains, entities, and titles in the trace.
- Run L1 retrieval by default even for unique L0 hits.
- Use a single clear L0 domain as a category filter for L1.
- Preserve existing result shape as much as possible.
**Non-Goals:**
- Do not migrate to Spring AI VectorStore.
- Do not implement BM25, RRF, rerank, or evidence packing.
- Do not change document upload, chunking, or Milvus schema.
- Do not remove L0.
## Decisions
### Decision: Add a structured L0 hint result beside existing exact matches
`KnowledgeIndexService` will expose an `analyzeQuery` style method that returns:
- matched entries
- matched keywords
- domains/categories
- entity terms
The existing `exactMatch` method can remain for compatibility.
Rationale: this avoids rewriting all callers while giving `LookupKnowledgeTool` richer data for tracing and filtering.
### Decision: Treat L0 unique hit as a hint, not a short circuit
`LookupKnowledgeTool` will no longer skip L1 solely because L0 matched one document. L1 will be called using the query and an optional category filter when L0 provides exactly one clear domain.
Rationale: the upcoming Spring AI retriever and evidence post-processing pipeline needs L0 and L1 to cooperate rather than use early return semantics.
### Decision: Keep `PRECISE` only when L0 and L1 both support the result
The relevance assessment should not mark `PRECISE` just because L0 matched once. It may mark `PRECISE` when L0 has one match and L1 returns evidence above the configured high relevance threshold, or when L0 has one match and L1 cannot run but the L0 result is still available.
Rationale: this preserves a graceful fallback while reducing overconfidence when semantic evidence disagrees.
### Decision: Persist L0 hints in retrieval details
`ToolInvocationRecorder.LookupKnowledgeRecord` will include fields for L0 matched keywords, domains, and entities. These will be serialized into `retrieval_details`.
Rationale: evidence trace and later evaluation need to explain why metadata filters or query augmentation happened.
## Risks / Trade-offs
- Increased latency because L1 is called more often -> keep topK small and allow category filter to reduce search scope.
- L0 domain filter may be too narrow -> only apply it when there is exactly one nonblank domain; otherwise search without filter.
- Existing tests may assume `L0` retrieval layer for unique hits -> update expectations to `L0+L1` when L1 participates.
- If L1 fails, the tool should still return L0 evidence rather than fail the entire knowledge lookup.
@@ -0,0 +1,30 @@
## Why
The current `lookup_knowledge` implementation treats a unique L0 keyword hit as high confidence and skips L1 semantic retrieval. That makes L0 too authoritative for the RAG refactor target: L0 should provide domain/entity hints, metadata-filter intent, and explainability while final evidence still comes from the retrieval pipeline.
## What Changes
- Change L0 from final retrieval decision maker to domain/entity hint provider.
- Add structured L0 hint output that includes matched keywords, domains, entities, and matched titles.
- Make `lookup_knowledge` run L1 semantic retrieval by default even when L0 has a unique hit.
- Use L0 domain hints to pass category metadata filters into L1 when a single clear domain is detected.
- Persist L0 hint details in `tool_invocation.retrieval_details`.
- Keep `lookup_knowledge` as the explicit Agent tool entry point.
- No Spring AI VectorStore migration in this change.
## Capabilities
### New Capabilities
- `rag-knowledge-retrieval`: Defines runtime behavior for the explicit RAG knowledge retrieval tool, including L0 hinting and L1 retrieval cooperation.
### Modified Capabilities
- None.
## Impact
- Affects `KnowledgeIndexService`, `LookupKnowledgeTool`, and `ToolInvocationRecorder`.
- May affect retrieval latency because L1 is no longer skipped for unique L0 hits.
- Improves traceability by recording L0 matched keywords/entities/domains in retrieval details.
- Does not change document upload, chunking, Milvus schema, or Agent flow.
@@ -0,0 +1,47 @@
## ADDED Requirements
### Requirement: Knowledge retrieval SHALL keep L0 as a hint provider
The `lookup_knowledge` retrieval flow SHALL retain L0 keyword/frontmatter matching but use it as domain, entity, and explainability hint data rather than as the sole final retrieval decision.
#### Scenario: L0 produces traceable hint data
- **WHEN** L0 matches one or more indexed knowledge entries
- **THEN** the retrieval flow SHALL expose matched titles, matched keywords, domains or categories, and entity terms as structured hint data
#### Scenario: L0 does not bypass semantic retrieval by default
- **WHEN** L0 returns exactly one match
- **THEN** the retrieval flow SHALL still attempt semantic L1 retrieval unless L1 is unavailable or explicitly disabled by configuration
### Requirement: Knowledge retrieval SHALL use L0 domain as optional L1 filter
The retrieval flow SHALL use L0 domain/category information as an optional metadata filter for L1 retrieval when the domain is unambiguous.
#### Scenario: Single domain filter
- **WHEN** L0 hint data contains exactly one nonblank domain or category
- **THEN** the L1 retrieval request SHALL include that category as a metadata filter
#### Scenario: Ambiguous domain fallback
- **WHEN** L0 hint data contains zero domains or multiple domains
- **THEN** the L1 retrieval request SHALL run without an L0-derived category filter
### Requirement: Knowledge retrieval SHALL preserve fallback evidence
The retrieval flow SHALL still return useful L0 evidence when L1 produces no usable result.
#### Scenario: L1 has no results
- **WHEN** L0 has at least one match and L1 returns no candidates
- **THEN** the tool SHALL return an L0-based primary result
- **AND** the relevance assessment SHALL not claim semantic support from L1
#### Scenario: L1 fails
- **WHEN** L0 has at least one match and L1 retrieval throws or fails
- **THEN** the tool SHALL return an L0-based primary result
- **AND** the tool invocation record SHALL preserve the L0 hint details
### Requirement: Knowledge retrieval SHALL persist L0 hints
The system SHALL persist L0 hint details in `tool_invocation.retrieval_details` for `lookup_knowledge` calls.
#### Scenario: Retrieval details include L0 hints
- **WHEN** a `lookup_knowledge` call records a tool invocation
- **THEN** `retrieval_details` SHALL include L0 matched keywords, domains, entities, and titles when available
#### Scenario: Retrieval layer reflects cooperating retrieval
- **WHEN** both L0 hint data and L1 candidates participate in a lookup
- **THEN** the recorded retrieval layer SHALL be `L0+L1`
@@ -0,0 +1,27 @@
## 1. L0 Hint Model
- [x] 1.1 Add structured L0 hint analysis in `KnowledgeIndexService`.
- [x] 1.2 Preserve `exactMatch` compatibility for existing callers.
## 2. Retrieval Flow
- [x] 2.1 Update `LookupKnowledgeTool` so unique L0 hits no longer skip L1 by default.
- [x] 2.2 Apply a category filter to L1 only when L0 hint data has one clear domain.
- [x] 2.3 Preserve L0 fallback evidence when L1 is empty or fails.
- [x] 2.4 Adjust relevance assessment so `PRECISE` no longer depends only on unique L0.
## 3. Trace Recording
- [x] 3.1 Extend `ToolInvocationRecorder.LookupKnowledgeRecord` with L0 matched keywords, domains, and entities.
- [x] 3.2 Persist L0 hint fields in `retrieval_details`.
## 4. Tests
- [x] 4.1 Add or update unit tests for L0 hint extraction.
- [x] 4.2 Add or update tests for `lookup_knowledge` unique-L0 plus L1 participation.
- [x] 4.3 Run targeted tests and the RAG retrieval baseline evaluator.
## 5. Validation
- [x] 5.1 Run OpenSpec validation for the change.
- [x] 5.2 Review git diff to confirm only expected code/spec/test files changed.
@@ -0,0 +1,50 @@
# rag-knowledge-retrieval Specification
## Purpose
Define the runtime contract for the explicit `lookup_knowledge` Agent tool, including how L0 keyword/frontmatter hints cooperate with L1 semantic retrieval while preserving metadata filters, fallback evidence, and traceable retrieval details.
## Requirements
### Requirement: Knowledge retrieval SHALL keep L0 as a hint provider
The `lookup_knowledge` retrieval flow SHALL retain L0 keyword/frontmatter matching but use it as domain, entity, and explainability hint data rather than as the sole final retrieval decision.
#### Scenario: L0 produces traceable hint data
- **WHEN** L0 matches one or more indexed knowledge entries
- **THEN** the retrieval flow SHALL expose matched titles, matched keywords, domains or categories, and entity terms as structured hint data
#### Scenario: L0 does not bypass semantic retrieval by default
- **WHEN** L0 returns exactly one match
- **THEN** the retrieval flow SHALL still attempt semantic L1 retrieval unless L1 is unavailable or explicitly disabled by configuration
### Requirement: Knowledge retrieval SHALL use L0 domain as optional L1 filter
The retrieval flow SHALL use L0 domain/category information as an optional metadata filter for L1 retrieval when the domain is unambiguous.
#### Scenario: Single domain filter
- **WHEN** L0 hint data contains exactly one nonblank domain or category
- **THEN** the L1 retrieval request SHALL include that category as a metadata filter
#### Scenario: Ambiguous domain fallback
- **WHEN** L0 hint data contains zero domains or multiple domains
- **THEN** the L1 retrieval request SHALL run without an L0-derived category filter
### Requirement: Knowledge retrieval SHALL preserve fallback evidence
The retrieval flow SHALL still return useful L0 evidence when L1 produces no usable result.
#### Scenario: L1 has no results
- **WHEN** L0 has at least one match and L1 returns no candidates
- **THEN** the tool SHALL return an L0-based primary result
- **AND** the relevance assessment SHALL not claim semantic support from L1
#### Scenario: L1 fails
- **WHEN** L0 has at least one match and L1 retrieval throws or fails
- **THEN** the tool SHALL return an L0-based primary result
- **AND** the tool invocation record SHALL preserve the L0 hint details
### Requirement: Knowledge retrieval SHALL persist L0 hints
The system SHALL persist L0 hint details in `tool_invocation.retrieval_details` for `lookup_knowledge` calls.
#### Scenario: Retrieval details include L0 hints
- **WHEN** a `lookup_knowledge` call records a tool invocation
- **THEN** `retrieval_details` SHALL include L0 matched keywords, domains, entities, and titles when available
#### Scenario: Retrieval layer reflects cooperating retrieval
- **WHEN** both L0 hint data and L1 candidates participate in a lookup
- **THEN** the recorded retrieval layer SHALL be `L0+L1`
@@ -19,9 +19,11 @@ import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import java.util.concurrent.CopyOnWriteArrayList;
import java.util.stream.Collectors;
/**
* 知识库索引服务
@@ -123,39 +125,73 @@ public class KnowledgeIndexService {
}
public List<KnowledgeEntry> exactMatch(String query) {
return analyzeQuery(query).matches();
}
public L0Hint analyzeQuery(String query) {
long startTime = System.currentTimeMillis();
if (query == null || query.trim().isEmpty()) {
log.debug("查询关键词为空,返回空结果");
return List.of();
return L0Hint.empty();
}
String queryLower = query.toLowerCase();
List<KnowledgeEntry> results = new ArrayList<>();
Set<String> matchedKeywords = new LinkedHashSet<>();
Set<String> domains = new LinkedHashSet<>();
Set<String> entities = new LinkedHashSet<>();
Set<String> titles = new LinkedHashSet<>();
List<KnowledgeEntry> results = knowledgeIndex.stream()
.filter(entry -> matchesKeywords(entry, queryLower))
.collect(Collectors.toList());
for (KnowledgeEntry entry : knowledgeIndex) {
List<String> entryMatchedKeywords = matchedKeywords(entry, queryLower);
if (entryMatchedKeywords.isEmpty()) {
continue;
}
results.add(entry);
matchedKeywords.addAll(entryMatchedKeywords);
entities.addAll(entryMatchedKeywords);
if (entry.getCategory() != null && !entry.getCategory().isBlank()) {
domains.add(entry.getCategory());
}
if (entry.getTitle() != null && !entry.getTitle().isBlank()) {
titles.add(entry.getTitle());
}
}
long elapsedTime = System.currentTimeMillis() - startTime;
log.debug("L0精确匹配: query={}, matches={}, indexSize={}, time={}ms",
query, results.size(), knowledgeIndex.size(), elapsedTime);
log.debug("L0 Hint分析: query={}, matches={}, domains={}, keywords={}, indexSize={}, time={}ms",
query, results.size(), domains, matchedKeywords, knowledgeIndex.size(), elapsedTime);
return results;
return new L0Hint(
List.copyOf(results),
List.copyOf(matchedKeywords),
List.copyOf(domains),
List.copyOf(entities),
List.copyOf(titles)
);
}
private boolean matchesKeywords(KnowledgeEntry entry, String query) {
if (entry.getKeywords() == null || entry.getKeywords().isEmpty()) {
return false;
return !matchedKeywords(entry, query).isEmpty();
}
private List<String> matchedKeywords(KnowledgeEntry entry, String query) {
if (entry.getKeywords() == null || entry.getKeywords().isEmpty()) {
return List.of();
}
List<String> matches = new ArrayList<>();
for (String keyword : entry.getKeywords()) {
String keywordLower = keyword.toLowerCase();
if (query.contains(keywordLower) || keywordLower.contains(query)) {
return true;
matches.add(keyword);
}
}
return false;
return matches;
}
public String readDocument(String filePath, int maxChars) {
@@ -224,4 +260,20 @@ public class KnowledgeIndexService {
public List<KnowledgeEntry> getAllEntries() {
return List.copyOf(knowledgeIndex);
}
public record L0Hint(
List<KnowledgeEntry> matches,
List<String> matchedKeywords,
List<String> domains,
List<String> entities,
List<String> titles
) {
public static L0Hint empty() {
return new L0Hint(List.of(), List.of(), List.of(), List.of(), List.of());
}
public String singleDomainOrNull() {
return domains.size() == 1 ? domains.get(0) : null;
}
}
}
@@ -6,7 +6,6 @@ import com.superbiz.agent.domain.entity.ToolInvocation;
import com.superbiz.agent.dto.LookupResult;
import com.superbiz.agent.repository.ToolInvocationRepository;
import com.superbiz.agent.dto.KnowledgeEntry;
import com.superbiz.agent.service.VectorSearchService;
import com.superbiz.agent.util.SessionContextHolder;
import lombok.Builder;
import lombok.extern.slf4j.Slf4j;
@@ -108,6 +107,15 @@ public class ToolInvocationRecorder {
if (record.l0Titles() != null && !record.l0Titles().isEmpty()) {
details.put("l0_titles", record.l0Titles());
}
if (record.l0MatchedKeywords() != null && !record.l0MatchedKeywords().isEmpty()) {
details.put("l0_matched_keywords", record.l0MatchedKeywords());
}
if (record.l0Domains() != null && !record.l0Domains().isEmpty()) {
details.put("l0_domains", record.l0Domains());
}
if (record.l0Entities() != null && !record.l0Entities().isEmpty()) {
details.put("l0_entities", record.l0Entities());
}
if (record.l1TopScore() != null) {
details.put("l1_top_score", record.l1TopScore());
}
@@ -196,12 +204,15 @@ public class ToolInvocationRecorder {
String evidenceStatus,
String errorMessage,
List<String> l0Titles,
List<String> l0MatchedKeywords,
List<String> l0Domains,
List<String> l0Entities,
Double l1TopScore,
Double l1TopSimilarity,
List<Double> l1Scores
) {
public static LookupKnowledgeRecord from(String query,
List<KnowledgeEntry> l0Matches,
KnowledgeIndexService.L0Hint l0Hint,
List<VectorSearchService.SearchResult> l1Results,
boolean highConfidence,
LookupResult result,
@@ -209,6 +220,7 @@ public class ToolInvocationRecorder {
String dedupReason,
int durationMs,
double l1TopSimilarity) {
List<KnowledgeEntry> l0Matches = l0Hint != null ? l0Hint.matches() : List.of();
boolean hasL0 = l0Matches != null && !l0Matches.isEmpty();
boolean hasL1 = l1Results != null && !l1Results.isEmpty();
String layer;
@@ -272,6 +284,9 @@ public class ToolInvocationRecorder {
.success(true)
.evidenceStatus(evidenceStatus)
.l0Titles(l0Titles)
.l0MatchedKeywords(l0Hint != null ? l0Hint.matchedKeywords() : List.of())
.l0Domains(l0Hint != null ? l0Hint.domains() : List.of())
.l0Entities(l0Hint != null ? l0Hint.entities() : List.of())
.l1TopScore(hasL1 ? (double) l1Results.get(0).getScore() : null)
.l1TopSimilarity(hasL1 ? l1TopSimilarity : null)
.l1Scores(l1Scores)
@@ -35,13 +35,13 @@ public class LookupKnowledgeTool {
private static final String HINT_REFERENCE = "当前结果为相关参考,如需更精准信息请明确缺少的具体维度";
@Value("${retrieval.normalization.max-l2-distance:2.0}")
private double maxL2Distance;
private double maxL2Distance = 2.0;
@Value("${retrieval.normalization.highly-relevant-threshold:0.75}")
private double highlyRelevantThreshold;
private double highlyRelevantThreshold = 0.75;
@Value("${retrieval.normalization.reference-threshold:0.5}")
private double referenceThreshold;
private double referenceThreshold = 0.5;
@Autowired
private KnowledgeIndexService knowledgeIndexService;
@@ -83,30 +83,28 @@ public class LookupKnowledgeTool {
log.info(">>> RequestId: {}", requestId);
log.info("----------------------------------------");
// Step 1: L0 精确匹配
// Step 1: L0 hint 分析
long l0Start = System.currentTimeMillis();
List<KnowledgeEntry> l0Matches = knowledgeIndexService.exactMatch(query);
KnowledgeIndexService.L0Hint l0Hint = knowledgeIndexService.analyzeQuery(query);
List<KnowledgeEntry> l0Matches = l0Hint.matches();
long l0Time = System.currentTimeMillis() - l0Start;
log.info("[L0 精确匹配] 完成: matches={}, time={}ms", l0Matches.size(), l0Time);
log.info("[L0 Hint] 完成: matches={}, domains={}, keywords={}, time={}ms",
l0Matches.size(), l0Hint.domains(), l0Hint.matchedKeywords(), l0Time);
if (!l0Matches.isEmpty()) {
log.info("[L0 精确匹配] 找到文档:");
log.info("[L0 Hint] 找到文档:");
for (int i = 0; i < Math.min(3, l0Matches.size()); i++) {
KnowledgeEntry entry = l0Matches.get(i);
log.info(" - [{}] 标题: {}, 路径: {}, 域: {}", i+1, entry.getTitle(), entry.getFilePath(), entry.getCategory());
}
}
// Step 2: 判断是否高置信度(唯一匹配)
boolean highConfidence = (l0Matches.size() == 1);
log.info("[置信度判断] highConfidence={}, reason={}",
highConfidence, highConfidence ? "唯一匹配" : "多个或零个匹配");
// Step 3: L1 条件调用
List<VectorSearchService.SearchResult> l1Results = null;
if (!highConfidence) {
log.info("[L1 语义检索] L0非唯一匹配,触发L1语义检索...");
// Step 2: L1 默认调用;L0 只提供可解释 hint 和可选 category filter
List<VectorSearchService.SearchResult> l1Results = List.of();
String l0CategoryFilter = l0Hint.singleDomainOrNull();
try {
log.info("[L1 语义检索] 触发L1语义检索, categoryFilter={}", l0CategoryFilter);
long l1Start = System.currentTimeMillis();
l1Results = vectorSearchService.searchSimilarDocuments(query, 3, null);
l1Results = vectorSearchService.searchSimilarDocuments(query, 3, l0CategoryFilter);
long l1Time = System.currentTimeMillis() - l1Start;
log.info("[L1 语义检索] 完成: matches={}, time={}ms",
l1Results != null ? l1Results.size() : 0, l1Time);
@@ -117,25 +115,27 @@ public class LookupKnowledgeTool {
log.info(" - [{}] 文档ID: {}, L2距离: {}", i+1, result.getId(), String.format("%.4f", result.getScore()));
}
}
} else {
log.info("[L1 语义检索] L0唯一匹配,跳过L1检索");
} catch (Exception e) {
log.warn("[L1 语义检索] 调用失败,保留L0 fallback: {}", e.getMessage());
l1Results = List.of();
}
// Step 4: 归一化质量等级判定
// Step 3: 归一化质量等级判定
float l1TopScore = (l1Results != null && !l1Results.isEmpty()) ? l1Results.get(0).getScore() : Float.MAX_VALUE;
RelevanceAssessment assessment = computeRelevance(l0Matches.size(), l1TopScore);
boolean highConfidence = isHighConfidence(l0Matches.size(), l1TopScore);
log.info("[归一化] relevanceLevel={}, completenessHint={}", assessment.level, assessment.hint);
if (l1TopScore != Float.MAX_VALUE) {
double similarity = normalizeL2(l1TopScore);
log.info("[归一化] L2距离={}, similarity={}", String.format("%.4f", l1TopScore), String.format("%.4f", similarity));
}
// Step 5: 组装结果
// Step 4: 组装结果
LookupResult result = buildResult(l0Matches, l1Results, highConfidence);
result.setRelevanceLevel(assessment.level);
result.setCompletenessHint(assessment.hint);
// Step 6: session 级去重过滤 + 域级行动记忆
// Step 5: session 级去重过滤 + 域级行动记忆
String sessionId = SessionContextHolder.getSessionId();
String domain = extractDomain(l0Matches, l1Results);
@@ -144,7 +144,7 @@ public class LookupKnowledgeTool {
if (docKey != null && retrievedDocTracker.isAlreadyRetrieved(sessionId, docKey)) {
log.info("[去重] 文档已在本会话中检索过,跳过: {}", docKey);
List<String> retrievedDomains = retrievedDocTracker.getRetrievedDomains(sessionId);
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result, domain, "doc_retrieved");
saveToolInvocation(query, l0Hint, l1Results, highConfidence, startTime, result, domain, "doc_retrieved");
return LookupResult.builder()
.found(false)
.message("文档已在本会话中检索过,无需重复召回: " + docKey)
@@ -197,7 +197,7 @@ public class LookupKnowledgeTool {
log.info("========================================");
// 记录 tool_invocation
saveToolInvocation(query, l0Matches, l1Results, highConfidence, startTime, result, domain, null);
saveToolInvocation(query, l0Hint, l1Results, highConfidence, startTime, result, domain, null);
return result;
}
@@ -225,11 +225,16 @@ public class LookupKnowledgeTool {
RelevanceAssessment computeRelevance(int l0MatchCount, float l1TopScore) {
double l1Similarity = (l1TopScore != Float.MAX_VALUE) ? normalizeL2(l1TopScore) : 0.0;
// L0 唯一匹配 → PRECISE
if (l0MatchCount == 1) {
// L0 唯一匹配 + L1 高分 → PRECISE
if (l0MatchCount == 1 && l1Similarity >= highlyRelevantThreshold) {
return new RelevanceAssessment(LEVEL_PRECISE, HINT_PRECISE);
}
// L0 唯一匹配但缺少 L1 支持 → REFERENCE
if (l0MatchCount == 1) {
return new RelevanceAssessment(LEVEL_REFERENCE, HINT_REFERENCE);
}
// L0 命中 + L1 高分 → HIGHLY_RELEVANT
if (l0MatchCount > 1 && l1Similarity >= highlyRelevantThreshold) {
return new RelevanceAssessment(LEVEL_HIGHLY_RELEVANT, HINT_HIGHLY_RELEVANT);
@@ -259,6 +264,16 @@ public class LookupKnowledgeTool {
return new RelevanceAssessment(null, null);
}
boolean isHighConfidence(int l0MatchCount, float l1TopScore) {
if (l0MatchCount != 1) {
return false;
}
if (l1TopScore == Float.MAX_VALUE) {
return true;
}
return normalizeL2(l1TopScore) >= highlyRelevantThreshold;
}
/**
* 归一化评估结果
*/
@@ -302,7 +317,7 @@ public class LookupKnowledgeTool {
/**
* 保存工具调用明细到 tool_invocation 表
*/
private void saveToolInvocation(String query, List<KnowledgeEntry> l0Matches,
private void saveToolInvocation(String query, KnowledgeIndexService.L0Hint l0Hint,
List<VectorSearchService.SearchResult> l1Results,
boolean highConfidence, long startTime,
LookupResult result, String domain, String dedupReason) {
@@ -317,7 +332,7 @@ public class LookupKnowledgeTool {
ToolInvocationRecorder.LookupKnowledgeRecord record = ToolInvocationRecorder.LookupKnowledgeRecord.from(
query,
l0Matches,
l0Hint,
l1Results,
highConfidence,
result,
@@ -98,6 +98,46 @@ class KnowledgeIndexServiceTest {
assertEquals(2, results.size());
}
@Test
void testAnalyzeQuery_returnsStructuredHint() {
KnowledgeEntry entry = KnowledgeEntry.builder()
.filePath("mysql.md")
.title("MySQL Doc")
.keywords(List.of("mysql", "connection pool"))
.category("database")
.build();
service.addToIndex(entry);
KnowledgeIndexService.L0Hint hint = service.analyzeQuery("mysql connection pool timeout");
assertEquals(1, hint.matches().size());
assertEquals(List.of("mysql", "connection pool"), hint.matchedKeywords());
assertEquals(List.of("database"), hint.domains());
assertEquals(List.of("mysql", "connection pool"), hint.entities());
assertEquals(List.of("MySQL Doc"), hint.titles());
assertEquals("database", hint.singleDomainOrNull());
}
@Test
void testAnalyzeQuery_multipleDomainsHasNoSingleDomain() {
service.addToIndex(KnowledgeEntry.builder()
.filePath("mysql.md")
.keywords(List.of("timeout"))
.category("database")
.build());
service.addToIndex(KnowledgeEntry.builder()
.filePath("api.md")
.keywords(List.of("timeout"))
.category("api")
.build());
KnowledgeIndexService.L0Hint hint = service.analyzeQuery("timeout");
assertEquals(2, hint.matches().size());
assertNull(hint.singleDomainOrNull());
}
@Test
void testExactMatch_noMatch() {
KnowledgeEntry entry = KnowledgeEntry.builder()
@@ -75,6 +75,9 @@ class ToolInvocationRecorderTest {
.success(true)
.evidenceStatus(ToolInvocationRecorder.EVIDENCE_STATUS_DEDUPED)
.l0Titles(List.of("payment/errors.md"))
.l0MatchedKeywords(List.of("ERR_TIMEOUT"))
.l0Domains(List.of("payment"))
.l0Entities(List.of("ERR_TIMEOUT"))
.build();
try {
@@ -92,5 +95,8 @@ class ToolInvocationRecorderTest {
assertEquals("doc_retrieved", saved.getDedupReason());
assertTrue(saved.getRetrievalDetails().contains("\"evidence_status\":\"deduped\""));
assertTrue(saved.getRetrievalDetails().contains("\"retrieved_domains\":[\"payment\"]"));
assertTrue(saved.getRetrievalDetails().contains("\"l0_matched_keywords\":[\"ERR_TIMEOUT\"]"));
assertTrue(saved.getRetrievalDetails().contains("\"l0_domains\":[\"payment\"]"));
assertTrue(saved.getRetrievalDetails().contains("\"l0_entities\":[\"ERR_TIMEOUT\"]"));
}
}
@@ -48,7 +48,7 @@ class LookupKnowledgeToolTest {
}
@Test
void testLookup_uniqueMatch_highConfidence() {
void testLookup_uniqueMatch_usesL0HintAndL1() {
// 准备 L0 唯一匹配
KnowledgeEntry entry = KnowledgeEntry.builder()
.filePath("test.md")
@@ -57,10 +57,16 @@ class LookupKnowledgeToolTest {
.summary("Test summary")
.build();
when(knowledgeIndexService.exactMatch("ERR_TIMEOUT"))
.thenReturn(List.of(entry));
when(knowledgeIndexService.analyzeQuery("ERR_TIMEOUT"))
.thenReturn(hint(entry));
when(knowledgeIndexService.readDocument("test.md", 2000))
.thenReturn("Test content * 用于构建紧凑摘要 * keyword2");
VectorSearchService.SearchResult l1Result = new VectorSearchService.SearchResult();
l1Result.setContent("L1 supporting content");
l1Result.setMetadata("l1-source");
l1Result.setScore(0.4f);
when(vectorSearchService.searchSimilarDocuments("ERR_TIMEOUT", 3, null))
.thenReturn(List.of(l1Result));
// 执行查询
LookupResult result = tool.lookupKnowledge("ERR_TIMEOUT");
@@ -75,10 +81,10 @@ class LookupKnowledgeToolTest {
assertTrue(content.contains("文档: Test Doc"));
assertTrue(content.contains("摘要: Test summary"));
assertTrue(content.contains("Test content"));
assertNull(result.getSupplement()); // 高置信度不调用 L1
assertNotNull(result.getSupplement()); // L0 唯一命中仍调用 L1
assertEquals("PRECISE", result.getRelevanceLevel());
// 验证 L1 未被调用
verify(vectorSearchService, never()).searchSimilarDocuments(anyString(), anyInt(), any());
verify(vectorSearchService).searchSimilarDocuments("ERR_TIMEOUT", 3, null);
}
@Test
@@ -96,8 +102,8 @@ class LookupKnowledgeToolTest {
.keywords(List.of("超时"))
.build();
when(knowledgeIndexService.exactMatch("超时"))
.thenReturn(List.of(entry1, entry2));
when(knowledgeIndexService.analyzeQuery("超时"))
.thenReturn(hint(entry1, entry2));
// 多匹配 + L1 有结果 → buildMetadataOnlySummary(),不读文件,不调用 readDocument
// 准备 L1 结果
@@ -134,8 +140,8 @@ class LookupKnowledgeToolTest {
@Test
void testLookup_noL0Match_onlyL1() {
// L0 未匹配
when(knowledgeIndexService.exactMatch("性能优化"))
.thenReturn(Collections.emptyList());
when(knowledgeIndexService.analyzeQuery("性能优化"))
.thenReturn(KnowledgeIndexService.L0Hint.empty());
// 准备 L1 结果
VectorSearchService.SearchResult l1Result = new VectorSearchService.SearchResult();
@@ -160,8 +166,8 @@ class LookupKnowledgeToolTest {
@Test
void testLookup_noMatch() {
// L0 和 L1 都未匹配
when(knowledgeIndexService.exactMatch("不存在的内容"))
.thenReturn(Collections.emptyList());
when(knowledgeIndexService.analyzeQuery("不存在的内容"))
.thenReturn(KnowledgeIndexService.L0Hint.empty());
when(vectorSearchService.searchSimilarDocuments("不存在的内容", 3, null))
.thenReturn(Collections.emptyList());
@@ -182,12 +188,12 @@ class LookupKnowledgeToolTest {
.keywords(List.of("test"))
.build();
when(knowledgeIndexService.exactMatch("test"))
.thenReturn(List.of(entry));
when(knowledgeIndexService.analyzeQuery("test"))
.thenReturn(hint(entry));
when(knowledgeIndexService.readDocument("nonexistent.md", 2000))
.thenReturn(null); // 读取失败
// L0 唯一匹配不会调用 L1,所以没有补充结果
when(vectorSearchService.searchSimilarDocuments("test", 3, null))
.thenReturn(Collections.emptyList());
// 执行查询
LookupResult result = tool.lookupKnowledge("test");
@@ -195,17 +201,16 @@ class LookupKnowledgeToolTest {
// 验证:found 为 false,因为无法读取内容且无 L1 补充
assertFalse(result.isFound());
assertNull(result.getPrimary());
assertNull(result.getSupplement()); // 唯一匹配不调用 L1
assertNull(result.getSupplement());
// 验证 L1 未被调用(因为是唯一匹配 = 高置信度)
verify(vectorSearchService, never()).searchSimilarDocuments(anyString(), anyInt(), any());
verify(vectorSearchService).searchSimilarDocuments("test", 3, null);
}
@Test
void testLookup_l1ReturnsNull() {
// L0 未匹配,L1 返回 null
when(knowledgeIndexService.exactMatch("query"))
.thenReturn(Collections.emptyList());
when(knowledgeIndexService.analyzeQuery("query"))
.thenReturn(KnowledgeIndexService.L0Hint.empty());
when(vectorSearchService.searchSimilarDocuments("query", 3, null))
.thenReturn(null);
@@ -224,14 +229,83 @@ class LookupKnowledgeToolTest {
.keywords(List.of("test"))
.build();
when(knowledgeIndexService.exactMatch("test"))
.thenReturn(List.of(entry));
when(knowledgeIndexService.analyzeQuery("test"))
.thenReturn(hint(entry));
when(knowledgeIndexService.readDocument("test.md", 2000))
.thenReturn("Content");
when(vectorSearchService.searchSimilarDocuments("test", 3, null))
.thenReturn(Collections.emptyList());
LookupResult result = tool.lookupKnowledge("test");
assertNotNull(result.getPrimary());
assertNull(result.getPrimary().getAvailableSections()); // MVP 返回 null
}
@Test
void testLookup_appliesSingleL0DomainAsL1Filter() {
KnowledgeEntry entry = KnowledgeEntry.builder()
.filePath("db.md")
.title("Database Doc")
.keywords(List.of("mysql"))
.summary("Database summary")
.category("database")
.build();
when(knowledgeIndexService.analyzeQuery("mysql timeout"))
.thenReturn(hint(entry));
when(knowledgeIndexService.readDocument("db.md", 2000))
.thenReturn("Database content");
when(vectorSearchService.searchSimilarDocuments("mysql timeout", 3, "database"))
.thenReturn(Collections.emptyList());
LookupResult result = tool.lookupKnowledge("mysql timeout");
assertTrue(result.isFound());
verify(vectorSearchService).searchSimilarDocuments("mysql timeout", 3, "database");
}
@Test
void testLookup_l1FailureKeepsL0Fallback() {
KnowledgeEntry entry = KnowledgeEntry.builder()
.filePath("fallback.md")
.title("Fallback Doc")
.keywords(List.of("fallback"))
.summary("Fallback summary")
.build();
when(knowledgeIndexService.analyzeQuery("fallback"))
.thenReturn(hint(entry));
when(knowledgeIndexService.readDocument("fallback.md", 2000))
.thenReturn("Fallback content");
when(vectorSearchService.searchSimilarDocuments("fallback", 3, null))
.thenThrow(new RuntimeException("milvus unavailable"));
LookupResult result = tool.lookupKnowledge("fallback");
assertTrue(result.isFound());
assertNotNull(result.getPrimary());
assertNull(result.getSupplement());
assertEquals("REFERENCE", result.getRelevanceLevel());
verify(vectorSearchService).searchSimilarDocuments("fallback", 3, null);
}
private KnowledgeIndexService.L0Hint hint(KnowledgeEntry... entries) {
List<KnowledgeEntry> matches = List.of(entries);
List<String> keywords = matches.stream()
.flatMap(entry -> entry.getKeywords() == null ? java.util.stream.Stream.empty() : entry.getKeywords().stream())
.distinct()
.toList();
List<String> domains = matches.stream()
.map(KnowledgeEntry::getCategory)
.filter(category -> category != null && !category.isBlank())
.distinct()
.toList();
List<String> titles = matches.stream()
.map(KnowledgeEntry::getTitle)
.filter(title -> title != null && !title.isBlank())
.distinct()
.toList();
return new KnowledgeIndexService.L0Hint(matches, keywords, domains, keywords, titles);
}
}