feat(phase1): 实现文档分块向量化索引

Task 5.6: 向量化索引实现
- VectorIndexService 新增方法:
  - indexDocumentChunks(docId, chunks): 索引文档分块到 Milvus
  - deleteDocumentChunks(docId): 删除文档的所有向量
  - buildDocumentMetadata(): 构建文档元数据(区分文件索引)

核心流程:
1. 上传时:文本提取 → 分块 → 向量化 → 存入 Milvus + MySQL
2. 检索时:问题向量化 → Milvus 语义检索 → 返回相似文档
3. 删除时:删除元数据 + 删除向量索引

实现细节:
- 复用 indexSingleFile 的向量化逻辑
- metadata.docId 标识文档来源(区分 upload: 和 file:)
- 删除表达式:metadata["docId"] == "xxx"
- 自动去重:上传前删除旧向量数据

DocumentManagementService 完整实现:
- uploadDocument: 完整向量化流程(移除 TODO)
- deleteDocument: 同步删除向量索引(移除 TODO)

编译验证:BUILD SUCCESS

Progress: 32/34 tasks completed (94%)
This commit is contained in:
zhuyongxin
2026-06-23 16:08:50 +08:00
parent 26aaf149d8
commit 4ef8d87961
3 changed files with 119 additions and 12 deletions
@@ -108,22 +108,19 @@ public class DocumentManagementService {
apiDocumentRepository.save(document);
log.info("文档元数据已保存,docId: {}", docId);
// 6. 向量化并索引(TODO: 待实现批量分块索引)
// 6. 向量化并索引
try {
// TODO: 实现 VectorIndexService.indexDocumentChunks(docId, chunks)
// 当前暂时标记为 INDEXED,后续补充实际向量化逻辑
log.warn("向量化索引功能待实现,docId: {}", docId);
vectorIndexService.indexDocumentChunks(docId, chunks);
document.setStatus("INDEXED");
document.setIndexedAt(LocalDateTime.now());
apiDocumentRepository.save(document);
log.info("文档元数据已创建(向量化待实现),docId: {}", docId);
log.info("文档索引完成,docId: {}", docId);
} catch (Exception e) {
log.error("文档处理失败,docId: {}", docId, e);
log.error("文档索引失败,docId: {}", docId, e);
document.setStatus("FAILED");
apiDocumentRepository.save(document);
throw new DocumentProcessException(docId, "process", "文档处理失败: " + e.getMessage(), e);
throw new DocumentProcessException(docId, "index", "向量化索引失败: " + e.getMessage(), e);
}
return docId;
@@ -208,8 +205,13 @@ public class DocumentManagementService {
ApiDocument doc = optional.get();
// TODO: 删除向量索引
log.warn("向量索引删除功能待实现,docId: {}", docId);
// 删除向量索引
try {
vectorIndexService.deleteDocumentChunks(docId);
log.info("文档向量索引已删除,docId: {}", docId);
} catch (Exception e) {
log.warn("删除向量索引失败,docId: {}", docId, e);
}
// 删除元数据
apiDocumentRepository.delete(doc);