# Decisions: rag-chunk-evidence-identity-dedup

## Capability sources

- sm-flow orchestration
- OpenSpec fallback protocol (file-based propose/apply/archive) — external openspec-propose/apply skills used as reference; execution via sm-flow fallback
- grill: fallback built-in protocol
- audit: fallback built-in protocol

## Scale

standard

## Clarify

- Problem: same-document multi-chunk evidence collapsed by source-level dedup.
- Outcome: Delivery 1 foundation before hybrid Delivery 2.
- Slug: `rag-chunk-evidence-identity-dedup`
- User authorized apply + archive in advance for sm-flow staged changes.

## Context

- Read: `devflow/glossary/CONTEXT.md`, modular-rag-pipeline brief, `openspec/specs/rag-knowledge-retrieval`, `rag-log-projections`, checklist doc §1.1
- Constraints into OpenSpec:
  - L0 hint-only remains
  - Do not thicken legacy SDK path
  - Agent tool name/input stable
  - Hybrid out of scope this change

## Question pool (grill)

| # | Dimension | Mode | Question | Status |
|---|---|---|---|---|
| Q1 | 术语 | evidence-driven | evidenceKey / document_id 语义？ | Resolved: evidenceKey=chunk id; projected document_id=evidenceKey |
| Q2 | 边界 | evidence-driven | Delivery 1 vs 2 边界？ | Resolved: per checklist; no schema/hybrid/SDK delete |
| Q3 | 验收 | evidence-driven | 如何验收多 chunk？ | Resolved: unit tests multi-chunk keep + projector |
| Q4 | 接口 | user-interview | document_id 改为 chunk 级是否可接受？ | **Pre-authorized by user** via “apply/archive 直接授权” + prior design agreement on scheme A (document_id=evidenceKey). Recorded as accepted behavior change. |
| Q5 | 技术 | evidence-driven | SearchPort 是否本 change 必须？ | Resolved: thin port required as foundation |

### Evidence-driven conclusions (reported)

1. Current collapse points: `KnowledgeEvidencePostProcessor.sourceKey` and `RagResultProjector` source fallback.
2. Metadata already has docId/chunkIndex on write path; not first-class on read path.
3. Existing main-spec still says source-level dedup — this change intentionally deltas that requirement.

### User-interview

- Q4 accepted under prior design alignment (scheme A) and explicit apply authorization for this sm-flow run. No remaining open product preference questions for Delivery 1.

## Audit

Module chain:

```text
LookupKnowledgeTool -> SearchPort -> Retriever -> PostProcessor -> Packer -> Assembler -> RagResultProjector
```

Risks:

1. Agent payload growth — mitigated by return-n + maxChunksPerDocument + projector budgets.
2. document_id semantic shift — documented L3 behavior change; tests updated.
3. Old data without chunkIndex — vector id fallback.

No ADR conflict with modular RAG L0/L1 boundary.

## Cross-artifact alignment

| From | To | Status |
|---|---|---|
| brief goals | proposal | 已对齐 |
| proposal scope | design decisions | 已对齐 |
| design identity/dedup/port | specs | 已对齐 |
| specs scenarios | tasks | 已对齐 |

## Interface impact

- L2 internal DTO
- L3 Agent `document_id` chunk-scoped

## Commit gate

- proposal/design/specs/tasks present
- no open user-interview blockers for Delivery 1
- apply authorized by user at sm-flow start

## Pre-apply research

Reference files:

- `LookupKnowledgeTool.java`
- `KnowledgeDocumentRetriever.java`
- `KnowledgeEvidencePostProcessor.java`
- `RagResultProjector.java`
- `LookupKnowledgeToolTest.java`
- `RagResultProjectorTest.java`
- `docs/milvus-hybrid-search-integration-checklist.md`

Stack notes:

- No MQ/request envelope changes
- Spring `@Value` config pattern for rag.* keys
- Tests use ReflectionTestUtils + Mockito