ICICLE: Expanding Retrieval with In-Context Documents

ICICLE, from National Taiwan University and Johns Hopkins University, reframes incremental generative retrieval as an in-context indexing problem rather than a parameter-update problem. Instead of retraining a retriever to learn new document–docid links, it supplies newly added documents and their titles as inference-time evidence, then teaches a Qwen3-1.7B-base model to route between contextual copy and parametric memory using a special [COPY] token, hard-negative in-context templates, Direct Preference Optimization, and a large-title-context LoRA adaptation stage. On MS MARCO and NQ320K, with 90/10 corpus splits and up to 100 in-context candidates, ICICLE reaches 0.607 Hits@1 and 0.800 Hits@10 on unseen MS MARCO documents and 0.649 Hits@1 and 0.772 Hits@10 on unseen NQ320K documents, outperforming incremental baselines such as DSI++, DOME, and no-retrain DSI while avoiding O(M) or O(N+M) corpus-update costs. The paper’s main technical finding is that scaling failures are driven less by retrieval ranking than by routing failure: the model often knows the answer but fails to activate [COPY] as the candidate set grows, and DPO substantially improves this source-selection calibration. The authors also show that abstractive document compression with Qwen2.5-14B-Instruct preserves retrieval quality better than keyword compression, enabling more candidates within the context window without destroying rank-1 accuracy.