Retrieval Augmented Generation Compositionality
AuthorsIdo Levy, Segev Shlomov, Leshem Choshen
AffiliationsIBM · Weizmann Institute of Science
A large-scale study finds that RAG techniques that work well alone may not work well together, and the best combination can change as the knowledge corpus grows.
Key results
Total configurations across the factorial study, ablations, and scaling experiments.
Predicted NDCG@10 lift from adding the three strongest single-component improvements.
NDCG@10 lift delivered when those three techniques were combined.
From this corpus size onward, BALANCED leads STANDARD in retrieval quality.
Relative NDCG@10 lead over STANDARD at 200K documents.
Highest joint retrieval-faithfulness-latency hypervolume at the 30 s reference.
What the paper found
This paper asks whether RAG techniques that look strong alone remain effective when assembled. It introduces LASAGNA, a stage-isolated framework, and evaluates 450 configurations across eight benchmarks and corpora from 5K to 200K documents, varying embeddings, chunking, enrichment, retrieval, query augmentation, reranking, compression, and agentic verification. Using NDCG@10 for retrieval and RAGAS faithfulness for answer grounding—with GPT-5.1 from OpenAI as generator and judge, plus a Claude Sonnet 4.5 cross-check—the study finds that the metrics favor different pipeline choices: embeddings and enrichment are most associated with retrieval quality, while query augmentation and agentic gates are most associated with faithfulness. Most strikingly, the three strongest isolated retrieval gains predict a +47% NDCG@10 lift, but their combined pipeline delivers only +17%, as overlapping functions limit additive gains. Corpus scale also changes the winner: STANDARD leads on smaller corpora, but BALANCED overtakes it from 25K documents and is 13.7% ahead at 200K. At a 30 s latency reference, STANDARD has the highest joint quality-latency hypervolume, 0.523, showing there is no universally best profile. The practical contribution is methodological: evaluate new components inside a representative full pipeline, on both retrieval and faithfulness, and at the intended deployment scale; isolated ablations can mislead.
Original abstract
Retrieval Augmented Generation (RAG) grounds large language models in external knowledge. Modern pipelines combine embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification, yet these techniques are typically evaluated one at a time. It remains unclear whether their isolated gains survive composition, whether retrieval quality and answer faithfulness favor the same design choices, and whether conclusions from small corpora hold at large scale. We pre-declare three hypotheses and introduce LASAGNA, an open-source framework for controlled RAG evaluation. We test 450 configurations across eight benchmarks and corpora ranging from 5K to 200K documents. Three findings emerge. First, individually strong techniques from different phases of a RAG pipeline do not necessarily compose: three retrieval techniques predict a +47% combined improvement but deliver only +17% when used together. Second, component gains stack differently for retrieval and faithfulness, and optimizing one can hurt the other. Third, the best composition changes as the corpus grows: the combination that wins on a small corpus may not remain best at large scale. These findings show that gains measured in isolation are not reliable predictors of realistic performance, and that new RAG techniques should be tested with the full surrounding stack. We call on researchers to do so and supply LASAGNA to simplify these experiments.
Read the original paperMore in Large Language Models
Browse all 87 papers →Copy-Paste Language Models
Nathan Godey, Yoav Artzi
Copy-Paste Language Models add a low-overhead copying mechanism that improves language-model training and performance, especially in long-context and retrieval settings.
Large Flow Language Models
Arnav Chavan, Chanhyuk Lee, Udbhav Bamba, Jaehoon Yoo, Seunghoon Hong, Aditi Raghunathan, Jinwoo Kim
Lafa turns large language models into fast continuous-flow generators, reporting up to 2.54× throughput at comparable quality.
HuatuoGPT-3: RL-Only Domain Adaptation from Base Models
Junying Chen, Xinyuan Xie, Ziniu Li, Wenyuan Gu, Jianquan Li, Xiang Wan, Guangjun Yu, Ruoyu Sun, Haizhou Li, Benyou Wang
OnePO uses temporary teacher guidance and reinforcement learning alone to turn general LLMs into stronger medical specialists without the usual supervised fine-tuning stage.