NTH

DeFT-RAG: Decomposing Multi-Hop Questions into Self-Contained Sub-Questions with Reusable Answers

AuthorsDingyi Kang, Bingzhe Li

AffiliationsUniversity of Texas at Dallas, Richardson, Texas, USA

November 9, 2026 3 min read
Watch on YouTube
The one-line take

DeFT-RAG helps language models answer multi-hop questions by breaking them into focused sub-questions and reusing answers they have already found.

Key results

0.629
MuSiQue accuracy, gpt-4o-mini

LLM-judged accuracy; runner-up Hi-Q scored 0.577.

0.636
MuSiQue accuracy, Qwen3.6-35B-A3B

LLM-judged accuracy; runner-up Hi-Q scored 0.583.

46%
Three-hop evidence-chain coverage

Share of three-hop MuSiQue questions where the LLM received all required passages.

23.1%
Cached sub-questions

Share answered from the QA cache across 1,000 MuSiQue questions.

12.9%
LLM-call reduction

Reduction from QA-cache reuse without lowering accuracy.

1569
LLM input tokens per question

Average for DeFT-RAG, lower than every other tested iterative method.

What the paper found

DeFT-RAG tackles multi-hop retrieval-augmented generation by asking an LLM to produce one self-contained sub-question at a time, naming each entity explicitly instead of planning a full chain in advance. A rule-updated question-and-answer table tracks what is known, while a fine-tuned lightweight decision model selects relevant passages and decides when enough evidence has been found; a cache reuses answers to equivalent sub-questions. Across MuSiQue, 2WikiMultiHopQA, and HotpotQA, it achieved the highest accuracy among 13 baselines with both OpenAI’s gpt-4o-mini and Qwen3.6-35B-A3B. On MuSiQue, accuracy reached 0.629 with gpt-4o-mini and 0.636 with Qwen3.6, compared with 0.577 and 0.583 for the respective runners-up. DeFT-RAG delivered the full evidence chain for 46% of three-hop MuSiQue questions, using an average of 390 words of passages per question. Its QA cache answered 23.1% of sub-questions across 1,000 MuSiQue questions, cutting LLM calls by 12.9% without lowering accuracy. The trade-off is latency: frequent decision-model calls make the method slower than several alternatives, even though its LLM input averages 1,569 tokens per question, less than any other tested iterative method.

Original abstract

Retrieval-augmented generation (RAG) lets a large language model (LLM) answer questions by retrieving relevant passages from a large corpus. However, RAG struggles with multi-hop questions, which require chaining facts spread across several documents through intermediate entities that the question never mentions. Graph-based RAG methods retrieve once from the entities a question names and so rarely recover the full evidence chain. Iterative RAG methods recover it step by step, but their accuracy is limited by the many tasks they ask the LLM to perform, such as planning the sub-questions, deciding their dependencies, rewriting them as answers arrive, and keeping track of what is known. We propose DeFT-RAG, a RAG framework built on three components. First, DeFT-RAG generates one self-contained sub-question at a time from the original question and the sub-questions answered so far with their answers (QA pairs). Each sub-question names every entity it asks about, which removes the need to plan, track dependencies or rewrite, and lets the LLM focus on a single fine-grained task at a time. Second, DeFT-RAG moves the remaining LLM work outside the LLM: rule-based updates to a QA table that keeps track of what is known, and yes/no decisions made by a fine-tuned lightweight decision model. Third, DeFT-RAG caches the QA pairs and reuses an answer when a later question asks a sub-question with the same meaning. On MuSiQue, 2WikiMultiHopQA and HotpotQA, DeFT-RAG achieves the best accuracy among thirteen baselines with both gpt-4o-mini and Qwen3.6 as the backbone, while giving the LLM fewer input tokens than any other iterative method. With a QA cache shared across 1,000 MuSiQue questions, 23.1% of sub-questions are answered from the cache, which reduces LLM calls by 12.9% without lowering accuracy.

Read the original paper

More in Large Language Models

Browse all 99 papers →