NTH

BYOD: Build Your Own DLM - Efficient Autoregressive-to-Diffusion Language Model Conversion

AuthorsRuurd J. A. Kuiper, Robert A. Bagheri

AffiliationsJulius Center for Health Sciences and Primary CareUniversity Medical Center Utrecht · Department of Methodology and StatisticsUtrecht University, The Netherlands

November 11, 2026 2 min read
Watch on YouTube
The one-line take

BYOD shows how to turn existing language models into diffusion generators with small LoRA adapters, while highlighting the quality gap that remains.

Key results

102.4M
Maximum conversion token budget

Maximum tokens used for a conversion.

4.0 hours
Maximum conversion time

Longest reported conversion on one NVIDIA RTX PRO 6000 GPU.

50.6%
BYOD-Gemma average accuracy

Average across nine benchmark tasks.

51.9%
LLaDA 8B Instruct matched-harness accuracy

Average across the same nine benchmark tasks.

100%
Bidirectional-test prediction-change rate

Each converted model changed its prediction when given the later clue.

What the paper found

BYOD offers a LoRA-only method for converting autoregressive models into bidirectional masked diffusion language models. It replaces causal attention with bidirectional attention and trains rank-1024 adapters on query and value projections, while freezing the base-model weights. The same recipe converts Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B—the first reported diffusion conversion of a Mistral-family model—using at most 102.4M tokens and taking up to 4.0 hours on a single NVIDIA RTX PRO 6000 GPU. On nine benchmarks including MMLU, GSM8K, HumanEval, and MBPP, BYOD-Gemma reaches 50.6% average accuracy, close to 51.9% for LLaDA 8B Instruct evaluated in the same harness. A right-context copying test confirms that every converted model uses bidirectional information: adding the later clue changed predictions on 100% of test items. The main shortcoming is open-ended fluency: at 64 denoising steps for 128-token generations, BYOD perplexity is 8.3–11.1, compared with 4.1 for LLaDA and 2.2–3.0 for the autoregressive parents. BYOD makes diffusion conversion accessible with modest training compute, but does not yet match the generation quality of its parent models or LLaDA.

Original abstract

Unlike autoregressive (AR) models, diffusion language models (DLMs) permit parallel token updates, flexible generation, and bidirectional attention, but leading systems rely on costly pretraining or full-model fine-tuning. We present Build Your Own DLM (BYOD), a generalizable LoRA-only recipe for parameter-efficient AR-to-DLM conversion. Without architecture-specific tuning, BYOD induces diffusion generation in Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B, and Ministral 8B; to our knowledge, this is the first reported AR-to-DLM conversion of a Mistral model. The adapters comprise 4.1-5.8% of final parameters while every base-model parameter remains frozen; each 25,000-iteration conversion needs at most 102.4M tokens, processed in 3.1-4.0 hours on one Google Colab GPU. All four models generate from fully masked sequences, use bidirectional context in a controlled attention test, and retain substantial parent-model knowledge. BYOD-Gemma is the strongest BYOD conversion on our nine-task evaluation, reaching 50.6% average accuracy compared with 51.9% for LLaDA 8B Instruct in the same harness. Open-generation quality at low network function evaluation (NFE) budgets remains the principal limitation: BYOD perplexity is 8.3-11.1 versus 4.1 for LLaDA and 2.2-3.0 for the AR parents when generating 128 tokens in 64 steps. To enable further experimentation and inspection, we provide a standalone training and evaluation implementation and interactive demos for all four conversions. BYOD thus provides a practical route for extending diffusion generation to other existing model families while identifying the gap that efficient conversion must next close.

Read the original paper

More in Diffusion Models

Browse all 82 papers →