Trees to Flows and Back: Unifying Decision Trees and Diffusion Models
AuthorsSai Niranjan Ramachandran, Suvrit Sra
Resources
This paper connects decision trees and diffusion models through a shared mathematical framework, then uses that link to speed up tabular generation and distill tree behavior into neural networks.
Key results
TREFLOW had the highest TSTR accuracy on 3 of 5 tabular benchmarks.
TREFLOW achieved the lowest Wasserstein distance on 4 of 5 benchmarks.
TREFLOW was reported to be 2× faster than TabDDPM.
DSM-TREE exceeded the teacher on Heart Disease by 3.7%.
What the paper found
In “Trees to Flows and Back: Unifying Decision Trees and Diffusion Models,” Sai Niranjan Ramachandran and Suvrit Sra from the Technical University of Munich and MCML argue that hierarchical decision trees and diffusion models are two limits of the same process: a tree induces a deterministic probability-flow ODE through dyadic refinement, while a suitable diffusion process induces a canonical dendrogram via moment-based merger times. They package this equivalence as Global Trajectory Score Matching, showing that greedy gradient boosting is asymptotically optimal for a discrete trajectory-matching objective and that standard diffusion training appears as a continuous version of the same loss. The paper’s two concrete algorithms are TREFLOW, which conditions conditional flow matching on tree path encodings and reaches the highest TSTR accuracy on 3 of 5 tabular benchmarks, the lowest Wasserstein distance on 4 of 5, and the lowest correlation error on 3 of 5, while running 2× faster than TabDDPM; and DSM-TREE, which distills full decision paths into a neural network and matches teacher performance within 2% on 4 of 5 datasets, exceeding the teacher by 3.7% on Heart Disease. On tabular generation, TREFLOW reports 0.981 TSTR accuracy on Wine with 1.816 runtime versus 3.901 for TabDDPM, and on distillation DSM-TREE reaches 75.31% accuracy on Heart Disease versus 71.60% for the base tree.
Original abstract
Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.
Read the original paper