TabFM: A Zero-Shot Foundation Model for Tabular Data

TabFM is a 400M-parameter tabular foundation model that treats supervised prediction as in-context learning: it receives labeled rows and produces calibrated zero-shot predictions for new rows in a single forward pass, without task-specific tuning. Pretrained entirely on synthetic tables generated by structural causal models, it combines learned Fourier cell embeddings, dyadic feature grouping, column-wise Induced Self-Attention Blocks with 256 inducing points, row-wise Self-Attention with RoPE, and a 24-layer in-context predictor. This design separates row and feature interactions, reducing column-attention complexity from quadratic in the number of rows and enabling contexts of up to 16,384 instances. On TabArena’s 51 datasets—38 classification and 13 regression—TabFM ranks first among default tabular foundation models, reaching 1768.6 Elo for classification and 2055.2 Elo for regression, ahead of alternatives including TabPFN-3, TabICLv2, EXAONE-Tabular, and AutoGluon. With the weights frozen, TabFM+ runs 32 transformed views, adds cross and truncated-SVD features, ensembles predictions, and calibrates outputs, gaining 69.4 Elo on classification and 134.0 Elo on regression. TabFM-Auto extends this approach with Gemini-3.8-Flash, which synthesizes dataset-specific preprocessing and feature-engineering programs; it improves 41 of 51 datasets and ranks first on both benchmark tracks without updating TabFM’s parameters.