NTH

LightTAR: Light and Fast Target-Aware Representations for Multimodal Tabular Learning

AuthorsEilam Shapira, Alan Arazi, Shoham Grunblat, Roi Reichart

AffiliationsTechnion – Israel Institute of Technology

November 20, 2026 2 min read
Watch on YouTube
The one-line take

LightTAR adds a small set of target-aware features to standard embeddings, helping tabular models make better predictions with little extra runtime.

Key results

19
MulTaBench text dataset wins

LightTAR outperforms the PCA baseline on 19 of 20 text datasets.

17
MulTaBench image dataset wins

LightTAR outperforms the PCA baseline on 17 of 20 image datasets.

1.72
Mean text regression gain

Improvement in R² metric points on MulTaBench.

0.66%
Representation-construction overhead

Additional construction time compared with the frozen PCA baseline.

5.9%
Text end-to-end runtime increase

Median runtime increase including downstream learner fitting and prediction.

What the paper found

LightTAR improves multimodal tabular learning by making embedding compression sensitive to the prediction target. Instead of relying only on PCA, which preserves input variance but can discard label-relevant signals, it keeps a 30-component PCA anchor and appends five target-aware features. These features come from regularized partial least squares over the final and intermediate layers of frozen encoders, using e5-small-v2 for text and DINOv3 ViT-S/16 for images; no encoder fine-tuning or gradient steps are required. Across 21 tabular learners—including CatBoost, TabPFN, and NVIDIA’s Kumo models—LightTAR improves results on MulTaBench: it wins on 19 of 20 text datasets and 17 of 20 image datasets, with mean text-regression improvement of 1.72 R² points. Building the extra representation adds just 0.66% to representation-construction time, while end-to-end runtime rises by a median 5.9% for text and 2.7% for images. On STRABLE, where the usefulness of text is less predictable, results are mixed; a cross-validation gate can choose between LightTAR and standard PCA. The central contribution is a lightweight, drop-in way to add task-relevant information without changing the encoder or downstream learner.

Original abstract

Multimodal tabular learning involves classification and regression on tables combining structured numerical and categorical features with unstructured text and images. These unstructured fields are commonly represented by pretrained encoder embeddings reduced with principal component analysis (PCA). This reduction prioritizes input variance but may overlook information relevant to predicting the target. We introduce LightTAR, a simple, inexpensive method that adapts dimensionality reduction to the prediction task. LightTAR augments the standard PCA representation with a few target-aware features extracted jointly from the final and an intermediate encoder layer using regularized partial least squares (PLS). We evaluate LightTAR with 21 tabular models, including tree ensembles, neural networks, and tabular foundation models. LightTAR improves average predictive performance for all models and, averaged across models, improves prediction on almost every MulTaBench dataset. These gains come with a small increase in total runtime relative to PCA representations. LightTAR integrates directly into existing encoding pipelines, improving tabular learners' predictive performance without modifying the encoder or the learner.

Read the original paper

More in Multimodal AI

Browse all 79 papers →
01Multimodal

ALIVE: Making Streaming Video Models Interactive

Jinzhao Li, Yuan Xu, Jinxuan Wu, Yukun Wang, Yijia Lei, Hanyu Ji, Yinuo Chen, Zining Huang, Chi Zhang, Hang Ye, Yiwu Zhong, Honglei Yan, Panwang Pan, Yizhou Wang, Yin Li, Miao Liu

ALIVE aims to make video-streaming AI an interactive partner that can monitor events, coordinate tasks, and ask for feedback.

Read analysis