Time-Series Foundation Model Embeddings for Remaining Useful Life Estimation
AuthorsAmir El-Ghoussani, Michele De Vita, Ronald Naumann, Valiseios Belagiannis
Resources
This paper shows that a frozen time-series foundation model can provide strong, data-efficient features for predicting when industrial equipment will fail.
Key results
Best result on Device A with Chronos-2 features and a 2-layer MLP head
Best result on Device A with Chronos-2 features and a 2-layer MLP head
Best result on Device B with Chronos-2 features and a 2-layer MLP head
Best result on Device B with Chronos-2 features and a 2-layer MLP head
Lookback length where Chronos-2 performance improves sharply before saturation
Ablation showing a linear head on frozen Chronos-2 embeddings on Device A
What the paper found
This paper from Friedrich-Alexander-Universität Erlangen–Nürnberg and Nokia Solutions and Networks shows that a frozen time-series foundation model can become a strong Remaining Useful Life estimator with minimal task-specific training. The authors feed resampled multivariate industrial sensor windows into Chronos-2, extract context embeddings, and train only a lightweight MLP regression head, rather than fine-tuning the backbone. On two real-world Nokia device datasets, the approach outperforms linear regression, random forests, gradient boosting, GRU, LSTM, Temporal CNN, and Transformer baselines under the same preprocessing and chronological split protocol. For 5-step windows, it reaches MAE 44 and MSE 6513 on Device A, and MAE 64 and MSE 7212 on Device B, while the strongest non-TSFM baseline remains at MAE 88 on Device A. The study also shows that context length is critical: increasing the Chronos-2 lookback to 80 steps sharply improves accuracy, reducing MAE by about 2× versus a 5-step context before saturating. An ablation confirms that the pretrained embeddings carry most of the signal: a linear head still achieves MAE 60 on Device A, while the default 2-layer MLP is best at MAE 44, with little benefit from a deeper 4-layer head. The result is a data-efficient RUL pipeline that turns a forecasting foundation model into an industrial prognostics feature extractor.
Original abstract
Remaining Useful Life (RUL) prediction is essential for industrial predictive maintenance, yet many learning-based approaches rely on extensive feature engineering or large labeled datasets to train task-specific sequence models. In this work, we introduce a lightweight learning approach, in which we leverage a frozen pretrained time-series foundation model (TSFM) and combine it with a small regression head for RUL estimation from multivariate sensor streams. More specifically, we use Chronos-2 as a frozen backbone to extract context window features and train a lightweight regression neural network for RUL prediction. Experiments on real-world industrial sensor data from two device types show that Chronos-2 features consistently improve over recurrent, convolutional, Transformer-based, and gradient-boosting baselines under the same preprocessing and evaluation protocol. We further analyze the impact of context length and find that performance improves significantly with longer histories, indicating that TSFM representation offer a practical and data-efficient alternative for RUL estimation in industrial settings.
Read the original paperMore in Foundation Models
Browse all 47 papers →How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text
Jenna Russell, Ben Glickenhaus, Katherine Thai, John Wieting, Mohit Iyyer, Max Spero, Bradley Emi
AI-generated web text can help language models at first, but beyond a tipping point it degrades performance on human writing, making data filtering and separate evaluation increasingly important.
TabFM: A Zero-Shot Foundation Model for Tabular Data
Weihao Kong, Erez Louidor Ilan, Shuxin Nie, Taman Narayan, Rajat Sen, Yichen Zhou, Deqing Fu, Samet Oymak, Abhimanyu Das
TabFM is a large synthetic-data-trained model that aims to make accurate tabular predictions instantly, without retraining for each new dataset.
When Do Biological Reasoning Models Use Their Biological Inputs?
Ada Fang, Nikitha Thoduguli, Lukas Fesser, Hanlin Zhang, Sham M. Kakade, Marinka Zitnik
The study finds that many biological reasoning systems appear to succeed without meaningfully using the biological inputs they were designed to reason over.