How TabICL and TabPFN Handle Missing Values
TabICL and TabPFN, two leading tabular foundation models, both accept datasets with missing entries, but they use distinct internal mechanisms. TabICL applies a simple preprocessing step that creates a missing‑value category for categoricals and fills numeric gaps with the column mean. TabPFN, by contrast, encodes NaNs with a binary indicator and sentinel values, concatenating this missingness channel to the feature vector before embedding. The differing approaches affect bias, interpretability, and suitability for complex missing‑data patterns.
Tabular ML Is Entering a New Benchmark Era
Tabular machine learning is moving from static paper‑based benchmarks to live, capability‑oriented leaderboards such as TabArena, ScoringBench, and MulTaBench. The rise of tabular foundation models like TabICL and TabPFN‑3.0 enables a single pre‑trained model to be evaluated across tasks—classification, regression,...
Time Series Forecasting with Tabular Foundation Models
The post demonstrates how tabular foundation models (TFMs) can be repurposed for time‑series forecasting by treating the problem as a regression task. By automatically enriching a simple timestamp‑target table with temporal features, models like TabICL and TabPFN‑TS generate one‑shot forecasts....
Context Is the New Training
Tabular foundation models such as TabPFN and TabICL replace classic training with in‑context learning, where the so‑called training data becomes context supplied at prediction time. Because model weights remain static after pre‑training, only the context data changes the embeddings and...
Regression Should Predict Full Distributions
The post argues that regression models should output full predictive distributions rather than single-point estimates. It highlights tabular foundation models such as TabPFN and TabICL, which internally generate discretized distributions and can return quantiles with a simple parameter change, without...
TabArena Explained
TabArena is a living benchmark for tabular machine‑learning models hosted on HuggingFace, featuring a strict preprocessing and evaluation protocol. It evaluates 51 curated tasks—13 regression and 38 classification datasets—using an Elo rating system to compare algorithms pairwise. Recent Prior Labs’...
The Interpretability Tax on Tabular Foundation Models
The post examines how classic model‑agnostic interpretability tools, such as permutation feature importance (PFI) and LOCO, operate on tabular foundation models (TFMs). While these methods function out‑of‑the‑box, TFMs flip the traditional cost balance: training is cheap but inference is expensive,...
The Random Forest of the 2030s?
Tabular foundation models (TFMs) are transformer‑based systems that perform in‑context learning on combined training and test data without parameter updates. The author outlines three adoption scenarios: as another algorithm (Level 1), as the go‑to quick‑and‑dirty baseline replacing Random Forests (Level 2), and...