Tabular data is the stubborn holdout in the deep-learning story. Text, images, audio, video — foundation models have convincingly taken over all of them. Tabular data, the structured rows and columns behind virtually every business database, has not. Gradient-boosted trees (XGBoost, LightGBM, CatBoost) keep winning the Kaggle competitions and the production benchmarks. The intuitions for why have been solid: local feature interactions, categorical variables, small dataset sizes, and massive sensitivity to irrelevant features. Every year or two, a deep learning approach claims to beat them; practitioners point out the comparison was unfair; gradient boosting holds its ground.

Google Research’s TabFM, published June 30, is the most technically interesting attempt yet to close that gap, and its approach is different enough from prior work to be worth unpacking.

The architecture

The core idea is in-context learning — the same paradigm that makes LLMs useful without fine-tuning. Given a training set and a query row at inference time, the model predicts the target in a single forward pass, no task-specific fine-tuning and no hyperparameter search. The architecture has three stages:

First, alternating attention across rows and columns builds representations that capture feature interactions. Most tabular deep learning approaches process columns independently or flatten them into a sequence; the alternating attention lets the model build richer cross-feature context.

Second, row compression collapses each training example’s contextualized representation into a single dense vector. This is the critical step that makes the whole thing tractable: without it, the ICL context for a 1,000-row training set with 50 features would be enormous.

Third, a Transformer ICL layer applies attention over those compressed row vectors — this is where the model actually generalizes from training examples to the query. The input length here is proportional to the number of training examples, not their dimensionality, which is what makes the approach scale.

The training data problem

The hard part of any tabular foundation model is training data. Unlike natural language, there is no web-scale corpus of tables with consistent structure and semantics. Every industrial dataset has its own schema, encoding conventions, and domain-specific feature relationships.

Google’s solution: generate hundreds of millions of synthetic datasets from structural causal models, with random functions drawn from diverse families. The idea is that if you cover enough of the distribution of possible table structures, a model trained on this synthetic distribution generalizes to real tables at evaluation time. This is similar in spirit to how physics simulators have been used to pre-train robotics models, or how synthetic text data has been used to train reasoning models: if the synthetic distribution is rich enough, real data becomes just another sample from it.

The benchmark

TabArena covers 51 datasets — 38 classification, 13 regression — and TabFM beats gradient-boosted trees on it. Two configurations are reported: the basic TabFM (single forward pass, zero tuning) and TabFM-Ensemble (32-way ensemble with cross features, SVD features, and Platt scaling). The ensemble numbers are stronger, as you’d expect.

The standard caveat for tabular benchmarks applies: XGBoost has years of accumulated hyperparameter search folklore tuned against public benchmarks. A comparison that matched compute budget — the same time spent tuning XGBoost as running TabFM — would be more meaningful than a zero-tuning comparison. The paper presumably addresses this, and the TabArena results are available on their GitHub for inspection.

Why this matters

The practical case for a zero-shot tabular foundation model is different from the academic case. In production, most tabular ML tasks never get a dedicated data scientist. They are one-off analyses, internal dashboards, quarterly reporting pipelines — good enough with a simple model, not worth a tuning sprint. If TabFM can match or exceed gradient boosting without any setup, it covers exactly that long tail. The BigQuery integration is the obvious delivery vehicle: analysts who already have their data in BigQuery and have never opened a Jupyter notebook.

The question the field has been asking for years is whether the foundation model paradigm, which relies on learning a universal prior from massive pre-training, can actually work for structured data with the wrong shape and wrong semantics at every table. TabFM’s synthetic pre-training approach is a credible answer to that question. Whether the numbers hold up when the community tries it on their private datasets is what gets settled over the next few months.