The Well: 15TB of Physics Simulation Datasets for ML

By Prahlad Menon Published 2026-07-19 3 min read

Training neural networks to predict physics simulations is hard — not because the models are tricky, but because getting good training data is a nightmare. You either generate it yourself (expensive compute, domain expertise required) or scrape together small academic datasets with inconsistent formats.

The Well from PolymathicAI solves this: 15TB of curated physics simulations across 16 domains, with a clean PyTorch API and pretrained baselines to beat.

Who Should Use This

ML researchers building neural PDE solvers or surrogate models
Physics-informed ML practitioners who need diverse training data
Anyone benchmarking spatiotemporal prediction architectures (FNO, U-Net, transformers on physics)

If you’re trying to train a model that predicts “what happens next” in a physical system — fluid flow, wave propagation, active matter dynamics — this is your dataset.

Quick Start: Train a Model in 10 Minutes

Install and download a small dataset:

pip install the_well[benchmark]
the-well-download --base-path ./data --dataset active_matter --split train

Train an FNO baseline:

cd the_well/benchmark
python train.py experiment=fno server=local data=active_matter

That’s it. You now have a trained Fourier Neural Operator on active matter dynamics. Compare your architecture against the pretrained baselines:

from the_well.benchmark.models import FNO
baseline = FNO.from_pretrained("polymathic-ai/FNO-active_matter")

Streaming Without Downloading Terabytes

Don’t want to download 15TB? Stream directly from Hugging Face:

from the_well.data import WellDataset
from torch.utils.data import DataLoader

# Streams from HF hub — no local download
trainset = WellDataset(
    well_base_path="hf://datasets/polymathic-ai/",
    well_dataset_name="active_matter",
    well_split_name="train",
)

for batch in DataLoader(trainset):
    inputs, targets = batch["input_fields"], batch["output_fields"]
    # inputs: previous timesteps
    # targets: next timestep to predict
    loss = your_model(inputs, targets)

This is perfect for prototyping — test your architecture on real physics data without waiting for downloads.

What’s Actually in the Data

Each sample is a spatiotemporal trajectory. You get:

Input fields: Previous N timesteps of the simulation (velocity, density, pressure, etc.)
Output fields: The next timestep(s) to predict
Metadata: Physical parameters, boundary conditions, grid resolution

The 16 datasets cover:

Domain	Example Use Case
`active_matter`	Predicting cellular/bacterial dynamics
`turbulence_*`	Fluid flow prediction, CFD surrogates
`acoustic_scattering`	Wave propagation, sonar simulation
`mhd_*`	Plasma physics, astrophysical simulations
`supernova`	Explosive dynamics, astrophysics

Dataset sizes range from 6.9GB to 5.1TB. Start with smaller ones like active_matter for experimentation.

Practical Workflow: Building a CFD Surrogate

Say you want to replace expensive CFD simulations with a neural network:

Download turbulence data:

the-well-download --base-path ./data --dataset turbulence_gravity_cooling --split train

Train your architecture:

from the_well.data import WellDataset

train_data = WellDataset(
    well_base_path="./data",
    well_dataset_name="turbulence_gravity_cooling",
    well_split_name="train"
)

# Your training loop here
for batch in DataLoader(train_data, batch_size=4):
    pred = model(batch["input_fields"])
    loss = mse_loss(pred, batch["output_fields"])

Benchmark against baselines:

from the_well.benchmark.models import FNO
baseline = FNO.from_pretrained("polymathic-ai/FNO-turbulence_gravity_cooling")
# Compare your model's MSE against baseline

Deploy: Your trained model now predicts simulation steps 1000x faster than running the actual CFD solver.

When NOT to Use This

If you need specific physics: The Well covers 16 domains, but if you need, say, semiconductor device physics, you’ll need domain-specific data.
If you need real-world data: This is all synthetic simulation data. Great for learning physics dynamics, but won’t help with sensor noise or real measurement artifacts.
If you’re compute-limited: Even streaming, training on 5TB datasets requires serious GPU resources.