Scientific Foundation Models for Chemistry, Materials, and Biology in 2026
The playbook that produced GPT-class language models — scrape a near-infinite corpus, pretrain one big network, adapt it to everything — does not transfer cleanly to science. Scientific foundation models borrow the recipe but hit a wall the internet never imposed on text: the data is scarce, expensive to generate, and answerable to physics. A protein sequence that violates thermodynamics is not a creative flourish; it is wrong. A crystal the model calls stable either survives synthesis or it does not. This is why the most important question about any science foundation model is not “how many parameters?” but “where did the training data come from, and does it cover the region you care about?”
That single question reorganizes the whole field. It explains why materials labs pour DFT simulations into training sets, why biology teams guard proteomic databases, and why the frontier is shifting toward models fed by autonomous labs that manufacture their own data.
What this covers: what a scientific foundation model actually is, the architectures that make each scientific modality tractable, where the pretraining data originates, how these systems are evaluated, and the specific ways they fail.
Context and Background
For most of the last decade, machine learning in the physical sciences meant bespoke models: one graph network for band-gap prediction, another for solubility, a third for binding affinity. Each was trained from scratch on a few thousand labeled examples, and each was brittle outside its narrow training distribution. The foundation-model thesis flips that pattern. You pretrain one large network, mostly with self-supervised objectives that need no human labels, on a broad corpus of molecules, crystals, or sequences. Then you adapt that single base to dozens of downstream tasks with comparatively little labeled data.
The proof points arrived fast. In biology, Meta’s ESM-2 protein language models — spanning roughly 8 million to 15 billion parameters, trained on the UniRef sequence database — learned enough about protein grammar that ESMFold could predict structure directly from sequence without the multiple-sequence-alignment step AlphaFold2 relies on. In materials, universal machine-learned interatomic potentials such as MACE-MP, MatterSim, and Orb learned to approximate quantum-mechanical energies across most of the periodic table from a single pretraining run on density functional theory (DFT) data. These are the same idea wearing different clothes: pretrain broadly, adapt narrowly.
What makes 2026 the interesting moment is the collision of two trends. Model architectures matured — equivariant networks and long-context sequence models are now standard, not exotic. And the community finally admitted the bottleneck is data provenance, not model capacity. That admission connects science foundation models directly to the architecture of closed-loop self-driving labs, which exist precisely to generate the experimental data these models are starving for. For a broader survey of the field’s data and tooling landscape, the 2025 arXiv review “A Survey of AI for Materials Science” is a useful map.
What a Scientific Foundation Model Actually Is
A scientific foundation model is a large network pretrained — mostly self-supervised — on broad scientific data such as molecular graphs, crystal structures, protein or DNA sequences, spectra, and simulation outputs, then adapted to many downstream tasks. The defining trait is not scale for its own sake. It is transfer: representations learned during pretraining carry information that helps tasks the model never saw labels for.
Figure 1: A taxonomy of scientific foundation models organized by scientific domain and representative model class.
Figure 1 groups the field by domain. Biology splits into protein language models (ESM-2, ESM3), structure predictors (the AlphaFold family), and genomic foundation models such as Evo 2. Chemistry spans SMILES-string transformers of the MolFormer class, message-passing graph neural networks over molecular graphs, and reaction models. Materials divides into foundation potentials (MACE-MP, MatterSim) and generative models (Microsoft’s MatterGen, DeepMind’s GNoME). The taxonomy matters because each branch made different architectural bets, and those bets follow directly from how each modality’s data is shaped.
Pretraining objectives do the heavy lifting
The reason these models generalize is the self-supervised objective. Protein and DNA language models use masked-token prediction: hide amino acids or nucleotides and force the network to reconstruct them from context. To reconstruct a masked residue well, the network has to internalize co-evolutionary constraints — which mutations tend to occur together because they preserve a fold or a function. That learned grammar is what transfers.
Interatomic potentials use a different but analogous trick. They are trained to regress the energy and forces of an atomic configuration against a DFT reference. Because energy is a physical quantity that constrains every downstream property, a model that predicts energy and forces accurately across millions of configurations has implicitly learned a transferable surrogate for the quantum-mechanical potential-energy surface. Generative models like MatterGen instead learn a denoising objective, starting from noise and iteratively refining atomic positions and types into a stable structure.
Adaptation is where value is captured
Pretraining builds the base; adaptation captures the value. In practice teams use three adaptation modes. Linear probing freezes the base and trains a small head on top of its embeddings — cheap, fast, and surprisingly strong when the pretraining distribution overlaps the task. Fine-tuning updates the base weights on task data, which is more powerful but risks catastrophic forgetting and overfitting on small labeled sets. In-context or zero-shot use skips training entirely — you query the model’s learned likelihoods directly, as when a protein language model scores the pathogenicity of a mutation by how surprising it finds the mutated sequence.
The moat is experimental data, not weights
Here is the uncomfortable truth for anyone who assumes model weights are the asset. Most leading science foundation models are open or semi-open — ESM-2, MACE-MP, and MatterGen all have public code and weights. What is not open is proprietary experimental data: the assay results, the synthesis outcomes, the failed reactions. That is the durable moat, and it is why the roadmap of every serious lab now routes through the architecture of AI hypothesis-generation systems and automated experimentation. Weights commoditize; data provenance compounds.
From Raw Data to Downstream Task: The Pretraining Pipeline
The mechanics of building one of these models follow a consistent pipeline regardless of domain, even though the specifics of each stage change dramatically. Understanding the pipeline is the fastest way to reason about where a given model is strong and where it will quietly fail.
Figure 2: The pretraining-to-downstream pipeline shared across scientific foundation models.
Figure 2 lays out the five stages. Raw scientific data is tokenized or represented in a form the network can consume. A self-supervised pretraining objective — masking, denoising, or contrastive learning — turns that data into a base foundation model. The base is then adapted, by fine-tuning or probing, into task-specific heads: property prediction, structure prediction, or generative design. Every design decision in this pipeline is a trade-off between fidelity to the underlying physics and the scale of data you can afford to process.
Tokenization and representation set the ceiling
The representation stage quietly determines the model’s ceiling. Protein language models tokenize each amino acid as a discrete symbol — a 20-letter alphabet plus special tokens — which makes the transformer architecture from natural language directly applicable. Genomic models like Evo 2 push this to single-nucleotide resolution over context windows reaching into the hundreds of thousands of bases, which demands architectures with near-linear scaling in sequence length rather than the quadratic cost of vanilla attention.
Chemistry has a representational fork. You can serialize a molecule as a SMILES string and treat it as a language, which is cheap and lets you reuse transformer tooling but throws away explicit 3D geometry. Or you can represent the molecule as a graph of atoms and bonds and use message passing, which preserves connectivity but not necessarily spatial coordinates. Materials go further still, representing a crystal as a periodic 3D point cloud where absolute position, rotation, and the choice of unit cell should not change the predicted energy.
Self-supervision converts scale into structure
The pretraining objective is the engine that converts raw scale into useful structure. For sequences, masked-language modeling is dominant and well understood. For 3D structures and materials, denoising diffusion has become the objective of choice for generative tasks because it naturally handles continuous coordinates and can be conditioned on target properties after the fact.
There is a subtle but important point about compute here. Pretraining a science foundation model is expensive, but the expense is dominated by data generation, not gradient steps. A materials potential trained on tens of millions of DFT-labeled configurations required an enormous upstream investment in quantum-chemistry simulations. The neural network training is almost an afterthought by comparison. This inverts the economics people import from language modeling, where data is cheap and compute is the constraint.
Adaptation cost is where projects live or die
In production, the adaptation stage is where most projects succeed or fail. A team with 500 labeled binding-affinity measurements cannot fine-tune a 15-billion-parameter protein model without overfitting. The pragmatic path is to freeze the base, extract embeddings, and train a small regressor — accepting that the frozen representation may not be optimal for their exact target. Knowing when to probe versus fine-tune is the single most valuable piece of practitioner judgment in this whole workflow, and it depends almost entirely on how much labeled data you hold and how far your task sits from the pretraining distribution.
Where the Training Data Comes From
If representation sets the ceiling, data provenance sets everything else. The central tension of scientific foundation models is that, unlike text, their fuel is scarce, costly, and physically grounded. There are three sources, and the balance among them is shifting.
Figure 3: Data provenance flow, from public databases and simulation through the training corpus and back to the lab.
Figure 3 shows the three inflows converging on the training corpus and the closed loop that is emerging. Public databases contribute curated, human-generated records. Simulation contributes synthetic-but-physical data at scale. Autonomous labs contribute fresh experimental measurements — and, critically, the model’s own designs flow back into those labs to generate the next round of data. That loop is the structural reason self-driving labs and foundation models are converging.
Public databases: the foundation and its limits
The public databases are the bedrock. Biology draws on the Protein Data Bank (PDB) for experimentally solved structures and UniProt for hundreds of millions of protein sequences. Chemistry leans on PubChem for compound records. Materials science depends on the Materials Project and the Open Quantum Materials Database (OQMD), each holding hundreds of thousands of DFT-computed crystal properties. These databases are extraordinary public goods, but they carry a hidden bias: they record what humans chose to study or synthesize. The chemical and structural space they cover is a tiny, non-random slice of what is possible, skewed toward molecules and materials that were interesting, fundable, or easy to make.
Simulation data: scaling past the human record
To escape the human-selection bias, teams generate their own data with simulation. DFT for electronic-structure and energy calculations, and molecular dynamics (MD) for conformational sampling, can produce labeled configurations that no experiment ever measured. This is how universal interatomic potentials achieve their coverage: MACE-MP was trained on Materials Project DFT data, and later variants extended coverage with additional large DFT datasets. Simulation is the workhorse that lets materials models blanket the periodic table. The catch, discussed below, is that the model inherits every systematic error of the reference method. A potential trained on a particular DFT functional cannot be more accurate than that functional; it can only reproduce it faster.
Autonomous-lab data: the emerging frontier
The frontier source is experimental data generated on demand by autonomous labs. Instead of waiting for the literature to report a measurement, a closed-loop system proposes candidates with the model, synthesizes and characterizes them robotically, and feeds the results back into the training corpus. This is the “experimental data moat” in action: fresh, targeted, in-distribution measurements in exactly the region the model is uncertain about. It is expensive per data point, but every point is chosen to reduce the model’s uncertainty where it matters, which makes it far more valuable than a random record scraped from a database. This active-learning loop is the reason materials and chemistry foundation models are increasingly discussed in the same breath as robotic experimentation — a theme explored in depth in our treatment of geometric deep learning for materials discovery with GNoME and MatterGen.
Architecture and Representation Choices by Modality
The architectures that dominate each domain are not arbitrary. They are the ones whose built-in assumptions — their inductive biases — match the structure of the data. Getting the inductive bias right is what lets a model generalize from limited data instead of memorizing it.
Figure 4: Representation and architecture choices per modality, from input type through encoder and equivariant layers to a shared embedding.
Figure 4 traces the three dominant paths. A sequence input — protein or DNA — flows into a transformer encoder. A molecular graph flows into a message-passing GNN. A 3D point cloud of atoms flows into an equivariant network with explicit E(3)-equivariant layers that encode symmetry priors. All three can converge on a shared embedding feeding a task head. The differences among the encoders are the whole game.
Equivariance is the defining prior for 3D science
The most important architectural idea in physical-science modeling is equivariance. A physical property of a molecule or crystal does not change if you rotate, translate, or reflect it in space. A naive network would have to learn this invariance from data, wasting capacity and data budget. An E(3)-equivariant network builds it in: rotate the input and the internal features rotate in lockstep, so predicted energies stay invariant and predicted forces rotate correctly by construction. This is the mathematical backbone of modern interatomic potentials like MACE, which combines equivariant message passing with higher-order tensor products to capture many-body interactions. The payoff is dramatic data efficiency — the model never wastes examples relearning geometry it could have been told.
Sequence models trade geometry for context length
Protein and genomic models make the opposite bet. They largely discard explicit 3D geometry and treat biology as language, betting that enough sequence data lets the network recover structural and functional constraints implicitly. ESM-2 demonstrated this works remarkably well for proteins. Genomic models like Evo 2 — reported at 40 billion parameters trained on roughly 9 trillion nucleotides across the tree of life — extend the bet to DNA, RNA, and protein-coding regions simultaneously, using architectures designed for extremely long context so a single model can reason across a whole gene or operon. The trade is deliberate: you give up hard geometric guarantees to buy scale and context.
Graphs sit in the middle
Molecular GNNs occupy the middle ground. A message-passing network over an atom-bond graph respects chemical connectivity and permutation symmetry — reorder the atoms and the prediction is unchanged — without necessarily committing to full 3D equivariance. This is often the right level of abstraction for property prediction where 2D topology carries most of the signal, and it keeps compute manageable relative to full 3D equivariant models. The choice among sequence, graph, and point-cloud representations is ultimately a bet about which symmetries your task actually needs enforced.
How These Models Are Evaluated
Evaluation is where scientific foundation models earn or lose trust, and it is harder than it looks because the failure modes are subtle. The community has converged on a set of benchmark suites, but the honest reading of any leaderboard requires knowing what the numbers do and do not certify.
For materials, Matbench and Matbench Discovery are the reference suites. Matbench Discovery specifically evaluates whether a model can predict the thermodynamic stability of hypothetical crystals — a task tied directly to real discovery workflows — and reports metrics such as the F1 score for stable-versus-unstable classification after structural relaxation. Universal interatomic potentials currently dominate its leaderboard. For molecules, MoleculeNet remains a widely cited collection of property-prediction tasks. For protein structure, the biennial CASP experiment has been the gold-standard blind assessment for decades, precisely because it tests on targets no model could have seen.
A results table for a materials stability task typically looks like the structure below. The numbers here are illustrative placeholders to show the shape of a comparison, not measured results — always pull live figures from the public leaderboard before quoting any score.
| Model class | Task | Metric | Illustrative value |
|---|---|---|---|
| Universal potential | Crystal stability | F1 (higher better) | strong |
| Generative diffusion | Novel stable rate | percent novel and stable | moderate |
| Graph predictor | Formation energy | MAE (lower better) | low error |
The methodological caveat that governs all of this: a benchmark score certifies performance on the benchmark’s distribution, and nothing more. CASP’s blind design partly guards against leakage; most other suites do not, which brings us to the failure modes.
Task-level metrics hide the questions that matter
A single aggregate score compresses away the information a practitioner actually needs. For an interatomic potential, energy mean-absolute-error is the headline, but force error, stress error, and the smoothness of the potential-energy surface during a dynamics run matter far more for whether the model can drive a stable simulation. A model can post a respectable energy MAE and still produce forces jagged enough to make a molecular-dynamics trajectory explode. The lesson is to read the sub-metrics, not the leaderboard rank, and to weight them by what your downstream use actually stresses.
The same holds in biology. A protein language model’s masked-token accuracy is only loosely correlated with how well its zero-shot likelihoods predict the effect of a mutation, which is the property most variant-effect applications actually depend on. When a benchmark and a use case diverge, the benchmark loses. Always trace the chain from the reported metric to the decision you plan to make with the model, and be suspicious wherever that chain has a weak link.
Scaling behavior is real but bounded
One reason the foundation-model framing took hold in science is that these models show scaling improvements: larger protein language models and larger potentials generally do better, up to a point. But the scaling story is bounded by data in a way language modeling is not. You cannot conjure more experimentally solved protein structures or more DFT-labeled configurations by spending on compute alone; someone has to generate them. This is the deepest reason the field’s center of gravity is moving from architecture to data acquisition, and why the labs that win will be the ones that industrialize data generation rather than the ones with the biggest networks.
Trade-offs, Gotchas, and What Goes Wrong
The gap between a strong benchmark number and a useful scientific tool is where most disappointment lives. These are the specific ways scientific foundation models fail in practice, and every one of them has bitten real projects.
Out-of-distribution collapse. The single most common failure is confident nonsense on inputs unlike the training data. A potential trained mostly on near-equilibrium crystal structures can produce garbage forces for the highly distorted geometries that appear during a reaction or a phase transition — exactly the regime you often care about. The model does not know it is extrapolating; it simply returns a number.
Data leakage and contamination. Because public databases overlap and models are trained on scrapes of them, it is easy for test examples to leak into training. A protein or crystal that appears in both sets inflates the reported score without reflecting any real generalization. This is why blind, time-split, or held-out-scaffold evaluations matter far more than random splits, and why a headline number from a random split should be treated with suspicion.
Physically invalid but plausible outputs. Generative models hallucinate. A molecule generator can emit a structure with impossible valences; a crystal generator can propose a composition that no chemistry allows. The output looks reasonable to the network because nothing in the loss forced hard physical constraints. Downstream validity filters and physics checks are mandatory, not optional, and they routinely reject a large fraction of generated candidates.
Inherited reference error. A potential trained on DFT can only be as accurate as the DFT functional it learned from. If that functional systematically misestimates a class of interactions, the model reproduces the error faithfully and quickly. Speed does not fix accuracy; it scales it. Teams sometimes forget that a fast surrogate of a flawed reference is still flawed.
The wet-lab validation cliff. Ultimately every prediction that matters must survive the bench. Synthesis and characterization are slow and expensive, so the throughput of computational proposals vastly exceeds the throughput of experimental validation. The bottleneck is not generating candidates; it is confirming them. This is the economic reality that makes the data-generation loop, not the model, the strategic asset.
Reproducibility drift. Different DFT settings, dataset versions, and evaluation splits make cross-paper comparison treacherous. Two models reporting the “same” benchmark may have used different reference data or preprocessing, rendering the comparison meaningless. Pin your data versions and evaluation protocol, and distrust any comparison that does not.
Uncertainty that is not calibrated. Most of these models emit a point prediction with no honest confidence attached. When they do provide an uncertainty estimate — through ensembles or evidential heads — it is frequently poorly calibrated, overconfident precisely in the out-of-distribution regions where you most need a warning. An active-learning loop that trusts a miscalibrated uncertainty will keep sampling the wrong candidates and slowly poison its own training set. Calibration is not a nicety here; it is what makes the closed loop converge rather than drift.
Anthropomorphized reasoning. It is tempting to describe a generative model as “designing” a molecule or “understanding” a fold. It is doing neither. It is sampling from a distribution shaped by its training data, and it has no notion of the synthetic accessibility, cost, or safety of what it proposes. Treating fluent outputs as reasoned choices is how teams end up with beautiful candidates that cannot be made in any real lab. The model proposes a distribution; judgment about feasibility stays with the scientist.
Practical Recommendations
For teams deciding how to deploy scientific foundation models, a few principles separate the projects that ship from the ones that stall. Start by interrogating provenance before capability. Ask what the model was pretrained on and whether that distribution overlaps your problem; a smaller model trained on relevant data usually beats a larger one trained elsewhere. Treat the pretraining corpus as the primary spec.
Match adaptation to your data budget honestly. If you hold only hundreds of labels, freeze the base and probe; reserve fine-tuning for when you have thousands of in-distribution examples and can afford the overfitting risk. Never fine-tune a giant model on a tiny set and trust the result.
Build validity checks into every generative pipeline. Assume a meaningful fraction of generated candidates are physically invalid and filter them with explicit rules and cheap physics before anything reaches a queue. And design for the loop: the highest-leverage investment is usually not a bigger model but a faster path from prediction to experiment to fresh training data.
A short field checklist:
- Audit the pretraining data distribution against your target region before adopting a model.
- Prefer probing over fine-tuning when labeled data is under a few thousand examples.
- Add hard physical-validity filters downstream of every generator.
- Use time-split or scaffold-split evaluation, never random splits, to detect leakage.
- Version-pin datasets, reference methods, and eval protocols for reproducibility.
- Route model outputs into an experimental loop to build a proprietary data moat.
Non-advisory note: this article analyzes systems, architectures, and methods for scientific foundation models. It is not scientific, clinical, or investment advice, and model or benchmark specifics change quickly — verify current figures against primary sources before relying on them.
Frequently Asked Questions
What is a scientific foundation model?
A scientific foundation model is a large neural network pretrained, mostly with self-supervised objectives, on broad scientific data such as protein or DNA sequences, molecular graphs, crystal structures, and simulation outputs, then adapted to many downstream tasks. Instead of training a separate model per task, you build one base whose learned representations transfer to problems it never saw labels for. The defining property is transfer, not raw scale, and its usefulness depends heavily on whether the pretraining data overlaps your target problem.
How is scientific data different from the text used to train language models?
Text is abundant, cheap, and forgiving; scientific data is scarce, expensive, and physically grounded. A generated sentence can be creatively wrong and still valuable, but a generated protein or crystal that violates physics is simply invalid. This scarcity flips the economics of pretraining: the dominant cost is generating labeled data through simulation or experiment, not running gradient steps. It is also why data provenance, rather than parameter count, is the most important question to ask about any science foundation model.
Which foundation models matter for biology, chemistry, and materials?
In biology, protein language models like ESM-2 and ESM3, the AlphaFold structure family, and genomic models such as Evo 2 are central. Chemistry uses SMILES-string transformers of the MolFormer class and message-passing graph networks. Materials rely on universal interatomic potentials like MACE-MP, MatterSim, and Orb, plus generative models such as Microsoft’s MatterGen and DeepMind’s GNoME. Model names and versions change quickly, so confirm the current release and any quoted specification against primary sources before relying on it.
Why does equivariance matter for materials and molecular models?
A molecule’s or crystal’s physical properties do n
