This article is a systems and research explainer for engineers. It is not operational, safety, or engineering guidance for any real reactor.

AI Plasma Control for Tokamak Fusion: How Reinforcement Learning Steers the Plasma

A tokamak holds a ring of plasma at over a hundred million degrees inside a magnetic bottle, and that bottle is never stable on its own. The plasma wants to drift, stretch, and slam into the wall within milliseconds. Keeping it centered and correctly shaped is a real-time control problem, and for decades the answer was a hand-tuned cascade of PID loops driving the magnetic coils. AI plasma control rewrites that answer: a single reinforcement-learning policy, trained entirely in simulation, learns to command all the coils at once directly from magnetic sensors. In 2022 a DeepMind and EPFL team ran exactly such a controller on a real tokamak and held configurations that classical control struggles with. This post explains why the problem is hard, how the RL controller is actually built and deployed, and where the approach still falls short.

This is a systems and research explainer for engineers, not operational, safety, or engineering guidance for any real reactor.

What this covers: the physics of the control problem, the classical baseline, the RL formulation and training loop, sim-to-real transfer, real-time deployment, disruption avoidance, and honest limits on the road to reactor scale.

Context and Background

Tokamak plasma control has two historically separate jobs. The first is magnetic control: keeping the plasma current, position, and cross-sectional shape where you want them by driving the poloidal field coils. The second is kinetic control: managing density, temperature, and instabilities through heating, fueling, and current drive. Magnetic control is the safety-critical inner loop. If the plasma drifts into the wall or its current collapses, you get a disruption — a sudden loss of confinement that dumps megajoules into the vessel and can damage hardware. Everything else depends on the plasma first staying put.

The incumbent design, refined over decades on machines like TCV, DIII-D, ASDEX Upgrade, and JET, is a layered controller. Magnetic pickup coils and flux loops feed a real-time equilibrium reconstruction that estimates the plasma boundary. A supervisory layer computes shape and position errors against a preprogrammed reference. Nested PID controllers, plus feedforward from a precomputed scenario, convert those errors into voltage requests for each coil. It works, and it has run thousands of shots. But every new plasma shape means re-deriving observers, re-tuning gains, and validating the cascade by hand — slow, expert-intensive engineering.

The reframing that makes machine learning interesting here is simple. Magnetic control is a high-dimensional, nonlinear, unstable control problem with a well-understood physics simulator available offline. That is precisely the setting where deep reinforcement learning has beaten hand-engineered controllers elsewhere. If you want the underlying confinement physics first, see our primer on how tokamak fusion reactors work. The landmark demonstration is the DeepMind and Swiss Plasma Center paper in Nature (2022), which is the anchor for everything below.

The Reference Architecture: One Policy Instead of a Cascade

The core thesis is that plasma magnetic control can be treated as a single learned mapping from sensors to coil voltages, replacing the reconstruct-then-PID cascade with one neural network policy. The RL controller observes the same magnetic measurements a classical system uses, and outputs voltage commands for the full set of control coils at kilohertz rates. It never explicitly reconstructs the equilibrium at run time; the mapping from measurements to correct actions is baked into the network weights during training.

Reinforcement learning replaces a hand-tuned cascade of equilibrium reconstruction and nested PID loops with a single neural-network policy. The policy reads magnetic sensor signals and directly commands coil voltages at kilohertz rates. It is trained offline against a physics simulator by maximizing a reward for hitting a target plasma shape, position, and current, then deployed onto the tokamak’s real-time control system.

Figure 1: The classical path reconstructs an equilibrium, computes errors, and drives nested PID loops with feedforward; the RL path maps sensors straight to coil voltages.

Figure 1 contrasts the two control paths. In the classical path, sensor data flows through equilibrium reconstruction to a shape and position estimate, then into nested PID controllers augmented by a feedforward trajectory, producing coil voltage commands. In the learned path, the same sensors feed a single policy network that emits coil voltages directly. The learned path removes the explicit reconstruction and the manually designed control law, folding both into one function approximator.

Why tokamak magnetic control is genuinely hard

Start with instability. A tokamak plasma is vertically unstable when elongated: nudge it up and the magnetic forces push it further up. This is an open-loop unstable mode with growth times on the order of milliseconds, so the controller must actively stabilize it every cycle, not merely track a setpoint. Miss a few cycles and the plasma is gone.

Then there is dimensionality and coupling. A machine like TCV has on the order of a dozen-plus independently powered poloidal field coils, and each coil affects the whole plasma boundary at once. You are not controlling one variable with one actuator; you are controlling shape, radial and vertical position, and total plasma current simultaneously with a bank of coils whose effects overlap. The control matrix is strongly coupled and changes with the plasma state.

Diagnostics are limited and indirect. You cannot see the plasma boundary. You infer it from magnetic pickup coils and flux loops mounted around the vessel — noisy, partial measurements of the field outside the plasma. The controller must reconstruct or otherwise infer an internal state it never observes directly, under sensor noise and the occasional dead channel.

Finally there is the timescale. Magnetic control loops run in the kilohertz range, so the controller has microseconds-to-milliseconds per decision. Whatever policy you deploy has to produce an action inside that budget, every cycle, deterministically. That constraint shapes the entire deployment story later in this post.

What the learned policy actually controls

In the Nature 2022 work, a single network commands all of TCV’s control coils and is trained to realize target plasma shapes and positions from the magnetic measurements alone. The team produced not just conventional elongated shapes but harder configurations: negative triangularity, a “snowflake” divertor geometry, and even two separate plasmas — “droplets” — held in the vessel at once. Those advanced shapes are exactly the cases where hand-tuning a cascade is most painful, which is why they make the point.

The important conceptual shift is that the policy learns the coupling for free. It never gets told which coil moves the boundary which way; it discovers, over millions of simulated timesteps, the joint voltage patterns that produce the desired equilibrium. The control designer’s job moves from deriving observers and tuning gains to specifying what the plasma should look like through a reward function.

The reward is where the engineering goes

Because the policy is defined by what it optimizes, reward shaping is the real design surface. A typical reward penalizes the distance between the achieved and target plasma boundary at a set of control points, plus errors in plasma current and position, plus terms that discourage dangerous states and excessive actuator use. Getting the weighting right — shape versus current versus actuator effort — is the modern equivalent of tuning PID gains, except it is done offline and once, then reused across scenarios. The reward also encodes soft safety: pushing the plasma away from limits the designer knows to fear.

Training the Controller: A Simulator in the Loop

You cannot train a reinforcement-learning agent by trial and error on a real tokamak. Each shot lasts seconds, costs a great deal to run, and a bad exploratory action can disrupt the plasma or stress the machine. RL needs millions of episodes; the hardware can offer a handful of shots per day. The resolution is to train entirely inside a physics simulator and transfer the finished policy to hardware. The simulator is not a nicety here — it is the thing that makes the whole approach possible.

Figure 2: The agent proposes coil voltages, the free-boundary simulator evolves the plasma and returns sensor signals, the reward scores the result, and the policy updates — repeated for millions of steps.

Figure 2 shows the training loop. A target shape and position define the reward. The free-boundary equilibrium simulator takes the agent’s coil-voltage actions and advances the plasma state, emitting simulated sensor signals. Those signals become the agent’s observation; the reward scores how close the plasma is to target; and the actor-critic algorithm updates the policy. The loop runs offline at massive scale, decoupled from the seconds-long cadence of real shots.

The free-boundary equilibrium simulator

The physics engine at the center is a free-boundary equilibrium solver. It computes, at each step, the plasma equilibrium consistent with the coil currents and the plasma’s own current profile, letting the boundary move freely rather than fixing it in advance. Coupled to a circuit model of the coils and their power supplies, it evolves the whole electromagnetic system forward in time. This is the environment the agent lives in during training: it captures the coupled, unstable, multi-coil dynamics the real controller will face.

Fidelity is a deliberate trade-off. A simulator accurate enough to transfer must model the coil circuits, the vessel’s conducting structures (which carry induced eddy currents), sensor placement, and actuator delays. But it must also be fast, because you need billions of simulated steps. Training-time simulators therefore run reduced-order or well-tuned physics rather than the most expensive first-principles codes. The gap between that reduced model and reality is the central risk, addressed next. For the broader trend of learned surrogates standing in for expensive physics solvers, see our explainer on neural operators for scientific simulation.

The actor-critic architecture and why it fits

The Nature 2022 controller was trained with Maximum a posteriori Policy Optimisation (MPO), an actor-critic method. Two networks share the work. The critic learns to estimate the long-run value of states and actions; it can be large and can see privileged information, because it is only used during training. The actor is the policy that will actually be deployed; it maps observations to actions and is kept small and fast. This asymmetry is the crucial engineering choice: you train with a heavyweight critic and any extra simulator state you like, but you ship a lightweight actor that runs inside the real-time budget.

Actor-critic methods suit continuous control. Coil voltages are continuous quantities, and the plasma responds smoothly to them, so a method that outputs continuous actions and learns a value estimate to reduce variance is a natural fit — far better than discretizing voltages or using pure policy-gradient methods that would need orders of magnitude more samples. The critic’s value estimates let the actor learn efficiently from the simulator’s rich but expensive rollouts.

Reward shaping and curriculum in practice

Training rarely starts from the final target. A curriculum eases the agent in: first hold a simple stable shape, then progress to elongated and exotic configurations, then chain shape changes together within a single simulated shot. The episode structure mirrors a real experiment — an episode is a shot, from plasma formation through a shape trajectory to a controlled rampdown. The reward accumulates over that episode, so the policy learns to manage transitions, not just static setpoints. This is directly analogous to how autonomous experimental systems chain actions toward a goal, a pattern we cover in self-driving lab architecture.

Deployment: From a Trained Network to a Microsecond Control Loop

Training solves the learning problem. Deployment solves a systems problem: get a neural network to emit correct coil voltages every control cycle on hardware that tolerates no jitter. The real-time control system of a tokamak runs on a fixed clock, often in the kilohertz range, and every subsystem must produce its output inside a hard deadline. A policy that is 99% fast enough is not fast enough.

Figure 3: In deployment the trained actor network runs inference each cycle, reading magnetic signals and commanding coil voltages that reshape the plasma, closing the loop near 10 kHz.

Figure 3 traces one real-time cycle. Magnetic sensors report flux and field to the controller. The controller runs a forward pass of the actor network and emits voltage commands to the coils. The coils change the shaping magnetic field, the plasma settles into a new equilibrium, and the sensors report it back — closing the loop at roughly ten thousand times a second. The only computation on the critical path is the actor’s forward pass.

The latency budget

Consider the arithmetic of a 10 kHz loop: 100 microseconds per cycle, and the network inference is only one item in it. Sensor acquisition, signal conditioning, the forward pass, and command output to the power supplies all have to fit. That leaves the actor perhaps tens of microseconds. This is why the actor is deliberately small — a modest multilayer network, not a giant model. The heavy critic that made training sample-efficient never ships; only the compact policy does. In practice the deployed network is compiled and run on the control system’s real-time hardware so that a forward pass is a bounded, deterministic operation.

Distillation and determinism

The deployed actor must be both fast and predictable. Two properties matter. First, the inference time must be constant — no dynamic control flow, no data-dependent branches that make one cycle slower than the next, because a single missed deadline can lose vertical stability. Second, the numerical behavior on target hardware must match training. Teams verify that the network implemented in the real-time environment produces the same outputs as the trained model, closing the gap between the Python training stack and the C or FPGA-adjacent deployment stack. The network’s small size is what makes both properties achievable inside the budget.

Reconstruction-free control as the newer direction

A 2025–2026 line of work pushes the idea further. Rather than mapping cleaned-up reconstructed quantities to actions, controllers map raw magnetic diagnostic signals straight to actuator commands, eliminating equilibrium reconstruction from the loop entirely. Work on DIII-D trained Soft Actor-Critic controllers with an asymmetric actor-critic architecture in the NSFsim simulator and demonstrated them on the machine, reaching target parameters on early discharges without per-shot tuning; the results were reported in a 2025 preprint (arXiv:2506.13267) and a Nuclear Fusion paper in early 2026. Reported figures include a mean shape error around two centimeters in simulation and tolerance to diagnostic dropouts, with the policy transferring to an independent simulator — treat these as reported single-study numbers, not settled benchmarks. Removing reconstruction shortens the latency chain and removes a component that itself needs careful tuning, reinforcing the single-mapping thesis.

Safety Interlocks and Disruption Avoidance

A learned controller does not get to be a black box that occasionally destroys the machine. Real tokamaks wrap any controller — classical or learned — in independent safety systems, and the most interesting new AI work is about steering the plasma away from instabilities before they happen rather than only reacting after.

Figure 4: A tearing-mode predictor and hard-limit interlock sit around the RL shape policy; predicted risk triggers avoidance moves, hard-limit breaches trigger a controlled rampdown.

Figure 4 sketches the protective architecture. The sensor stream feeds both a tearing-mode predictor and an independent interlock monitor. If predicted tearing risk exceeds a limit, the system takes an avoidance action — for example, easing the plasma pressure — rather than continuing to chase the shape target. If a hard limit is breached, the interlock overrides everything and commands a controlled rampdown. Only when both checks pass do the policy’s coil and heating commands reach the actuators.

Tearing modes and predictive avoidance

Among the most damaging instabilities are neoclassical tearing modes — magnetic islands that grow in the plasma, cool it, degrade confinement, and can trigger a full disruption. The Princeton Plasma Physics Laboratory and General Atomics team addressed exactly this on DIII-D in Nature (2024). They trained deep neural networks to predict the onset of tearing instability from real-time signals — reported as up to a few hundred milliseconds ahead — and used a reinforcement-learning controller to keep the tearing likelihood below a chosen threshold. Crucially, the controller maintained high-performance H-mode operation under conditions (low safety factor, low torque) where conventional preprogrammed control tended to tear. That is disruption avoidance through learned foresight, not disruption mitigation after the fact.

Defense in depth

The right mental model is layers, not a single smart controller. The learned shape policy is the innermost loop. Above it sits a predictive avoidance layer that can override the shape objective when instability is forecast. Above that sits a hard interlock — deterministic, simple, independently verified — that will always trigger a safe rampdown if measured quantities cross machine-protection limits, regardless of what the neural networks want. This layering matters because it lets the learned components be optimized for performance while a simple, auditable system retains ultimate authority over machine safety. The AI improves the odds of never reaching the interlock; the interlock guarantees the outcome if it does.

Classical Control vs. RL Control: A Comparison

The two approaches are not strictly better or worse; they trade different things. The table below summarizes where each wins.

Dimension	Classical cascade (reconstruction + PID)	RL policy (learned)
Design effort per new shape	High — re-derive observers, re-tune gains	Low — change the reward, retrain offline
Run-time computation	Reconstruction + control law each cycle	One small forward pass each cycle
Interpretability	High — inspectable gains and signals	Low — weights are opaque
Verification	Mature, well-understood methods	Immature; empirical, still being developed
Handling of coupling	Manual decoupling, approximate	Learned jointly across all coils
Exotic shapes (negative triangularity, snowflake)	Hard, laborious	Demonstrated more readily
Dependence on model fidelity	Moderate — observers tolerate some error	High — sim-to-real gap is the main risk
Adaptation to a new machine	Re-engineer, but transferable expertise	Retrain in that machine’s simulator

The honest reading: RL shifts human effort from run-time control engineering to offline reward design and simulator fidelity, and it excels at the coupled, exotic-shape problems that punish hand-tuning. Classical control keeps the advantages of interpretability and a mature verification story, which is why the near-term reality is hybrid — learned inner loops inside classical safety envelopes, not wholesale replacement.

Trade-offs, Gotchas, and What Goes Wrong

The most important limit is the sim-to-real gap. The policy is only ever as good as the physics it trained against. If the simulator misestimates eddy currents in the vessel wall, an actuator delay, or a sensor’s response, the deployed policy can be biased or, worse, unstable in a regime the simulator got wrong. Teams narrow the gap with domain randomization — training across a distribution of plant parameters so the policy is robust to which reality it lands in — and by validating the trained policy against higher-fidelity or independent simulators before ever touching hardware. But randomization costs performance: a policy robust to a wide parameter spread is more conservative than one tuned to the true plant.

Sample cost and iteration speed cut against fast progress. Even in simulation, training a competent policy consumes enormous compute, and every change to the reward or the plant model means retraining. Unlike a PID loop you can nudge between shots, a learned policy is a batch artifact. This makes rapid on-machine iteration awkward and pushes teams toward getting the simulator and reward right before a costly training run.

Verification and trust are unresolved. A neural network gives no guarantee about how it behaves on inputs far from its training distribution — exactly the off-normal events where you most want guarantees. There is no equivalent yet of the gain and phase margins that make classical controllers certifiable. This is the single biggest reason a learned policy will, for the foreseeable future, sit inside a hard, deterministic interlock rather than replace it. A sensor dropout, an unmodeled instability, or a plasma regime the simulator never showed can all push the policy outside its competence with no built-in warning.

Generalization across machines is weak. A policy trained in TCV’s simulator does not transfer to DIII-D or JET; it encodes one machine’s geometry, coils, and dynamics. Each device needs its own accurate simulator and its own training run. The transferable asset is the method, not the weights. That matters for the field’s efficiency: the expensive simulator-building work has to be redone per machine.

Practical Recommendations

For engineers evaluating learned plasma control, the framing that survives contact with reality is “learned inner loop, classical safety envelope.” Treat the RL policy as a high-performance component whose authority is bounded by simpler, verifiable systems, not as a replacement for the whole control stack. The value shows up first on the problems hand-tuning does worst: coupled multi-coil shape control and exotic configurations.

Invest disproportionately in the simulator. The sim-to-real gap dominates outcomes, so simulator fidelity, careful circuit and eddy-current modeling, and honest validation against independent codes matter more than architecture novelty. A modest actor-critic trained against an excellent simulator beats a clever architecture trained against a poor one.

A short checklist for reasoning about a learned-control proposal:

Is there a free-boundary simulator faithful enough to train against, and validated against a second independent model?
Is domain randomization covering the plant uncertainties the real machine will actually show?
Does the deployed actor fit the real-time latency budget with constant, deterministic inference?
Is there an independent, deterministic interlock with final authority over machine protection?
Is there a predictive avoidance layer for the specific instabilities (e.g., tearing modes) this scenario risks?
Are reported performance numbers from a single study labeled as such, not treated as settled benchmarks?

Frequently Asked Questions

What is AI plasma control in a tokamak?

It is the use of machine learning — specifically deep reinforcement learning — to control the magnetic fields that confine and shape a fusion plasma. Instead of a hand-engineered chain of equilibrium reconstruction and PID controllers, a single neural network learns to read magnetic sensor signals and command the tokamak’s control coils directly. It is trained offline in a physics simulator to hit target plasma shapes and positions, then deployed onto the machine’s real-time control system running at kilohertz rates.

Did reinforcement learning actually control a real tokamak?

Yes. In 2022, a team from DeepMind and EPFL’s Swiss Plasma Center published in Nature a demonstration of a reinforcement-learning controller running on the TCV tokamak. It commanded all the control coils from a single policy and produced a range of plasma configurations, including elongated shapes, negative triangularity, a snowflake divertor geometry, and two simultaneous plasmas. Later work through 2025 and 2026 extended reinforcement-learning magnetic control to DIII-D, including reconstruction-free controllers that map raw signals to actuator commands.

How does RL plasma control handle safety and disruptions?

Through layering. The learned policy is the innermost loop and is wrapped by independent safety systems. A predictive layer can forecast instabilities — a 2024 PPPL and General Atomics study on DIII-D trained networks to predict tearing modes and used RL to steer away from them before they grew. Above that, a simple, deterministic interlock retains final authority and will command a controlled rampdown if measured quantities cross machine-protection limits, regardless of what the neural network requests.

Why train in simulation instead of on the real machine?

Because reinforcement learning needs millions of trial-and-error episodes, and a real tokamak offers only a handful of seconds-long shots per day, each costly, with bad actions risking a disruption. Training in a fast free-boundary equilibrium simulator makes the required scale feasible and keeps exploratory mistakes off the hardware. The finished policy is then transferred to the machine. The catch is the sim-to-real gap: the policy is only as trustworthy as the physics it trained against.

What is the sim-to-real gap in plasma control?

It is the mismatch between the simulator the policy trained in and the real tokamak it runs on. Differences in eddy currents, actuator delays, sensor responses, or plasma dynamics that the simulator gets slightly wrong can bias or destabilize the deployed policy. Teams reduce it with domain randomization — training across a spread of plant parameters — and by validating the policy against higher-fidelity or independent simulators before deployment. It remains the central risk and the main reason learned control stays inside a deterministic safety envelope.

Can this scale to ITER or a power-plant reactor?

Not directly yet. A policy encodes one machine’s geometry and dynamics, so each device needs its own accurate simulator and training run; the transferable asset is the method, not the weights. Reactor-scale operation also raises the bar on verification, since off-normal behavior is far more costly. The credible near-term path is hybrid: learned inner loops handling shape and instability avoidance inside mature, classical safety systems, with fidelity and verification maturing before learned components take on more authority.

AI Plasma Control for Tokamak Fusion: Reinforcement Learning (2026)