Sim-to-Real Transfer for Robot Learning: Domain Randomization, System ID and Isaac Lab (2026)

A quadruped can learn to walk over rough terrain in roughly twenty minutes of wall-clock training on one workstation GPU, according to the 2021 ETH Zurich and NVIDIA paper that popularised massively parallel simulation. The policy that results is nearly useless on the real robot unless someone has dealt with the thing that separates the simulator from the machine. That separation is the reality gap, and closing it is the whole job of sim-to-real transfer.

It matters in 2026 because GPU simulators now produce policies faster than most teams can evaluate them on hardware. Training is cheap; trust is expensive. The bottleneck has moved from “can we learn a behaviour” to “will it survive contact with a real motor, a real floor and a real camera, and how do we know before it falls over”.

You will leave with a working mental model of where the gap comes from, how domain randomization, system identification and learned actuator models each attack a different part of it, how teacher-student distillation turns privileged simulation state into deployable policies, and how to evaluate on hardware without breaking it.

What this covers: the four sources of the reality gap, randomization versus adaptation versus system identification, GPU-parallel robot reinforcement learning in Isaac Lab, actuator networks, curricula and teacher-student training, perception transfer, a digital twin calibration loop, an illustrative randomization table for a quadruped, evaluation protocol and failure modes.

Context and Background

Reinforcement learning (RL) needs enormous numbers of trials. A legged robot might need billions of control steps before a walking policy is robust, and real hardware cannot provide that: motors overheat, gearboxes wear, and a falling machine is a repair bill. Simulation offers unlimited, safe, parallel experience. The price is that every simulator is wrong about something.

Three developments define the current state of the art. First, massively parallel GPU physics. Rudin and colleagues showed in “Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning” that flat-terrain locomotion policies train in under four minutes and uneven-terrain policies in about twenty, on a single workstation GPU, with the policy transferring to the ANYmal quadruped. Second, structured randomization: instead of hoping the simulator is accurate, you train across a distribution of simulators. Third, learned models of the hardest-to-simulate component, typically the actuator.

NVIDIA’s Isaac Lab is the most widely used open framework that packages these ideas. Its 2025 paper describes it as a GPU-accelerated simulation framework for multi-modal robot learning and the successor to Isaac Gym, with actuator models, multi-frequency sensor simulation, data collection pipelines and domain randomization tools, and with planned integration of the Newton physics engine. Earlier open-source work such as the legged_gym codebase from the Rudin paper established the pattern that Isaac Lab generalises.

If you work in the wider ROS 2 and NVIDIA stack, the surrounding tooling matters as much as the training loop. Our comparison of Isaac ROS 5.0 against 4.6 covers the deployment side on Jetson-class hardware, and the ROS 2 Kilted to Lyrical migration guide covers the middleware your trained policy will eventually run under.

A note on honesty about numbers. Training-speed figures depend on robot, terrain, number of parallel environments and GPU. Where this article gives a number from a paper it names the source; where it gives a number for a worked example it labels it illustrative. Treat illustrative ranges as starting points for your own measurement, never as tuned values.

The Reality Gap: Where Simulation Lies to You

The reality gap is the systematic difference between simulated and real behaviour that causes a policy trained in one to degrade in the other. It is not one error; it is at least four independent ones, and each needs a different remedy.

Dynamics mismatch

Rigid-body simulators approximate contact with penalty or complementarity formulations, a fixed time step and simplified friction cones. Real contact involves compliance, micro-slip, surface contamination and rate-dependent friction. Mass and inertia from the CAD model ignore cabling, batteries, and the paint. Joint friction and backlash are rarely modelled at all. For a legged robot the dominant dynamics errors are usually ground contact and the mass distribution of the trunk.

Actuation mismatch

This is the quiet killer. Simulators typically model a motor as an ideal torque source, or as a position target tracked by a PD controller with a tidy gain. A real geared electric actuator has torque saturation that depends on speed, a current loop with its own bandwidth, gearbox friction, thermal derating and communication jitter. Hwangbo and colleagues showed in their 2019 Science Robotics paper on ANYmal that replacing an analytical actuator model with a neural network fitted to measured data was the key to getting agile policies to transfer. We return to this below.

Perception mismatch

Rendered images differ from camera images in noise, exposure, motion blur, lens distortion, rolling shutter and material appearance. Depth sensors add dropouts on reflective or dark surfaces and structured noise that renderers do not reproduce. Even proprioception is not clean: encoder quantisation, IMU bias drift and filter lag all differ from the simulator’s exact state.

Latency and timing mismatch

A policy that runs at 50 Hz in simulation sees observations that are exactly one step old. On hardware the observation passes through sensor sampling, a fusion filter, serialisation, middleware and inference before the action is applied, then through a motor driver. Total latency of tens of milliseconds is common and it varies. Policies trained with zero latency can be unstable when delay is introduced, and the effect is worst for fast, high-gain behaviours.

Figure 1 summarises how these four sources map onto the remedies discussed in the rest of the article.

Figure 1: Four sources of the reality gap and the remedy that addresses each. Most failed transfers come from applying one remedy to a gap that belongs to another.

The long description of the figure: dynamics errors feed into domain randomization over physical parameters and into system identification of mass and friction; actuation errors feed into a learned actuator network; perception errors feed into visual randomization and synthetic data; latency errors feed into observation and action delay randomization. All four converge on a single trained policy that is then validated on hardware.

Sim-to-Real Transfer Strategies Compared: Randomization, Adaptation and System ID

Direct answer: Sim-to-real transfer strategies fall into three families. Domain randomization trains one policy across many randomized simulators so reality looks like another sample. System identification measures the real robot and tunes the simulator toward it. Domain adaptation adjusts the policy or its inputs after deployment. Production pipelines combine all three.

Domain randomization

Domain randomization (DR) was popularised for vision by Tobin and colleagues in 2017, who trained an object detector entirely on non-photorealistic simulated images with random textures and lighting and used it for real-world grasp positioning. Peng and colleagues applied the idea to dynamics in 2018, randomizing physical parameters of a simulated arm so a pushing policy transferred without real-world training. OpenAI’s work on in-hand Rubik’s cube manipulation extended it to automatic domain randomization (ADR), where the randomization ranges widen automatically as the policy masters the current ones.

The theory is simple. If the real world lies inside the support of your training distribution, a policy that performs well across the distribution performs well on reality, provided it can either be robust to the variation or infer it from history. The practical difficulty is that the distribution has to be wide enough to cover reality but narrow enough that a solution exists and training converges.

System identification

System identification (system ID) goes the other way. You excite the real robot with chirp or step inputs, record torques and motion, and fit simulator parameters such as link masses, centre of mass, joint friction, damping and motor constants by minimising the difference between simulated and measured trajectories. The result is a narrower, better-centred simulator. System ID is cheap for the parameters that are identifiable and hopeless for those that are not, such as exact contact stiffness on every terrain you will meet.

Domain adaptation and online adaptation

Domain adaptation covers methods that change something after the fact. For perception this includes image-to-image translation that makes rendered frames look real, or feature-alignment losses. For dynamics it includes online system identification, where a policy infers latent environment parameters from a short history of observations and actions. The ETH rapid-motor-adaptation line of work and several Berkeley and CMU legged-locomotion papers use this idea: an adaptation module estimates a compact latent describing friction, payload and motor strength, and the policy conditions on it.

How they combine

The strategies are complements, not competitors. System ID sets the centre of the distribution. Randomization sets the width around that centre. Adaptation lets the policy narrow the width again at run time. Table 1 gives a practical comparison.

Dimension	Domain randomization	System identification	Online adaptation
What it needs	Parameter ranges, sim	Real robot data, fitting code	Training with history or latent encoder
Where it acts	Training time	Before training	Deployment time
Strength	Robust to unmodelled variation	Accurate nominal model	Handles per-episode changes
Weakness	Over-wide ranges give timid policies	Only identifiable parameters	Needs rich privileged signal in sim
Cost on hardware	None	Moderate, a few minutes of data	None at training, compute at run
Typical use	Friction, mass, gains, delays	Inertia, link mass, motor constants	Payload, terrain, wear

Table 1: The three transfer families and what each costs and buys.

The key conceptual point is that randomization trades off against fidelity. Spending effort on a better simulator narrows the range you need to randomize, which makes learning easier and policies less conservative. Spending nothing on fidelity and randomizing enormously costs training time and often gives a worse final policy. We return to this trade-off in the analysis section.

Massively Parallel Robot Reinforcement Learning in Isaac Lab

Robot reinforcement learning on a GPU changes the economics because thousands of environments step together in a single physics batch, and observations, rewards and actions stay on the device as tensors. There is no per-environment process, no CPU-GPU copy per step, and policy inference runs on the same card. This is the architecture that made the twenty-minute walking result possible.

Why scale changes the algorithm choices

With thousands of environments, on-policy algorithms such as Proximal Policy Optimization (PPO) become attractive because the batch is huge and fresh. The Rudin paper observed that with very large batches, the usual practice of long rollouts per environment stops being necessary; short rollouts across many environments gave good results. The consequence for transfer is subtle. Because experience is cheap, you can afford a wide randomization distribution and a long curriculum, both of which cost sample efficiency but buy robustness.

Where randomization lives in an Isaac Lab project

Isaac Lab organises a task as a manager-based environment: observation terms, action terms, reward terms, termination terms, curriculum terms and event terms are declared as configuration objects. Randomization is expressed as events. Events carry a mode, which determines when they fire: at startup (once per environment, for properties that cannot change during a run such as link masses), at reset (each episode), or at fixed intervals (for disturbances such as random pushes). The framework ships with event functions for randomizing rigid-body material properties, masses, joint parameters, external forces, and root state resets. Check the current documentation for exact function names in your installed version, since the API has evolved between releases.

The following is an illustrative sketch of the shape of such a configuration. Treat names as indicative, not as a copy-paste recipe for any specific release.

# Illustrative Isaac Lab style event configuration (verify names for your version)
@configclass
class EventCfg:
    # startup: fixed for the lifetime of each parallel environment
    physics_material = EventTerm(
        func=mdp.randomize_rigid_body_material,
        mode="startup",
        params={"static_friction_range": (0.4, 1.2),
                "dynamic_friction_range": (0.3, 1.0),
                "restitution_range": (0.0, 0.1),
                "num_buckets": 64},
    )
    base_mass = EventTerm(
        func=mdp.randomize_rigid_body_mass,
        mode="startup",
        params={"asset_cfg": SceneEntityCfg("robot", body_names="base"),
                "mass_distribution_params": (-1.0, 3.0),
                "operation": "add"},
    )
    # reset: new value each episode
    reset_joints = EventTerm(
        func=mdp.reset_joints_by_scale,
        mode="reset",
        params={"position_range": (0.9, 1.1), "velocity_range": (0.0, 0.0)},
    )
    # interval: disturbance while running
    push = EventTerm(
        func=mdp.push_by_setting_velocity,
        mode="interval",
        interval_range_s=(8.0, 15.0),
        params={"velocity_range": {"x": (-0.5, 0.5), "y": (-0.5, 0.5)}},
    )

Two details matter more than the syntax. Friction and restitution are bucketed in some GPU physics backends, which limits how many distinct material values exist at once, so the number of buckets effectively quantises your friction distribution. And mass randomization at startup means the policy cannot see a different mass per episode within the same environment instance, which affects how quickly it can learn to adapt.

Observation and action pipelines

The part of the stack that most often decides whether a policy transfers is not physics at all. Observation noise models should reflect the real estimator: if your state estimator has a lag of two control steps and a bias drift, the simulated observation should contain both. Action pipelines should include the same limits and filters as the real controller: torque clipping, rate limits, and any low-pass filter applied before the motor. If the real robot clamps joint position targets, the simulation must clamp them identically or the policy will learn to rely on an action authority that does not exist.

A good test is to log the full observation and action vector on the robot for a few minutes of teleoperation, replay the same actions in simulation, and compare the resulting observation distributions term by term. Terms whose distributions differ visibly, such as joint velocity noise, belong on your randomization list. Our article on agentic digital twins for industrial analysis discusses how the same twin data can feed this comparison continuously instead of as a one-off exercise.

Actuator Networks: Modelling the Part Simulators Get Wrong

The actuator is the interface between the learned policy and physics, so errors there act on every step of every behaviour. Most legged and arm policies output joint position targets that a low-level PD loop turns into torques, and the real behaviour of that loop is rarely an ideal PD.

The analytic baseline

The standard simulated model is torque = Kp (q_target – q) – Kd (dq), clipped to the motor’s maximum torque. Real behaviour departs from this in several ways. Torque limits fall as joint speed rises, following the motor’s torque-speed curve. Gearbox friction adds a velocity-dependent loss and a stiction term. The current loop adds a lag between commanded and delivered torque. Thermal limits derate sustained torque. A simulator that ignores these gives policies that demand torques the motor cannot deliver at speed.

The learned alternative

Hwangbo et al. (Science Robotics, 2019, “Learning agile and dynamic motor skills for legged robots”) trained a small multilayer network on data collected from the real ANYmal series-elastic actuators. The network takes a short history of joint position error and velocity and outputs the torque. During policy training in simulation, this network replaces the analytic motor model. The key point is that the policy then trains against the same input-to-torque mapping that the hardware exhibits, so a large part of the actuation gap vanishes before randomization is applied. Isaac Lab exposes actuator models as a first-class component, including explicit PD, ideal PD, DC motor with a torque-speed curve, and learned networks. Check the documentation for the exact list in your release.

Collecting actuator data

Good actuator identification data is not walking data. It needs wide excitation: chirp sweeps in position target at multiple amplitudes, step responses at different loads, and sweeps at speeds up to the operating limit, collected at the real control rate with the real communication path. Aim to cover torque and speed combinations the policy will use, and to log the target, measured position, velocity and measured current or torque at a fixed rate. Hold out a portion of the data for validation and check error in torque units, not just loss.

When an analytic model is enough

A learned actuator net is not mandatory. If your joints are quasi-direct-drive with low gear ratio, small friction and a fast current loop, a DC motor model with a measured torque-speed curve and a friction term may capture most of the effect. Learned models earn their keep with high-ratio gearboxes, series elastic elements and cheap hobby-grade servos, where backlash and slow internal loops dominate. The trade-off is data: a network is only valid within the excitation range it was trained on, and extrapolating beyond it produces confident nonsense.

Figure 2: Actuator identification loop. Real excitation data trains a torque network that replaces the ideal PD model during simulation training, then is validated on held-out sweeps.

The figure shows data flowing from the physical robot through a chirp excitation logger into a training set, then to a torque network. The network sits inside the simulator between the policy’s position target and the physics engine, and a held-out validation arrow returns to the real robot to measure torque error.

Curriculum, Privileged Teachers and Distillation

Randomization makes the task harder, and a hard task with sparse reward can fail to learn at all. Two techniques make wide randomization trainable: curricula and privileged teacher-student training.

Curricula

A curriculum starts easy and gets harder as the policy improves. In the Rudin work, terrain difficulty was increased per robot: a robot that traversed its terrain tile far enough was promoted to a harder tile, and one that failed was demoted. This automatic game-inspired scheme keeps most robots training near the edge of their competence, which is where gradient signal is richest. The same logic applies to randomization ranges. OpenAI’s ADR does this explicitly by widening a parameter’s bounds when performance at the boundary exceeds a threshold. In effect the simulator becomes a generator of harder and harder worlds, and the policy cannot overfit to a narrow band.

Privileged teacher, deployable student

A simulator knows things the real robot cannot measure: exact friction, ground height under every foot, true base velocity, applied external forces. A teacher policy trained with this privileged information learns fast and well. The student then has to behave identically using only what real sensors provide, namely proprioception history and perhaps a depth image.

Lee and colleagues (Science Robotics, 2020, “Learning quadrupedal locomotion over challenging terrain”) formalised this for ANYmal: the teacher is trained with privileged terrain and dynamics information; the student, a temporal convolutional network over a history of proprioceptive observations, is trained to imitate the teacher’s actions. The student implicitly infers terrain and dynamics from how the robot has been moving. This is online system identification emerging from imitation.

There are two common variants. In behaviour cloning, the student regresses teacher actions on states visited by the teacher. In DAgger-style training, the student’s own rollouts are labelled by the teacher, so the student sees and corrects its own distribution of mistakes. The latter is generally more robust for transfer because the student encounters off-teacher states during training, but it costs extra simulation time since the teacher must run alongside.

The latent bottleneck

A popular design compresses the privileged state into a small latent vector with an encoder, trains the teacher on observations plus latent, and then trains an adaptation module to predict the same latent from history. The advantage is interpretability: the latent can be inspected, and you can see whether it correlates with payload or friction. It also gives you a direct diagnostic, the latent prediction error, which tracks how well the student identifies the environment.

Figure 3: Two-stage training. A privileged teacher learns under randomized dynamics, then a history-based student distills it using only deployable sensors.

The figure shows the randomized simulator providing privileged state to a teacher policy trained with PPO. The teacher’s actions supervise a student network that consumes only proprioceptive history. The student is exported and deployed to the robot, while a latent prediction head provides a training-time diagnostic.

Perception Sim-to-Real: Textures, Noise and Synthetic Data

Proprioceptive policies, those using only joint states and an IMU, avoid most perception problems, which is why blind locomotion transferred first. Manipulation and navigation need cameras or depth sensors, and image transfer has its own toolkit.

Visual randomization

The original Tobin recipe randomizes object textures, distractor objects, lighting position and colour, camera pose and noise, with deliberately unrealistic textures. The argument is that if the network sees enough variety, the real image becomes just another variation. It works best when the task-relevant geometry is stable and the appearance is not, such as detecting a block position. It works less well for tasks that depend on subtle material cues, such as judging whether a cable is slack.

Sensor models

Depth cameras deserve explicit modelling. Typical effects include range-dependent noise, missing pixels on specular or very dark surfaces, edge flying pixels, and a minimum range. Add them as augmentation during training: random dropout masks, Gaussian noise that scales with depth, and small extrinsic perturbations of the camera mount. For RGB, add exposure and white-balance jitter, blur, JPEG-like compression artifacts and lens distortion. Isaac Lab’s description highlights multi-frequency sensor simulation, which matters because a camera at 30 Hz, an IMU at hundreds of hertz and a control loop at 50 Hz are never aligned on hardware.

Synthetic data and foundation features

An alternative is to avoid learning perception from simulated pixels altogether. Train a perception model on large mixed real and synthetic datasets, or use a pretrained visual encoder, and feed its low-dimensional output to the control policy. The simulation then only needs to produce the encoder’s output distribution, not photorealistic images. This pattern also keeps the reinforcement learning cheap, since the policy network is small. The cost is a second system to validate, and a failure mode where the encoder is confidently wrong in an environment unlike its training data.

A pragmatic rule

For every sensor input, ask whether the policy needs the raw signal or a processed estimate. Heightmap-based locomotion policies, which take a local terrain elevation grid, bypass image transfer by moving it into a separate mapping module with its own validation. This is a legitimate engineering choice, and it converts one hard transfer problem into two easier ones with a clean interface.

A Digital Twin Calibration Loop for Continuous Transfer

Treating sim-to-real transfer as a one-shot event is the most common process mistake. Robots wear out, payloads change, floors differ, firmware updates alter latency. The better framing is a loop in which the real robot continuously corrects the simulator, and the simulator continuously improves the policy.

The loop

Deploy the current policy with full logging of observations, actions, commands and timestamps.
Replay the logged actions in simulation from matched initial states and compute the discrepancy in resulting trajectories.
Update the identified parameters, such as friction, motor constants and latency, using the discrepancy as a fitting target.
Recentre and, if the discrepancy shrinks, narrow the randomization ranges around the new estimate.
Retrain or fine-tune the policy, and gate its release on the evaluation protocol described below.

The loop is where a digital twin earns its name in robotics: the twin is a calibrated, versioned model of one specific machine and its environment rather than a generic asset. For the product-lifecycle side of this thinking, including how twin versions and configuration state are managed, see the MoveIt Pro 10.0 versus 9.4 migration notes, which touch on how manipulation stacks handle model and configuration drift.

Figure 4: Closed calibration loop. Hardware logs drive parameter re-estimation, which recentres randomization ranges and triggers gated retraining.

The figure shows five stages arranged as a cycle: deploy and log, replay in simulation, fit parameters, update randomization ranges, retrain and gate. A side branch from the gate returns to deploy only when evaluation passes, and otherwise loops back to the fitting stage.

What to fit, and what not to

Fit parameters that are identifiable from the data you can log: link mass and centre of mass from static poses and quasi-static motions, motor constants from sweeps, latency from step-response timing, and joint friction from slow reversals. Do not try to identify everything. Contact parameters couple strongly and a fit that matches one terrain will mislead on another. For those parameters, keep randomization, and use the logs only to check that real behaviour sits inside the simulated spread.

Estimating latency

Latency is both important and easy to measure. Command a step, timestamp the command at the policy output, and timestamp the first measurable motion at the sensor input. Do this many times, because variance matters as much as the mean. Then add random delay of one to a few control steps in training that covers the measured range with margin. If latency jitter is high, fix the system first; no amount of randomization compensates for a scheduler that occasionally stalls for 100 ms.

Worked Example: Randomization Ranges for a Quadruped

The following table is illustrative. It shows the style of reasoning, not tuned values for any robot, and every range must be re-derived from your own measurements. The hypothetical robot is a 20 kg quadruped with 12 geared joints, a 50 Hz policy and a PD low-level loop.

Parameter	Nominal	Illustrative range	Reason for range
Ground friction coefficient	0.8	0.4 to 1.2	Wet tile to rubber matting
Trunk mass offset	0 kg	-1 to +3 kg	Battery position, payload, cabling
Centre-of-mass shift	0 cm	plus or minus 2 cm in x and y	Payload mounting error
Joint PD gain scale	1.0	0.85 to 1.15	Gain tuning tolerance, thermal drift
Motor strength scale	1.0	0.8 to 1.0	Battery sag, derating
Observation delay	0 steps	0 to 2 steps	Measured 20 to 40 ms pipeline
Action delay	0 steps	0 to 1 step	Driver and bus latency
Joint velocity noise	0	1.0 to 1.5 rad/s std	Encoder differencing noise
Random push	none	up to 0.5 m/s every 8 to 15 s	Disturbance robustness

Table 2: Illustrative randomization ranges for a hypothetical 20 kg quadruped. Not tuned values.

Reading the table

Consider what each row protects against. A friction range of 0.4 to 1.2 forces the policy to avoid gaits that need high traction, which is correct for a robot that will meet wet floors. Motor strength capped at 1.0 acknowledges that the simulator’s nominal motor is the best-case motor. Observation delay in whole control steps is a coarse but common way to model latency: at 50 Hz one step is 20 ms, so a range of 0 to 2 steps covers 0 to 40 ms and matches the hypothetical measured pipeline.

Checking range width with a simple budget

A useful sanity test is the effective-difficulty budget. Rank the parameters by sensitivity: perturb each individually at its range extreme with the final policy and measure the drop in survival time or tracking reward. Parameters causing large drops at the extremes are those where you should either tighten the range through system ID or invest in adaptation. Parameters causing no drop at all can often be widened for free or removed to save training time. This one-at-a-time sweep is cheap on a GPU with thousands of environments and tells you more than any intuition.

Interactions

One-at-a-time tests miss interactions. Low friction together with high delay and a weak motor is often the real failure combination. Run a coarse grid or Sobol sample over the three or four most sensitive parameters and plot the failure region. If the corner where everything is worst is a failure, you must decide whether it is physically plausible. Training for a corner that never occurs makes everything else more conservative; ignoring a corner that does occur breaks the robot.

Evaluating on Real Hardware Without Breaking It

Evaluation is where sim-to-real programmes quietly lose credibility. A policy that is demonstrated once on a good day proves nothing. You need a protocol that is repeatable, staged and tied to measurable pass criteria.

Stage 0: simulation-only gates

Before anything touches hardware, evaluate the policy in a simulator configured differently from the one used for training, ideally a different physics engine or a held-out parameter set. A policy that only works in the engine it was trained in has likely exploited simulator artefacts. Cross-engine evaluation is a strong, cheap early filter, and several research groups use it for exactly that purpose.

Stage 1: tethered or suspended

Run the policy with the robot on a stand or gantry with torque and velocity limits reduced. Check that outputs are smooth, joint limits are respected, and no oscillation exists. Log the action spectrum: high-frequency content in actions is a reliable early indicator of an unstable closed loop.

Stage 2: safe-envelope floor testing

Test on flat, forgiving ground inside a barrier, with an emergency stop within reach and a software watchdog that cuts torque on abnormal states: excessive tilt, joint velocity spikes, missed control deadlines. Command small velocities first. Record every run.

Stage 3: structured trials

Define a fixed test matrix: for locomotion, commanded speeds, turning rates, terrain types and pushes; for manipulation, object positions, object weights and lighting. Run each cell several times. Report success rate with a confidence interval, not a single run. With ten trials and eight successes, a rough 95 percent interval on success rate is wide, roughly 49 to 94 percent by the Wilson method, which tells you that ten trials are not enough to claim 80 percent. Plan trial counts accordingly.

Metrics worth recording

Success rate, mean time to failure, tracking error against commanded velocity, energy per distance (cost of transport), peak joint torque relative to limits, peak joint temperature, and a count of watchdog trips. Comparing simulated and real distributions of these metrics, rather than only the success rate, tells you where the gap remains. If simulated cost of transport is 40 percent lower than real, your actuator or friction model is optimistic, even if the robot walks.

Regression discipline

Version the policy, the simulation configuration, the randomization ranges and the calibration data together. When a new policy fails on hardware, you need to know whether the cause is the policy, a changed simulator or a changed robot. Without a recorded baseline policy re-run on every hardware session, you cannot distinguish a worse policy from a worse day.

Trade-offs, Gotchas, and What Goes Wrong

Sim-to-real transfer fails in recognisable ways. Knowing them in advance is worth more than another percentage point of tuning.

Over-randomization produces timid policies

A policy trained across a very wide distribution must succeed in the worst corner, so it behaves conservatively everywhere. Symptoms are low swing heights, slow gaits, wide stance and poor command tracking even in nominal conditions. If your policy is safe but sluggish, narrow the ranges using system ID before touching the reward function.

Under-randomization produces brittle policies

The opposite failure is a policy that looks superb in simulation and falls within seconds on hardware. Typical causes are missing latency, an optimistic motor model, or training only on one friction value. The fix is a diagnostic ablation: introduce each missing effect into the simulator alone and see which one reproduces the real failure.

Exploiting simulator artefacts

RL optimises whatever the simulator rewards. Policies discover foot-skating on penalty-based contact, jittering joints that exploit solver stiffness, or sliding against thin collision meshes. Always inspect behaviour visually and in the action spectrum, and cross-check in a second engine. Smoothness penalties and torque or acceleration limits in the reward discourage the worst artefacts.

Privileged information leaks

If the student’s input accidentally includes a quantity that real sensors cannot supply, such as true base linear velocity, it transfers perfectly in simulation and fails on hardware. Audit the observation list against the real estimator output. A related error is training with perfect state estimates while the deployed estimator drifts under slip.

Friction and contact are hard to randomize honestly

Real friction varies with speed, contamination and contact area. A single Coulomb coefficient drawn uniformly is a crude proxy. Terrain with compliance, such as sand, grass or foam, is not a friction problem at all, and no friction range will cover it. Treat soft terrain as a separate modelling effort.

Safety and hardware wear

A policy producing high-frequency torque commands can destroy gearboxes or trip thermal limits long before it falls over. Hardware-side safety must be independent of the policy: torque and speed limits in the motor drivers, a watchdog on control deadlines, a tilt cutoff, and an emergency stop that does not depend on the software stack. For manipulation add force limits and workspace fences, and keep humans outside the reachable envelope during early trials. Compliance with machinery safety standards is a separate engineering discipline and should not be inferred from policy performance in tests.

The evaluation gap

The final trap is organisational. Because GPU training is fast, teams generate more candidate policies than hardware time can evaluate, and selection falls back on simulated reward. Simulated reward does not rank policies by real performance beyond a point. Budget hardware time as the scarce resource, and use cross-engine simulation, adversarial parameter sweeps and staged protocols to cut the candidate list before it reaches the robot.

Practical Recommendations

Begin with the cheapest, highest-leverage fixes. Measure latency and fix it in the system, then model it in training. Model the actuator, either with a measured torque-speed DC motor model or a learned network from sweep data. Make the observation and action pipeline in simulation structurally identical to the real controller, including clipping and filtering. Only then spend effort on broad randomization of friction, mass and gains.

Use system ID to centre your ranges, domain randomization to set their width, and a history-conditioned student to adapt at run time. Use a curriculum so wide ranges remain trainable. Keep a versioned record of the calibration data and of the simulator configuration that produced each policy. Evaluate in stages and report confidence intervals, not best runs. When something fails on hardware, run the ablation: change one simulated effect at a time until the simulator reproduces the failure, then keep that effect.

For teams building on the NVIDIA stack, Isaac Lab provides the managers, event terms, actuator models and sensor simulation to implement all of this in one configuration tree, and its documentation is the source of truth for names and defaults in your installed release. The ROS 2 side of deployment, covered in our Lyrical migration and Isaac ROS articles, should be pinned and tested together with the policy.

Checklist before the first hardware trial

[ ] End-to-end latency measured, including variance, and modelled in training.
[ ] Actuator model validated against held-out sweep data in torque units.
[ ] Observation list audited: no privileged quantities in the student’s inputs.
[ ] Randomization ranges justified by measurement or physical reasoning.
[ ] One-at-a-time sensitivity sweep and a small interaction sweep completed.
[ ] Policy evaluated in a second physics engine or a held-out parameter set.
[ ] Independent hardware safety limits, watchdog and emergency stop verified.
[ ] Test matrix, trial counts and pass criteria written before running.
[ ] Policy, simulator config, ranges and calibration data versioned together.

Frequently Asked Questions

What is sim-to-real transfer in robotics?

Sim-to-real transfer is the process of training a robot controller in simulation and deploying it on physical hardware with acceptable performance. It is difficult because simulators differ from reality in dynamics, actuation, perception and latency, a mismatch known as the reality gap. Techniques such as domain randomization, system identification, learned actuator models and teacher-student distillation reduce that mismatch so a simulated policy keeps working on the real machine.

What is the reality gap?

The reality gap is the systematic difference between simulated and real-world behaviour that makes a policy trained in simulation perform worse on hardware. It has several independent sources: contact and mass modelling errors, unmodelled actuator behaviour, sensor and rendering differences, and timing delays. Because the sources differ, no single technique removes the gap. Practitioners diagnose which source dominates, then apply the matching remedy.

What is the difference between domain randomization and system identification?

Domain randomization trains a policy across many randomly varied simulators so reality becomes one more sample, and it does not require real data. System identification uses measurements from the real robot to tune simulator parameters toward the true values. Randomization buys robustness to what you cannot measure; system ID buys accuracy for what you can. Strong pipelines use system ID to centre the ranges and randomization to set their width.

Is Isaac Lab required for sim-to-real transfer?

No. The techniques are framework independent, and other open simulators and codebases support similar workflows. Isaac Lab is a popular option because it bundles GPU-parallel physics, manager-based environments, event-driven randomization, actuator models and sensor simulation, and it succeeds Isaac Gym. Choose by robot support, physics fidelity for your contact conditions and how well it integrates with your deployment stack.

Why use a teacher-student setup instead of training the deployable policy directly?

A teacher with privileged simulator information, such as exact friction and terrain height, learns quickly and robustly. A student limited to real sensor inputs, usually a history of proprioception, then imitates the teacher and implicitly infers the hidden quantities. Training the student directly under wide randomization is often slower and less stable because the hidden parameters make the reward signal noisy.

How many real-world trials do I need to validate a policy?

More than most teams expect. With 10 trials and 8 successes, a 95 percent Wilson interval spans roughly 49 to 94 percent, so you cannot claim 80 percent reliability. Tighter claims need dozens of trials per condition. Use staged testing to catch gross failures cheaply, and reserve statistically meaningful trial counts for the final release candidate and the conditions that matter most.