Humanoid Robot Control Stack: Layer-by-Layer Architecture (2026)

Last Updated: October 2026

A humanoid that folds laundry on video and the same humanoid that survives eight hours on a factory floor are separated by a stack most demos never show. The part that decides whether a robot keeps its balance is not the large model that understands “put the tote on the shelf”. It is a chain of controllers running at very different speeds, each trusting the one below it to behave, and each capable of ruining the one above it when it does not. Good humanoid robot control is mostly the discipline of making those hand-offs boring.

Two things changed in the last twelve months. Learning moved down the stack: Figure now describes a roughly 10-million-parameter neural network replacing a large hand-written C++ whole-body controller, and NVIDIA open-sourced a general motion-tracking policy for the Unitree G1. At the same time, safety moved the other way: the draft ISO 25785-1 standard pushes manufacturers toward small, deterministic, auditable layers that sit outside the learned system. This article argues that this split, learned inside and certified outside, is the defining architecture of 2026 humanoids.

You will leave with a layer-by-layer map of the stack, the timing budget that governs it, working code for a balance MPC, an RL action bridge and a safety monitor, and a list of failure modes to design against.

What this covers: the reference stack and timescales, perception and state estimation, planners and VLA models, whole-body control (model-based versus learned), sim-to-real, actuation, safety and standards, real-time middleware, how current platforms compare, and what goes wrong in practice.

Context and Background

Until about 2022 a humanoid controller was a model-based pipeline. Boston Dynamics-style stacks combined a state estimator, a footstep or centroidal planner, and a whole-body quadratic program (QP) that turned desired accelerations and contact forces into joint torques. The approach was explainable and tunable, and it worked, but each new behaviour needed expert hands, and every model error became a hand-tuned gain somewhere.

Reinforcement learning (RL) in massively parallel simulators changed the economics of locomotion. Work such as Rudin et al., “Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning”, showed that legged policies train in minutes to hours on a single GPU, and Radosavovic et al. showed in Science Robotics (2024) that a causal transformer trained in simulation could walk a full-size humanoid outdoors without fine-tuning. Locomotion, the hardest part of the old pipeline, became a training problem rather than a derivation problem.

The second shift is above the controller. Vision-language-action (VLA) models such as RT-2, OpenVLA (7 billion parameters) and Physical Intelligence’s pi-0 family map camera images and language to actions, which gave humanoids a credible route to general manipulation. Our own Physical Intelligence pi-0.5 breakdown covers that model family in depth.

The third shift is industrial. Boston Dynamics showed a production version of Atlas at CES 2026, reported by The Robot Report as 56 degrees of freedom (DoF), IP67 rated, with a four-hour battery and autonomous swapping, with all 2026 units committed to Hyundai’s Robot Metaplant Application Center and Google DeepMind. Agility introduced Digit 5 in September 2026 as a humanoid designed for cooperatively safe work. Once robots are products rather than research platforms, the interface questions (what may the model command, what may it never command) become engineering requirements. The ROS 2 Kilted to Lyrical migration notes show how much of this lives in middleware.

A caution on sources: manufacturers publish architecture claims, not audited specifications. Where this article cites a platform number, it is as reported by the vendor or a named outlet, and where I could not verify something it is labelled.

The Reference Architecture: Three Timescales and One Referee

The humanoid control stack is a hierarchy of loops. A semantic planner runs at roughly 1 to 10 Hz, a visuomotor policy at 50 to 200 Hz, a whole-body controller at 500 Hz to 1 kHz, and joint servo loops at 10 kHz and above. A separate deterministic safety monitor can cut authority at any level. Each layer outputs a target for the layer beneath, never a torque for a joint it does not own.

Figure 1: Layered humanoid robot control stack with timescales and the safety monitor acting on the lowest layers.

The figure shows the structure that nearly every serious platform converges on, even when marketing names differ. Sensing flows up, targets flow down, and the dotted safety path bypasses everything learned. The numbers are typical ranges, not a standard: Figure publicly describes a three-tier design (System 2 for semantics, System 1 at 200 Hz, System 0 at 1 kHz), Google DeepMind describes an orchestrator model plus an action model, and NVIDIA’s SONIC runs its tracking controller at 50 Hz feeding a lower-level PD loop.

Why timescale separation is a physical requirement, not a style choice

A biped is an inverted pendulum. For a centre-of-mass (CoM) height of 0.85 m the pendulum’s natural rate is the square root of g over z, which is sqrt(9.81 / 0.85) = 3.40 rad/s, a time constant of about 0.29 s. A fall that begins is therefore a roughly 300 ms event. A language model that takes 300 ms to answer cannot be in the balance loop; a 5 ms visuomotor policy barely can; a 1 ms controller comfortably can.

The corollary is that every layer must be safe to ignore for the duration of the next slower layer’s latency. If the planner stalls for two seconds, the lower layers must keep the robot standing, then settle it. This “graceful stale-command” behaviour is the first thing to test and the last thing demos show.

The contract between layers is the architecture

What passes between layers matters more than what runs inside them. There are four common interface styles:

Task-space targets: base velocity, hand poses, gaze. Easy to debug, limited expressiveness.
Joint targets: positions for all joints with PD gains, the pattern in most RL stacks, including NVIDIA’s released G1 deployment, where per-motor stiffness and damping scaling is a deployment parameter.
Motion references: a short window of future poses. SONIC consumes a lookahead of 10 frames at 20 ms spacing by default, about 200 ms, and has a low-latency teleoperation variant with 4 frames.
Latent tokens: SONIC’s controller produces 64-dimensional latent motion tokens through an encoder/decoder pair, so a higher layer can speak a learned motion vocabulary instead of joint angles.

Latents are powerful and dangerous. They compress well and transfer across tasks, but a latent cannot be inspected with a unit limit. That is why safety, covered below, almost never sits behind a latent interface.

Where the model-based controller still earns its place

Model predictive control (MPC) predicts the robot’s motion over a horizon, picks the control sequence that minimises a cost under constraints, applies the first step and repeats. Constraints are the point: friction cones, joint limits and the zero moment point (ZMP) staying inside the foot are written down, not hoped for. Learned policies satisfy constraints softly, through reward shaping. When a constraint is a safety boundary, soft is not enough.

Even Figure’s System 0, which it says replaced 109,504 lines of hand-engineered C++ with a learned prior, still emits explicit joint-level commands, which is an interface a separate monitor can bound. Learned-versus-model-based is not a binary choice; it is a question of which constraint you can afford to violate occasionally.

A balance MPC you can run

The following linear inverted pendulum (LIPM) MPC is deliberately small. State is CoM position, velocity and acceleration; the input is jerk; the ZMP is a linear function of state, and it is constrained to the foot. It needs numpy and cvxpy and solves a 40-step, 0.8 s horizon in well under a second on a laptop including problem construction.

import numpy as np, cvxpy as cp

g, z, dt, N = 9.81, 0.85, 0.02, 40          # 0.8 s horizon at 50 Hz
A = np.array([[1, dt, dt**2/2], [0, 1, dt], [0, 0, 1]])
B = np.array([dt**3/6, dt**2/2, dt])
C = np.array([1, 0, -z/g])                   # ZMP output row

def solve(s0, v_ref, zmp_lo, zmp_hi, w_v=1.0, w_j=1e-6, w_zmp=1e-2):
    S = cp.Variable((3, N+1)); J = cp.Variable(N)
    cost, cons = 0, [S[:, 0] == s0]
    for k in range(N):
        cons += [S[:, k+1] == A @ S[:, k] + B * J[k],
                 C @ S[:, k+1] >= zmp_lo[k], C @ S[:, k+1] <= zmp_hi[k]]
        cost += w_v*cp.square(S[1, k+1]-v_ref) + w_j*cp.square(J[k]) \
              + w_zmp*cp.square(C @ S[:, k+1] - 0.5*(zmp_lo[k]+zmp_hi[k]))
    cp.Problem(cp.Minimize(cost), cons).solve(solver=cp.OSQP, warm_start=True)
    return S.value, J.value

lo = np.full(N, -0.05); hi = np.full(N, 0.10)   # foot spans 15 cm fore-aft
S, J = solve(np.zeros(3), 0.4, lo, hi)
zmp = C @ S[:, 1:]
print("final v=%.3f m/s  max zmp=%.3f  min zmp=%.3f" % (S[1,-1], zmp.max(), zmp.min()))

I ran this on a stock Python environment. It printed final v=0.497 m/s max zmp=0.100 min zmp=-0.050. Two things are worth noticing. The ZMP bounds are respected exactly, so the constraint behaves as a hard limit, not a penalty. And the final velocity overshoots the 0.4 m/s reference, because nothing in the cost or constraints pins the terminal state: the optimiser happily ends the horizon mid-acceleration. Real controllers add a terminal capture-point constraint for exactly this reason.

Deeper Analysis: What Runs in Each Layer

Perception and state estimation

The state estimator is the least glamorous and most load-bearing layer. A floating-base humanoid has no fixed frame; the controller needs base pose and velocity, which come from fusing an inertial measurement unit (IMU), joint encoders, leg kinematics with contact detection, and optionally vision or LiDAR odometry. The standard workhorse is an Extended Kalman Filter, often a contact-aided invariant filter; our multi-sensor fusion architecture guide and SLAM architecture overview cover the filter and map-building details.

Three engineering details matter more than the filter choice. First, time synchronisation: an IMU sample that is 5 ms stale at 1 kHz is five control cycles of error in the estimated tilt rate. Second, foot-contact classification: leg odometry assumes the stance foot is stationary, so a misclassified contact injects a velocity error that the controller faithfully tries to correct. Third, estimator output must carry a covariance or health flag so the safety layer can distrust it.

Learned policies often consume a stripped-down proprioceptive observation (joint positions and velocities, base angular velocity, projected gravity, last action) and do not need a global pose at all. That is a feature: it removes a dependency that can fail. Vision enters at the policy layer for terrain or manipulation, where a 30 Hz camera stream is adequate.

High-level planners: LLMs, VLMs and VLAs

The top of the stack answers “what next”. Three patterns are in production use or in credible announcements:

Orchestrator plus action model. Google DeepMind’s Gemini Robotics 1.5 (September 2025) is a VLA that turns vision and instructions into motor commands, paired with Gemini Robotics-ER 1.5, a vision-language model for embodied reasoning that plans and issues natural-language instructions per step. DeepMind says the VLA can generate internal reasoning before acting, and that motions learned on one robot transferred to others, including Apptronik’s Apollo. ER 1.5 is available through the Gemini API; the VLA was limited to select partners at launch.
Dual-system. Figure’s Helix family splits a slower semantic System 2 from a fast visuomotor System 1. System 1 runs at 200 Hz as a transformer conditioned on System 2 latents, consuming head and palm cameras, fingertip tactile and full-body proprioception.
Behaviour trees with learned skills. A deterministic tree sequences skills and handles retries, with learned policies as leaves. This is less fashionable and far easier to audit; see our behavior trees for robot task planning guide.

The failure that matters at this layer is not hallucination in the chatbot sense; it is mismatch between what the planner believes the skill library can do and what it can reliably do. A planner that requests an out-of-distribution skill gets an unpredictable low-level response. Constrain the action vocabulary and have the lower layers refuse unknown requests rather than extrapolate.

Visuomotor policies and imitation

Between language and joints sits the policy that closes the loop on pixels and touch. Two training families dominate: diffusion or flow-matching action generation from demonstrations (pi-0 uses a flow-matching action expert; see our diffusion policy guide), and RL fine-tuning or distillation in simulation. Demonstration data comes from teleoperation, which is why headset teleop (NVIDIA’s SONIC release supports PICO VR) is a first-class feature rather than an accessory.

Figure 2: Data flow in humanoid robot control, showing the state estimator feeding both the task policy and the whole-body layer.

Figure 2 makes a point easy to miss: the estimator feeds two consumers at two rates. The task policy sees a filtered, lower-rate view; the whole-body layer sees the fast one. Giving both the same stale state is a classic source of low-frequency oscillation.

Whole-body control: QP, MPC and learned priors

Whole-body control (WBC) is the layer that turns “move the hand here, keep the torso level, hold the left foot planted” into joint torques that respect the body’s dynamics. It is the layer whose implementation is changing fastest.

The classical form. The robot’s equation of motion is M(q) q-ddot + h(q, q-dot) = S-transpose tau + J-transpose-c f, where M is the inertia matrix, h collects Coriolis and gravity terms, S selects actuated joints, and the contact Jacobian J-c maps ground reaction forces f into joint space. A WBC QP chooses joint accelerations, torques and contact forces that satisfy this equation exactly, subject to friction-cone, torque and joint limits, while minimising weighted task errors. Strict priority is handled either with hierarchical QPs or with weights spanning several orders of magnitude. At 500 Hz to 1 kHz, a QP with 30 to 50 joints is tractable with a warm start and a specialised solver, and libraries such as Pinocchio, Crocoddyl and Drake provide the dynamics and solvers.

MPC on top. Solving the full-body nonlinear MPC at 1 kHz is still beyond most onboard computers, so practical systems stack models. A simplified centroidal or LIPM MPC, like the one above, runs at 30 to 100 Hz and produces CoM and contact-force references. The WBC QP then tracks them at kilohertz rate with full dynamics. The simplified layer chooses where to step; the full-body layer decides how every joint cooperates to get there.

The learned form. The 2025 to 2026 trend replaces the QP, or the QP plus footstep logic, with a neural network trained by RL to track motion references. Figure states that System 0 is a roughly 10M-parameter network running at 1 kHz, trained on over 1,000 hours of retargeted human motion in simulation with more than 200,000 parallel environments and domain randomisation. NVIDIA’s open SONIC uses a different recipe: motion tracking as the training task, 64-dimensional latent tokens, and a 50 Hz controller on the Unitree G1; the repository states the Bones-SEED motion dataset (about 288 hours retargeted to G1) can be used to train from scratch.

Both are worth reading critically. A 10M-parameter network is tiny by modern standards, which is the point: it fits in a 1 kHz budget on an embedded GPU or CPU, and it is easier to bound and test than a large model. What neither disclosure supplies is a failure-rate curve. “Replaced 100,000 lines of C++” is a statement about maintenance burden, not about reliability.

Model-based versus learned whole-body control

Dimension	QP / MPC whole-body control	Learned tracking policy
Behaviour authoring	Expert tuning of tasks, weights, gait logic	Reward design, data curation, randomisation
Constraint handling	Hard, exact in the solver	Soft, via reward and termination
New behaviour cost	Weeks of engineering per gait	Retrain on new reference motions
Compute at runtime	Solver with variable iteration time	Fixed forward pass, deterministic latency
Verification	Inspectable, solver status reported	Test coverage only, no proofs
Contact-rich robustness	Sensitive to model error	Often more robust to unmodelled dynamics
Failure signature	Infeasible QP, gain oscillation	Out-of-distribution action, silent drift

Two rows decide most real choices. Deterministic latency favours learned policies, since a forward pass takes the same time every cycle, while QP solve time varies with active constraints. Exact constraint satisfaction favours the QP. Hybrid architectures use an RL policy for gait and a model-based layer for contact-force limits, or run the learned policy under a model-based safety projector.

A numeric sanity check on the timing budget

Suppose the stack runs a 30 Hz camera, a 200 Hz visuomotor policy, a 1 kHz whole-body layer and a 10 kHz servo. Worst-case staleness composes. A camera frame is up to 33 ms old when sampled and, with 30 ms of encode, transfer and inference (illustrative), the policy acts on a scene about 63 ms old; the whole-body layer then holds that target for up to 5 ms until the next policy output. A walking humanoid moving at 1 m/s therefore acts on a world model that is 6 to 7 cm out of date for the terrain ahead. This is acceptable for a flat floor, marginal at a stair edge, and the reason stair policies lean on proprioception and short-horizon height maps rather than raw video.

Figure 3: Sim-to-real loop for learned humanoid robot control, from retargeted data through randomised simulation to deployment and failure logging.

Sim-to-Real: How Learned Controllers Reach Hardware

The loop in Figure 3 is the same in most labs. Source motions from motion capture or teleoperation are retargeted to the robot’s skeleton; a physics simulator runs thousands to hundreds of thousands of environments in parallel; domain randomisation perturbs mass, friction, motor strength, latency and sensor noise; the resulting policy is exported to ONNX or TensorRT and tested on hardware; and failures flow back into the data and the randomisation ranges.

Choosing the simulator and the physics

Simulator choice affects contact fidelity and throughput. Our comparisons of Isaac Lab, Isaac Sim and Gazebo Harmonic and of Newton, MuJoCo Warp and Isaac Lab for GPU physics cover the trade-offs, and the Isaac Lab RL training tutorial walks through a full training run. Robot description quality matters as much as the engine: wrong link inertias or joint damping in URDF or MJCF are baked into every rollout. See robot description formats.

What domain randomisation must cover

The reality gap for a humanoid is concentrated in five places, and a policy that has not seen variation in each will be brittle there:

Actuator dynamics. Torque saturation, backlash, friction and gearbox efficiency. A common fix is an actuator network or a calibrated friction model rather than ideal torque sources.
Latency. Real stacks add several milliseconds between the policy and the motor current loop. Randomise delay, including jitter, across the range your bus can exhibit.
Contact. Floor friction, compliance, and foot-sole geometry.
Mass properties. Payload changes, battery position, cable mass and hand attachments.
Sensing. IMU bias and noise, encoder quantisation, dropped frames.

Randomisation ranges that are too narrow produce brittle policies; ranges that are too wide produce timid ones that never use the robot’s capability. The practical method is to measure the real system, set ranges to cover the measured spread plus margin, and widen only where hardware failures show a gap.

The action bridge

Most RL humanoid policies output offsets from a default joint pose at 50 Hz, and a PD loop runs at kilohertz rate. The glue is small, but the choices in it, such as action scale, filtering and clipping, determine smoothness and safety. Here is a minimal bridge:

import numpy as np

class ActionBridge:
    def __init__(self, q_default, kp, kd, action_scale=0.25, lpf_alpha=0.6,
                 q_min=None, q_max=None):
        self.q0, self.kp, self.kd = q_default, kp, kd
        self.scale, self.alpha = action_scale, lpf_alpha
        self.q_min, self.q_max = q_min, q_max
        self.q_tgt = q_default.copy()

    def on_policy(self, action):                 # called at 50 Hz
        tgt = self.q0 + self.scale * np.clip(action, -3.0, 3.0)
        self.q_tgt = self.alpha * tgt + (1 - self.alpha) * self.q_tgt
        if self.q_min is not None:
            self.q_tgt = np.clip(self.q_tgt, self.q_min, self.q_max)

    def torque(self, q, qd):                     # called at 1 kHz
        return self.kp * (self.q_tgt - q) - self.kd * qd

b = ActionBridge(np.zeros(12), kp=np.full(12, 80.0), kd=np.full(12, 2.0))
b.on_policy(np.ones(12))
print(np.round(b.torque(np.zeros(12), np.zeros(12))[:3], 2))

Run as written, it prints [12. 12. 12.] N·m: a unit action becomes a 0.25 rad offset, the low-pass filter passes 60 percent of it (0.15 rad), and a stiffness of 80 N·m/rad turns that into 12 N·m. Those numbers are illustrative, not from any shipping robot. The point is that stiffness, action scale and filter coefficient form one coupled design: change kp after training and you have deployed a different policy. NVIDIA’s deployment notes treat per-motor Kp/Kd scaling as a deployment parameter for the same reason.

Digital twin as the test harness

A digital twin of the robot is most valuable not for training but for regression. Every control-software change should replay recorded sensor logs and fault injections against the twin before touching hardware: a stale IMU, a dropped joint, a 20 ms bus stall. If your twin cannot reproduce last month’s fall from its logs, it is not a twin yet. This is the practice described in our TwinOps lifecycle architecture article.

Actuation: Where the Control Stack Meets Physics

Every layer above assumes the actuator does roughly what it is told. Actuators are where that assumption is tested, and the control architecture should be shaped by their properties.

Quasi-direct drive, series elastic and geared hands

Humanoid joint designs fall on a spectrum. Quasi-direct-drive (QDD) joints use low-ratio gearboxes, giving high backdrivability and good torque transparency, so current control approximates torque control, at the cost of lower peak torque density. Series elastic actuators (SEAs) place a spring between motor and load, which makes force measurable from spring deflection and provides shock tolerance, at the cost of bandwidth. High-ratio harmonic or planetary drives deliver torque density but add friction and reflected inertia, so torque control requires a joint torque sensor or a good friction model. Atlas’s reported fully rotational joints and the 56 DoF count are consistent with a high-DoF, serviceable design, but Boston Dynamics has not published the control bandwidth details.

Why the inner loop stays classical

Below the whole-body layer there is almost always a field-oriented-control (FOC) current loop at 10 to 40 kHz running on a motor driver, wrapped by a position or impedance loop at 1 to 10 kHz. These loops are tuned analytically and verified against step responses. No one trains a neural network to commutate a motor, and for good reason: a 50 microsecond current-loop deadline is hostile to anything that is not a fixed-time computation.

Thermal and energy limits are control inputs

A leg joint can produce a peak torque for a second and a lower continuous torque indefinitely. A controller that ignores winding temperature will work in a five-minute demo and fail in hour three. Production stacks therefore feed motor temperature and battery state into the planners: reduce squat depth, slow the gait, or reorder tasks so a hot joint can rest. Agility reports Digit 5 at a 90-minute runtime with a 9-minute charge, a 10:1 run-to-charge ratio, which is as much a scheduling constraint on the fleet layer as a battery fact.

Safety Layers and Standards: Certified Outside the Learned System

A learned policy cannot certify itself. Standards and risk assessments want a small number of auditable behaviours that hold regardless of what the policy outputs. That is why the safety architecture of a 2026 humanoid looks old-fashioned next to its planner.

What a humanoid adds to the safety problem

Conventional industrial robot standards assume a machine that stays where it is when power is removed. A humanoid does not. According to the iScoop explainer on ISO 25785-1, the draft covers industrial mobile robots with “actively controlled stability”, including legged robots that would become unstable without power, and it calls for defined stability parameters, fall detection, controlled descent, test protocols and recovery procedures. It notes ISO 10218 assumes a robot that will not tip over, and that stability as a hazard previously fell between ISO 10218, ISO 13482 (personal care robots) and ISO 3691-4.

Status matters here. As of early 2026 the draft was still a working draft, and the same source reports industry estimates of 18 to 36 months from the May 2025 draft to a ratified standard, so publication in late 2026 or 2027. I could not verify the current stage as of October 2026; check ISO’s project page before citing it as published. Agility says it is contributing to ISO 25785-1 and to ANSI/A3 TR R15.108, a technical report for dynamically stable industrial mobile robots.

An independent monitor that can only remove authority

The design pattern that follows is a deterministic monitor on separate hardware or a separate real-time core, with its own sensors where possible. Agility’s Digit 5 announcement describes an independent safety controller that oversees responses when people are detected within an unsafe distance, with avoid, stop or sit-down behaviours and visual and audible intent cues. NVIDIA’s Halos for Robotics platform is named as the path for its safety certification work, with NVIDIA IGX Thor as the compute platform; Agility says it passed an independent field evaluation on a customer production line. These are vendor statements, and Digit 5 was announced as early access for the first half of 2027, so treat them as design direction. Our Digit 5 safety reference design works through the architecture in more detail.

A monitor in this style has three rules. It never trusts a learned layer for its own watchdog. It can only reduce authority (scale down, damp, sit). And once tripped it latches until a human resets it. This is a minimal version:

from dataclasses import dataclass
import math

@dataclass
class Limits:
    max_tilt_rad: float = math.radians(25)
    max_tilt_rate: float = 3.0
    max_joint_vel: float = 12.0
    max_cmd_age_s: float = 0.010
    human_stop_dist_m: float = 1.0

RUN, DAMP, SIT = 0, 1, 2

def step(lim, now, imu_tilt, imu_tilt_rate, joint_vel, cmd_stamp, human_dist, state):
    if state != RUN:
        return state, 0.0                       # latched until human reset
    if now - cmd_stamp > lim.max_cmd_age_s:     # stale command
        return DAMP, 0.0
    if abs(imu_tilt) > lim.max_tilt_rad or abs(imu_tilt_rate) > lim.max_tilt_rate:
        return SIT, 0.0                         # controlled descent
    if max(abs(v) for v in joint_vel) > lim.max_joint_vel:
        return DAMP, 0.0
    if human_dist < lim.human_stop_dist_m:      # separation-style slowdown
        return RUN, max(0.0, (human_dist - 0.3) / (lim.human_stop_dist_m - 0.3))
    return RUN, 1.0

Four test calls print (0, 1.0) for a healthy state, (0, 0.5) when a person is 0.65 m away, (1, 0.0) for a command 21 ms old against a 10 ms limit, and (2, 0.0) for a 0.60 rad tilt. All thresholds here are illustrative; real values come from a hazard analysis and the robot’s measured stopping behaviour.

Figure 4: Safety monitor sequence for humanoid robot control, covering slowdown on human detection, damping on a stalled policy and latched operator reset.

Fall response is a design decision

When a fall is unavoidable, the options are limited: a protective step, a controlled sit or kneel, or damping all joints. A back-of-envelope bound: a 70 kg robot with its CoM 0.9 m above ground stores up to roughly m·g·h = 70 × 9.81 × 0.9 ≈ 618 J of potential energy. In practice CoM drop is less, but the figure explains why controlled descent and a low-energy pose matter. The draft standard’s emphasis, as reported, is on controlled descent rather than collapse on power loss, and some manufacturers already use a kneeling zero-energy pose.

The corollary for control design: the fall response should be reachable from the monitor without the policy’s cooperation. If sitting down requires the RL policy to be healthy, it will fail in exactly the situation where the policy is the problem.

Speed and separation, in numbers

For a person-aware behaviour, stopping distance sets the zone. If a hand moves at 1 m/s and the monitor takes 20 ms to detect and actuate plus 60 ms of mechanical stop, it travels about 8 cm. Add a 0.5 m/s walking base, and the body travels 4 cm in the same time. These products set the minimum separation, and they are the reason the monitor runs at 1 kHz on a dedicated core rather than as a task in the main control process. ISO/TS 15066’s biomechanical force and pressure limits, now integrated into ISO 10218-2:2025, supply the contact thresholds for collaborative operation, though how they apply to a legged platform is exactly what the new draft standard is meant to settle.

Real-Time Middleware and Compute

Operating system and scheduling

Most humanoid stacks run Linux with a PREEMPT_RT kernel for the 500 Hz to 1 kHz loops, with CPU isolation (isolcpus, nohz_full), locked memory and a fixed-priority SCHED_FIFO thread. Rules that avoid the usual jitter: no dynamic allocation in the loop, no logging that blocks, no page faults, and no sharing a core with the inference process. The inference process (policy, perception) typically runs on a GPU module, such as NVIDIA’s Jetson Thor-class parts, while the hard real-time loop runs on CPU cores or a microcontroller.

Buses and middleware

Between the main computer and joints, EtherCAT is the common industrial choice because it gives deterministic cycle times and distributed clocks; CAN FD is common in lighter robots. Above it, ROS 2 with DDS is a convenient integration layer for the non-hard-real-time parts: perception, planning and logging. ROS 2’s ros2_control framework gives a hardware-interface abstraction, and its real-time executor patterns are what you use for the 1 kHz controller manager. Keep the safety path off DDS: put it on a dedicated link with a bounded, tested latency. The ROS 2 Kilted to Lyrical migration guide lists current distribution changes that affect these deployments.

Latency accounting as a first-class artifact

Maintain a latency table the way you maintain a power budget: sensor timestamp, bus transfer, policy inference, command publish, driver latch, current-loop response. Measure each with hardware timestamps, not wall-clock logging. In my experience this often reveals a link nobody owned, and sim-to-real gaps frequently trace to a delay mismatch rather than a physics error.

How Current Platforms Compare

Public specifications are marketing-adjacent; the table lists what the cited sources actually say, and leaves blanks as “not disclosed” rather than guessing.

Platform	Reported control approach	Reported facts	Caveats
Figure (Helix 02)	Three tiers: S2 semantic, S1 at 200 Hz, S0 learned whole-body at 1 kHz	S0 about 10M parameters; over 1,000 hours of retargeted human motion; 200,000+ sim environments	Vendor disclosure; no independent failure-rate data
Boston Dynamics Atlas (production)	Not published in detail; DeepMind Gemini Robotics partnership announced	56 DoF, 2.2 m reach, up to 49.8 kg payload, IP67, 4 hr battery with swapping (Robot Report)	2026 units committed to Hyundai RMAC and DeepMind
Agility Digit 5	Independent safety controller; AI human detection	22.7 kg repeated lift, 129 kg, 90 min runtime; early access H1 2027	Pre-release; certification path, not certification
NVIDIA GR00T / SONIC on Unitree G1	50 Hz motion-tracking controller with latent tokens	Open code (Apache 2.0), weights under NVIDIA Open Model License	Research platform; G1 checkpoints only
Tesla Optimus	Not publicly detailed	V3 reveal reported as delayed; late-2025 call targeted production start late 2026	I could not verify current status; treat as unconfirmed
Apptronik Apollo	Used with Gemini Robotics 1.5 in DeepMind’s cross-embodiment demo	Large funding rounds reported	Control stack not detailed publicly

Notice the pattern. Nobody publishes the interface between layers, the monitor’s thresholds or fault-injection results, which are exactly what a buyer needs. If you are evaluating a humanoid for a site, ask for those before asking for demos.

Trade-offs, Gotchas, and What Goes Wrong

Learned controllers fail silently. A QP that goes infeasible tells you. A neural policy given an out-of-distribution observation returns a confident wrong action. Defend with input range checks, ensemble disagreement or a learned novelty score, and with the independent monitor.

Latent interfaces hide intent. If the planner talks to the controller in latent tokens, a unit test cannot assert “the elbow must not exceed 120 degrees”. Put hard limits after the decoder, in joint space, where they can be checked.

Simulation exploits. RL policies find bugs in simulators: contact artefacts, unrealistically low friction or free energy. Symptoms include a gait that is suspiciously fast or a policy that works only at one control delay. The remedy is randomisation plus hardware tests that reproduce simulated metrics within a tolerance.

Estimator coupling. A controller that uses estimated base velocity and an estimator that uses the controller’s contact schedule form a feedback loop. When one degrades, the other amplifies it. Log estimator innovations and make the safety layer distrust the estimate when they spike.

Thermal and fatigue. Controllers tuned on a cold robot degrade at hour three. Test long duty cycles, not clips.

Teleoperation data contamination. Demonstrations collected through a latency-laden or low-fidelity teleop interface teach policies the operator’s compensation behaviour, not the task. Measure and record teleop latency with each episode.

Overlapping authority. The most common integration bug is two layers both believing they own the same degree of freedom, for example a manipulation policy commanding the torso while the locomotion policy balances it. Each joint group should have exactly one writer at a time, enforced by the arbitration layer.

Benchmark theatre. An eight-hour shift claim, a dishwasher-loading video or a parkour clip says little about mean time between falls. The metric to demand is falls or protective stops per hundred operating hours, with the task mix stated.

Practical Recommendations

If you are building a stack, start from the interfaces, not the models. Decide what each layer may command and at what rate, write that contract as a schema, and then pick implementations. Keep a QP or MPC layer where a hard constraint matters, adopt a learned whole-body policy where robustness to contact and ease of authoring matter, and keep the safety monitor non-learned in either case.

If you are buying a humanoid, treat the vendor’s safety architecture as the product. Ask what happens on a stale command, a lost IMU, a power loss and a person entering the work zone, and ask to see logs from a forced fault. For a deployment before ISO 25785-1 is published, document your own risk assessment against ISO 10218, ISO 13482 where relevant and your jurisdiction’s machinery rules, with the draft as a design reference only; I am not offering compliance advice.

A short checklist:

Write the layer contract: rates, units, frames, timeouts, and who owns each joint.
Run the deterministic monitor on separate hardware or an isolated core, and give it a path to damp or sit without the policy.
Measure end-to-end latency with hardware timestamps and randomise it in simulation.
Keep per-joint limits and gain scaling outside the policy, in configuration you can audit.
Replay logs and fault injections against a digital twin before every release.
Track falls and protective stops per 100 operating hours as a release gate.
Test thermal duty cycles of at least a full shift.
Version policies, gains, action scales and URDF together; deploy them as one unit.

Frequently Asked Questions

What are the layers of a humanoid robot control stack?

A typical stack has a semantic planner (language or VLA, 1 to 10 Hz), a visuomotor policy (50 to 200 Hz), a whole-body controller (500 Hz to 1 kHz, either QP/MPC or a learned policy), and joint servo loops (10 kHz and above). A deterministic safety monitor sits beside them with authority to damp or stop. Sensing flows up through a state estimator, and each layer sends targets only to the layer immediately below it.

What is whole-body control in a humanoid?

Whole-body control computes joint torques that achieve several simultaneous goals, such as hand position, torso posture and foot contact, while respecting the robot’s dynamics, friction limits and joint limits. Traditionally this is a prioritised quadratic program solved at 500 Hz to 1 kHz. Increasingly it is a neural network trained with RL to track motion references, as in Figure’s System 0 and NVIDIA’s SONIC.

Do humanoid robots use reinforcement learning or MPC?

Both. MPC with a simplified model (like a linear inverted pendulum) is common for gait planning and where hard constraints matter. Reinforcement learning, trained in massively parallel simulation, is now widely used for locomotion and whole-body tracking because it handles unmodelled contact well. Production designs often combine them: learned policies for motion, model-based or rule-based layers for constraints and safety.

What is a VLA model and where does it sit in the stack?

A vision-language-action (VLA) model takes camera images and a language instruction and outputs robot actions. It sits at the top or middle of the stack, running at low to moderate rates, and hands targets to faster controllers below. Examples include RT-2, OpenVLA and Gemini Robotics 1.5. It should not drive joints directly at balance-critical rates.

Are humanoid robots safe to work near people?

It depends on the platform and the deployment. The relevant standard, ISO 25785-1 for robots with actively controlled stability, was still in draft as of the sources I could verify, and vendors such as Agility describe independent safety controllers and certification paths for products not yet generally available. Until standards and independent evidence mature, plan site-specific risk assessments, exclusion zones and fall-zone marking.

Why use a real-time OS for humanoid robot control?

Balance loops have hard deadlines. A late command in a 1 kHz loop means a stale torque applied to an unstable system. PREEMPT_RT Linux with isolated cores, or a microcontroller, bounds scheduling jitter so the worst case, not the average, meets the deadline. Inference workloads run elsewhere, such as on a GPU module, so that neural network latency cannot delay the control loop.