Visual-Inertial Odometry: VINS-Mono vs OpenVINS vs ORB-SLAM3

A camera alone cannot tell you how big the world is, and an inertial measurement unit (IMU) alone drifts off by meters within seconds. Fuse them and each repairs the other’s blind spot: the IMU supplies metric scale, gravity direction and high-rate motion, while the camera supplies drift-free geometric constraints. That fusion is visual inertial odometry (VIO), and it is the default way a drone, headset or low-cost mobile robot knows where it is when GPS and lidar are unavailable.

The hard part in 2026 is no longer “does visual inertial odometry work.” It is choosing between three philosophies that disagree about how to spend a CPU budget: a Kalman filter that never re-linearizes the past (MSCKF and OpenVINS), a sliding-window nonlinear optimizer (VINS-Mono, OKVIS), or a full feature-based SLAM system with maps and loop closure (ORB-SLAM3 in its inertial modes). Each wins somewhere and fails somewhere else.

What this covers: IMU preintegration and camera models, the filter-versus-optimization argument at the level of the equations, how each open-source stack actually works, observability and gauge freedom, initialization, calibration with Kalibr, failure modes, carefully cited EuRoC and TUM-VI results, embedded compute budgets, and a decision matrix.

Context and Background

Visual-inertial navigation grew from two lineages. In the filtering lineage, Mourikis and Roumeliotis introduced the Multi-State Constraint Kalman Filter (MSCKF) at ICRA 2007. Its trick was to keep a short window of past camera poses in the state and to use each feature’s observations across that window as a constraint on poses, without ever putting the feature itself in the state. In the optimization lineage, Leutenegger and colleagues’ OKVIS (IJRR 2015) showed that a keyframe-based window of reprojection and inertial residuals, solved by Gauss-Newton style optimization, gives higher accuracy at higher cost. Qin, Li and Shen’s VINS-Mono (IEEE T-RO 2018) packaged that idea with robust initialization, loop closure and a four-degree-of-freedom pose graph, and became the most widely forked visual inertial odometry code base.

A third lineage is full SLAM. ORB-SLAM3 by Campos, Elvira, Gomez Rodriguez, Montiel and Tardos (arXiv 2007.11898, IEEE T-RO 2021) adds tightly coupled inertial terms to the ORB-SLAM feature map, plus a multi-map “Atlas” so that tracking loss starts a new map rather than ending the run. Its authors report average stereo-inertial errors of 3.6 cm on EuRoC and 9 mm on the TUM-VI room sequences, figures they describe as two to five times better than prior approaches; treat those as the authors’ claims under their evaluation protocol.

On the filtering side, the open research platform is OpenVINS from the Robot Perception and Navigation Group at the University of Delaware (Geneva, Eckenhoff, Lee, Yang and Huang, ICRA 2020). Its GitHub repository describes an extended Kalman filter fusing inertial data with sparse feature tracks, MSCKF-style sliding-window updates, optional SLAM landmarks, and online estimation of camera-IMU extrinsics, time offset, camera intrinsics and inertial intrinsics. The latest tag I could see on its releases page was v2.7, which the README news ties to May 2023; check the repository for anything newer.

All three families rest on the same two sensors. A typical rig pairs a global-shutter camera at 20 to 30 Hz with a MEMS IMU at 200 Hz or more. The EuRoC micro-aerial-vehicle dataset (Burri et al., IJRR 2016) uses 752×480 stereo at 20 Hz with a 200 Hz IMU; the TUM VI benchmark uses 1024×1024 fisheye global-shutter cameras at 20 Hz, a Bosch BMI160 IMU at 200 Hz, and motion-capture ground truth only in a single room. Those two datasets define most published numbers, so we will return to what they do and do not prove.

For readers building the wider estimation stack, this post pairs with our guides on factor graph SLAM with GTSAM and iSAM2 and on lidar-inertial odometry with LIO-SAM, FAST-LIO2 and KISS-ICP, which solve the same problem with a different exteroceptive sensor.

The Core Architecture of Visual Inertial Odometry

Direct answer: Visual inertial odometry estimates a sensor rig’s pose, velocity and IMU biases by combining high-rate inertial integration with low-rate camera feature observations. The inertial data gives metric scale and gravity, the camera bounds drift. Systems differ mainly in how they marginalize the past: an EKF, a sliding-window optimizer, or a keyframe map.

Figure 1: Common visual inertial odometry pipeline. Every stack has these blocks; they differ in what the estimator core is and whether the loop closure branch exists.

Figure 1 shows the shared skeleton. Camera frames feed a front end that tracks features, typically with Kanade-Lucas-Tomasi (KLT) optical flow (VINS-Mono, OpenVINS by default) or ORB descriptors (ORB-SLAM3). IMU samples are integrated between camera timestamps. Both streams enter an estimator core that maintains the state and consumes calibration priors. A loop-closure branch, present in VINS-Mono and ORB-SLAM3 but not in plain MSCKF, feeds global corrections back.

The state and the IMU model

A VIO state typically contains orientation (a rotation in SO(3)), position, velocity, gyroscope bias and accelerometer bias. A MEMS IMU measures angular rate and specific force corrupted by slowly varying biases and white noise:

measured gyro = true angular rate + gyro bias + white noise
measured accel = rotation-to-body of (true acceleration minus gravity) + accel bias + white noise

The biases evolve as random walks. Datasheet-style noise figures matter because they set the information weighting. For the BMI160 in the TUM VI rig, the dataset paper reports Allan-deviation values of about 1.4e-3 m/s^2/sqrt(Hz) accelerometer white noise, 8.6e-5 m/s^3/sqrt(Hz) accelerometer bias random walk, 8.0e-5 rad/s/sqrt(Hz) gyro white noise and 2.2e-6 rad/s^2/sqrt(Hz) gyro bias random walk. Using generic values instead of these is a common cause of overconfident or sluggish estimators, and the usual practice is to inflate datasheet or Allan numbers by a factor of several to absorb unmodeled effects.

IMU preintegration

Naively, every time the optimizer changes the starting orientation or bias of a keyframe interval, you would have to re-integrate hundreds of raw IMU samples. Preintegration, introduced by Lupton and Sukkarieh and put on the rotation manifold by Forster, Carlone, Dellaert and Scaramuzza (IEEE T-RO 2017, “On-Manifold Preintegration for Real-Time Visual-Inertial Odometry”), integrates the samples once in the body frame of the first keyframe. The result is a relative-motion measurement (delta rotation, delta velocity, delta position) independent of the absolute start state, with a covariance and a first-order Jacobian with respect to the biases.

When the bias estimate shifts slightly, the delta terms are corrected through those Jacobians rather than recomputed. Numerically: at 200 Hz IMU and 20 Hz camera, each keyframe interval holds about 10 samples; a window of 10 keyframes would otherwise mean re-integrating roughly 100 samples per iteration per residual evaluation. Preintegration collapses each interval to one 9-dimensional residual, which is why optimization-based visual inertial odometry fits on a CPU at all. VINS-Mono and ORB-SLAM3 both use preintegration; VINS-Mono uses a quaternion-based formulation, while ORB-SLAM3 follows the Forster on-manifold approach.

Filters handle inertial data differently. MSCKF propagates the state and covariance forward sample by sample (or over small batches), so no preintegration factor is needed, but the covariance propagation cost is paid on the whole filter state.

Camera models and the reprojection residual

A pinhole camera projects a 3D point in the camera frame to pixel coordinates via focal lengths and a principal point; real lenses add distortion. Common models in visual inertial odometry are radial-tangential (plumb bob), equidistant (Kannala-Brandt) for fisheye, and the Kalibr “omni”/double-sphere variants for wide field of view. ORB-SLAM3 explicitly supports pinhole and fisheye models; OpenVINS supports radtan and equidistant models in its config files; VINS-Mono supports pinhole and, through its camera module, several wide-angle models. Check each repository’s config documentation for the exact list before committing, since this changes between releases.

The visual residual is the difference between the observed pixel and the projection of the landmark through the current pose estimate, usually in the normalized image plane and weighted by a pixel-noise standard deviation (commonly about one pixel, typically divided by focal length for normalized-plane use). VINS-Mono parameterizes each feature by inverse depth in its first observing frame, which behaves well for distant points; MSCKF triangulates a temporary 3D point and eliminates it.

Why monocular vision needs the IMU

Monocular structure from motion recovers geometry only up to an unknown global scale. Acceleration integrated twice gives metric displacement, but only if scale, gravity direction and biases are known. Joint estimation resolves this: the IMU makes scale observable provided the motion includes enough acceleration excitation. A rig moving at constant velocity in a straight line leaves scale (and accelerometer bias) weakly observable, which is the root of several failure cases discussed below.

Filtering vs Sliding-Window Optimization: The Real Difference

Figure 2: Two ways to spend the same measurements. The filter does one linearized update per feature track; the optimizer iterates over a window and carries a marginalization prior.

Figure 2 contrasts the two inner loops. The left column is the MSCKF recipe; the right column is the keyframe-window optimization recipe.

The MSCKF update, step by step

Between camera frames, the filter integrates the IMU to propagate state and covariance. At each new image, it clones the current camera pose into the window, augmenting the state and covariance. As features are tracked, their observations accumulate. When a track ends (the feature leaves view or the track breaks), the filter triangulates the feature from all poses in the window that saw it, then forms the stacked reprojection residual. That residual depends on the poses and, through the feature position, on a nuisance variable.

The MSCKF trick is a null-space projection. Let the residual be approximated as r = H_x * dx + H_f * dp_f + n, where dp_f is the feature position error. Multiply by a matrix whose columns span the left null space of H_f. The feature term vanishes, leaving r_o = H_o * dx + n_o, a constraint purely on the poses. A single EKF update then uses it, and the oldest clone is dropped when the window is full. Cost is linear in the number of features and cubic only in the (small) state dimension, which is why a filter state of about 15 plus 6 per clone is cheap.

Concretely, with 11 clones and a 15-dimensional core state, the filter state is about 81 dimensions; each EKF update touches dense matrices of that size, tens of microseconds to a few milliseconds on a modern ARM core, depending on the number of tracked features. These are order-of-magnitude figures from the structure of the algorithm, not benchmark results.

The consistency problem and first-estimates Jacobians

An EKF linearizes once and never revisits. That creates the best-known theoretical weakness of filter-based visual inertial odometry: the linearized system gains observability it should not have. Hesch, Kottas, Bowman and Roumeliotis showed that a naive filter observes yaw, a direction that is truly unobservable in visual inertial odometry (gravity-aligned rotation and global position are gauge freedoms). Spurious information then shrinks the yaw covariance, so the filter becomes overconfident and, eventually, inaccurate.

The fix, due to Hesch and colleagues and extended by Li and Mourikis (IJRR 2013), is First-Estimates Jacobians (FEJ): always evaluate Jacobians at the first available estimate of each state, so the filter’s unobservable subspace matches the true system’s. OpenVINS offers FEJ, and the group’s research papers (Huang’s lab) explicitly frame OpenVINS as a platform for studying consistency. Observability-constrained filtering is the principled answer to “why does MSCKF drift in yaw on a straight line.”

The sliding-window optimizer

VINS-Mono and OKVIS instead keep a window of keyframes and minimize a sum of squared Mahalanobis residuals: a prior from marginalized states, preintegrated IMU factors between consecutive keyframes, and visual reprojection factors. The solver is a Gauss-Newton or Levenberg-Marquardt iteration (VINS-Mono uses the Ceres Solver; the README recommends Ceres 1.14.0 because 2.x had compile issues for that code base). Because all residuals in the window are re-linearized at each iteration, nonlinearity is handled more faithfully than in a one-shot EKF update.

When the window is full, the oldest keyframe or the least informative frame is marginalized using the Schur complement, producing a dense prior on the remaining states. VINS-Mono maintains a fixed number of keyframes (ten in the original paper’s setting) and, when the second-newest frame has small parallax, discards that frame instead of the oldest one, preserving a longer time baseline. The marginalization prior has its own consistency issue, essentially the same FEJ problem in disguise: after marginalization, the prior is linearized at old estimates. Production codes reuse the linearization point of the prior for exactly this reason.

Equivalence and the honest comparison

In the limit of one iteration and an identical window, the Gauss-Newton optimizer and the iterated EKF are equivalent; the practical difference is how many times linearization points move and how far back they can. The optimizer can correct errors in earlier window frames when later information arrives; the MSCKF cannot, except through its limited covariance coupling. Conversely, the filter delivers a pose at IMU rate with bounded latency, while an optimizer solves per keyframe and outputs high-rate poses by IMU forward propagation from the latest optimized state.

My reading of the literature, offered as opinion: on benchmark sequences the best tuned optimizers generally edge filters on accuracy at roughly 2 to 5 times the compute, while a well-tuned FEJ filter is much closer than folklore suggests, and its lower, more predictable latency matters for control loops. The question is rarely “which is more accurate” and usually “what does my latency budget and loop-closure requirement allow.”

The Three Stacks Compared

Figure 3: Visual-inertial initialization as a staged solve, with the ORB-SLAM3 style recovery branch that starts a new map on tracking loss.

VINS-Mono and VINS-Fusion

VINS-Mono is a monocular-plus-IMU estimator. The paper’s abstract lists robust initialization and failure recovery, tightly coupled nonlinear optimization of preintegrated IMU measurements and feature observations, loop detection with relocalization, and four-degree-of-freedom pose-graph optimization. Pose-graph optimization is restricted to four degrees of freedom (x, y, z, yaw) because roll and pitch are observable from gravity and so need no global correction. That is a direct application of observability: do not optimize what is already known.

Its repository is GPLv3, targets ROS Kinetic on Ubuntu 16.04, and records two features that matter in practice. Setting estimate_td to 1 enables online camera-IMU time offset estimation, tied to the IROS 2018 paper on online temporal calibration. Extrinsics are refined online from an initial guess, and setting estimate_extrinsic to 2 asks the system to estimate them from scratch by rotating the device for a few seconds. Loop closure uses DBoW2 bag-of-words place recognition. The README notes that in December 2017 map merging, pose-graph reuse, online temporal calibration and rolling-shutter support were added, and that in January 2019 the authors announced VINS-Fusion, which adds stereo, stereo-plus-IMU and mono-plus-IMU. Its iOS port is VINS-Mobile.

A practical warning: the main branch is built around older ROS and library versions. Teams in 2026 typically run community ports or VINS-Fusion forks on newer distributions, and the maintenance state of those forks varies. I did not verify any particular fork’s currency.

OpenVINS

OpenVINS is the MSCKF family’s research-grade implementation. The repository advertises: MSCKF sliding-window updates, optional SLAM landmarks (ArUco tags or sparse feature landmarks kept in the state), five feature representations, static and dynamic initialization, zero-velocity detection and update, a simulator, and online calibration of camera-IMU transform, time offset, camera intrinsics and IMU intrinsics including g-sensitivity. It is GPL-3.0 and ships evaluation scripts for EuRoC MAV, TUM-VI, UZH-FPV and KAIST Urban.

Its distinguishing design choice is that everything is a modular “type” in the state, so adding a calibration parameter is a bookkeeping exercise rather than a rewrite. It has no built-in loop closure in the core estimator; the group has published extensions that add maps and relocalization, but treat those as separate projects from the core release.

ORB-SLAM3 in visual-inertial mode

ORB-SLAM3 is a different animal: a full feature-based SLAM library. The abstract claims it is the first system to perform visual, visual-inertial and multi-map SLAM with monocular, stereo and RGB-D cameras using pinhole or fisheye models. Its visual-inertial component uses maximum-a-posteriori estimation, including in initialization, and a place recognition method with improved recall lets it start a new map after tracking loss and merge maps on revisit. The repository labels its README “V1.0, December 22th, 2021,” is GPLv3, and offers a closed-source commercial version through the authors. Shipped examples cover EuRoC and TUM-VI in monocular and stereo, with and without IMU.

The architectural consequence: ORB-SLAM3 maintains a persistent map of keyframes and map points, with local bundle adjustment tightly coupled to inertial factors, and covisibility-based loop closure and map merging. That makes it the only one of the three that naturally supports multi-session relocalization, and also the heaviest in memory, since the map grows with explored space.

Side-by-side summary

Property	OpenVINS	VINS-Mono / Fusion	ORB-SLAM3 inertial
Estimator	EKF, MSCKF window plus optional SLAM landmarks	Sliding-window nonlinear optimization	Local BA plus map, MAP initialization
Front end	KLT or descriptor tracking	KLT with outlier rejection	ORB features and descriptors
Loop closure	Not in core	DBoW2 plus 4-DoF pose graph	DBoW2 style place recognition, Atlas merge
Online calibration	Extrinsics, time offset, intrinsics, IMU intrinsics	Extrinsics, time offset	Not a headline feature; verify in config
Map persistence	None	Pose graph save and reuse	Multi-map Atlas
License	GPL-3.0	GPLv3	GPLv3 plus commercial option

Observability, Gauge Freedom and Initialization

What visual inertial odometry can and cannot know

Visual-inertial systems have four unobservable directions: global position (three) and rotation about gravity, i.e. yaw (one). Roll, pitch, velocity, scale, and IMU biases are observable given sufficient motion. This four-dimensional gauge is the reason the VINS-Mono pose graph is four-degree-of-freedom, why odometry drifts in position and yaw only, and why FEJ matters for filters.

Gauge freedom is also a practical engineering issue. When you fuse VIO with GPS or loop closures, you are adding constraints that fix the gauge. When you fuse it with a second odometry source, you must choose whether to treat their relative transform as a free parameter. Double-counting the gauge produces inconsistent fused estimates.

Degenerate motions

Several motion patterns make additional directions unobservable. Pure translation at constant velocity makes accelerometer bias and scale unobservable together with gravity; pure rotation without translation makes the scale and depth of features unobservable; and planar motion with constant body-frame acceleration makes some extrinsic parameters unobservable (Yang, Huang and colleagues have published on these degenerate cases for online calibration). The practical consequence: do not enable all online calibration states on a ground robot that only drives straight and turns on a plane. The estimator will fit nonsense into poorly constrained parameters and sometimes degrade.

Initialization

Initialization is the single most failure-prone stage. The standard staged approach, shown in Figure 3, starts with vision-only structure from motion up to scale, aligns that trajectory with preintegrated IMU measurements to solve gyroscope bias first (rotations are known from vision, so bias is a linear problem), then solves scale, gravity vector and velocities in a linear least-squares step, then refines the gravity direction by constraining its norm to 9.81 m/s^2 on a tangent-space sphere. VINS-Mono follows this recipe, requiring sufficient parallax and rotation before declaring the system initialized.

ORB-SLAM3 reformulates initialization as MAP estimation with an inertial-only stage followed by a joint visual-inertial bundle adjustment, which the authors credit for faster and more reliable convergence. OpenVINS offers both static initialization (rig at rest, estimate gravity and gyro bias from stationary IMU readings) and dynamic initialization for moving starts.

Two operational rules fall out. First, a drone should wiggle before takeoff; wait for the initialized flag rather than assuming it. Second, never initialize while a large moving object fills the image, because the structure-from-motion step will lock onto the wrong rigid body.

Calibration: Extrinsics, Time Offset and Kalibr

A visual inertial odometry estimator is only as good as the camera-IMU calibration it starts from. Three groups of parameters matter: camera intrinsics and distortion, the rigid-body transform between camera and IMU (extrinsics), and the time offset between camera timestamps and IMU timestamps.

The standard offline tool is Kalibr from ETH Zurich’s Autonomous Systems Lab, based on Furgale, Rehder and Siegwart’s continuous-time batch estimation work (IROS 2013). You record an AprilTag board sequence while exciting all six degrees of freedom, and Kalibr jointly estimates camera intrinsics, camera-IMU extrinsics and time offset using a B-spline trajectory. The TUM VI authors used the Kalibr toolbox for intrinsics and stereo extrinsics, and then optimized camera-IMU extrinsics jointly, which is a good model for a lab workflow.

Time offset

Time offset is underappreciated. A 5 ms offset on a rig rotating at 100 degrees per second is half a degree of apparent rotation error between the sensors; at a feature tracked 5 m away that is roughly 4 cm of mismatch, before any estimator noise. These are arithmetic illustrations, not measurements. The TUM VI paper itself corrects a 5.3 ms IMU readout offset, which shows how visible these effects are even on a purpose-built, hardware-synchronized rig. If you cannot hardware-trigger your camera from the IMU clock, enable online time-offset estimation (VINS-Mono’s estimate_td, OpenVINS’s calibration of camera-IMU time offset), but start from a reasonable measured offset because the online estimate is local.

Rolling shutter

Most consumer cameras are rolling shutter: each row is exposed at a slightly different time, so under fast motion straight lines bend. Global-shutter cameras avoid this and are the default recommendation for robots. If you must use rolling shutter, VINS-Mono’s changelog claims rolling-shutter support, and the OpenVINS group has published rolling-shutter-aware formulations; verify that your chosen release and config actually enable it, and calibrate the readout time.

Failure Modes and What Goes Wrong

Figure 4: A practical selection flow for visual inertial odometry stacks. Loop closure needs and CPU budget are the first two forks.

Low texture and repetitive texture

White walls, long corridors, glass, and fog give the front end too few trackable features. Filters and optimizers both degrade gracefully only if the IMU can carry through. A MEMS IMU can bridge perhaps a second or two before position error grows past usable levels: with an uncompensated accelerometer bias of just 0.05 m/s^2, position error grows as one half bias times t squared, which is about 0.1 m after 2 s and 2.5 m after 10 s. That arithmetic is exact for a constant bias; real behavior depends on how well the bias was estimated beforehand.

Motion blur and exposure

Long exposures under low light blur features and break KLT tracks. The fix is hardware: short exposure with higher gain, a global-shutter sensor, and auto-exposure bounds. The TUM VI rig sets both cameras to a shared exposure from a light sensor and records per-frame exposure, which supports photometric calibration in direct methods.

Dynamic scenes

A crowd, a moving vehicle or elevator filling the field of view violates the static-world assumption. RANSAC on the fundamental matrix and chi-square gating on reprojection residuals handle small fractions of outliers. When most of the image moves, a feature-based estimator may track the moving object and conclude the robot is moving. IMU residuals provide a consistency check but cannot fix it alone.

Pure rotation, constant velocity and hovering

As discussed, these motions weaken observability. A hovering drone is an interesting case: zero acceleration makes scale and accelerometer bias drift slowly; zero-velocity updates in OpenVINS help ground vehicles that stop but do not help a hover.

Inconsistency and overconfidence

Covariance from a visual inertial odometry estimator is often too small. For fusion with GPS or other sensors this matters: an overconfident visual inertial odometry will overpower a correct GPS fix. Inflate the visual inertial odometry covariance or use a robust kernel, and test with normalized estimation error squared (NEES) on a dataset with ground truth before trusting reported uncertainty.

Compute spikes

Optimization-based systems have variable solve time: a window that suddenly contains many new features or a loop closure triggering pose-graph optimization can cause latency spikes. Run the estimator thread at a priority separate from perception and cap solver iterations or time, as VINS-Mono does with a solver time budget in its configuration. Filters have flatter latency profiles.

Licensing

All three major open-source stacks discussed here are GPL-family licensed. For a commercial product, plan for that early: ORB-SLAM3 explicitly offers a closed-source commercial arrangement through its authors; for the others, consult counsel or consider permissively licensed alternatives such as Basalt (BSD-3, per its repository, which I did not re-verify for this post).

Benchmarks: Reading EuRoC and TUM-VI Carefully

What the datasets are

EuRoC MAV (Burri et al., IJRR 2016) provides 11 sequences from a micro aerial vehicle in a machine hall and a Vicon room, with stereo images at 20 Hz and IMU at 200 Hz, rated easy, medium and difficult by speed and lighting. TUM VI (Schubert et al., IROS 2018) provides 28 evaluation sequences, about 20 km total, across room, corridor, magistrale, outdoors and slides environments, with 1024×1024 fisheye global-shutter images at 20 Hz and a 200 Hz IMU. Its ground truth comes from a motion-capture system that covers only one room, so most sequences have ground truth only at the start and end; error is therefore measured as end-of-sequence drift, not absolute trajectory error along the whole path. The ORB-SLAM3 repository says the same for its TUM-VI evaluation.

What published numbers say

The ORB-SLAM3 authors report average stereo-inertial error of 3.6 cm on EuRoC and 9 mm on the TUM-VI room sequences (authors’ claim, from the abstract). The VINS-Mono paper reports root-mean-square absolute trajectory errors on EuRoC in the range of a few to tens of centimeters depending on the sequence. I am deliberately not reproducing a per-sequence table here: numbers vary with metric (absolute trajectory error versus relative pose error), alignment (se3 versus sim3), whether loop closure is on, whether results are median of several runs, and whether the monocular scale is aligned. A comparison across papers that mixes these is not valid.

Caveats that change the ranking

Loop closure on or off. EuRoC sequences revisit areas, so a system with loop closure can post better ATE than a pure odometry system without being a better odometry.
Tuning. Each paper tunes its own system; third-party reproductions often show different relative orderings.
Nondeterminism. Multi-threaded systems give different results on each run; report medians over at least five runs.
Dataset saturation. Centimeter-level numbers on EuRoC say little about robustness in your own environment, which has different texture, lighting, vibration and synchronization quality.

For decision-making, run the three stacks on your own data with ground truth from motion capture, a total station or RTK-GNSS, and report both drift per distance and failure rate.

Compute Budgets on Embedded Hardware

I could not verify third-party frame-time numbers for each stack on specific boards, so the following is structural reasoning, with any numbers labeled as illustrative.

A feature-tracking front end at 20 to 30 Hz on 640×480 images, with 150 to 300 features, is a small, deterministic cost on any Cortex-A class core; image pyramids and KLT scale with feature count. The estimator is where the families diverge. An MSCKF update costs roughly the number of features times the window length for the null-space projection, plus a dense covariance update of a few hundred squared entries. This is why OpenVINS and similar filters have been demonstrated on phone-class and small-computer hardware. A sliding-window optimizer needs several Gauss-Newton iterations over a window of ten keyframes with a few hundred landmarks; the Jacobian assembly and Schur complement dominate and typically need one or two full cores. ORB-SLAM3 adds ORB extraction and matching on every frame, local bundle adjustment on a separate thread, and a map whose size is unbounded, so memory and place-recognition queries scale with trajectory length.

Practical advice for embedded targets:

Budget two cores for a window optimizer plus one for the front end; a filter can run in one.
Reduce image resolution before reducing feature count; tracking quality is more sensitive to feature distribution than to raw pixel count.
Cap map size or use map-merge-and-prune settings for long-duration runs; ORB-SLAM3 memory grows with explored area.
Use a global-shutter camera with hardware trigger before spending effort on software time-offset estimation.
Pin threads and set real-time priority for the estimator; jitter in sensor timestamps is worse than a slightly slower solver.

A frequent ceiling is not compute but IO and timestamps. A USB camera with variable latency undermines any estimator regardless of CPU headroom.

Decision Matrix

Requirement	OpenVINS (MSCKF)	VINS-Mono / Fusion	ORB-SLAM3 inertial
Tightest CPU and latency budget	Best fit	Moderate	Heaviest
Global consistency, loop closure	Needs extension	Built in, 4-DoF graph	Built in, strongest
Multi-session map reuse	Not core	Pose-graph reuse	Atlas multi-map
Online calibration research	Best coverage	Extrinsics and time offset	Limited
Monocular plus IMU on a small drone	Good	Good	Good, heavier
Large environment, long run	Drifts without loop closure	Pose graph grows slowly	Map memory grows
Easiest to hack and extend	Modular state types	Moderate	Large C++ code base
Commercial licensing	GPL-3.0	GPLv3	GPLv3 plus commercial option

The filter-versus-optimization choice maps to one sentence: choose the filter when latency, determinism and calibration experiments dominate, choose the optimizer when peak odometry accuracy and loop closure matter, and choose full SLAM when the map itself is a product.

Practical Recommendations

Start from the sensors, not the algorithm. A hardware-synchronized global-shutter camera plus a decent MEMS IMU with logged Allan variance gets you more accuracy than any estimator swap. Spend your first week on calibration with Kalibr, a measured time offset, and proper noise parameters.

Then prototype with two stacks, not one. Run OpenVINS and either VINS-Fusion or ORB-SLAM3 on recordings from your own robot, including the worst conditions it will face: corridors, glare, vibration, a person walking through the scene. Score them on drift per distance, failure rate and worst-case latency rather than average ATE.

Turn on online calibration states one at a time, and only when the motion excites them. Fuse visual inertial odometry with absolute sensors (GPS, UWB, fiducials, lidar maps) in a downstream factor graph or filter so that the gauge is fixed explicitly, as discussed in our factor graph SLAM guide. If your robot is a fleet asset, health-monitor the estimator’s innovation statistics: rising reprojection residuals or time-offset drift can foreshadow a failing camera mount, a use case that connects naturally to remaining useful life prediction and condition monitoring.

Checklist:

Hardware trigger or measured timestamp offset documented.
Kalibr intrinsics, extrinsics and offset stored with the robot ID.
IMU noise parameters from Allan variance, inflated and recorded.
Initialization motion scripted and the initialized flag gated in the state machine.
Estimator covariance validated with NEES on ground-truth data.
Fallback behavior defined for tracking loss (hover, stop, or switch to wheel odometry).
License reviewed before shipping.

Frequently Asked Questions

What is visual inertial odometry?

Visual inertial odometry is the process of estimating a moving platform’s pose and velocity by fusing camera feature tracks with IMU measurements. The camera constrains drift through geometry, the IMU provides metric scale, gravity direction and high-rate motion between frames. It produces local, drifting estimates in a gauge-fixed frame; adding loop closure or absolute sensors turns it into visual-inertial SLAM or globally referenced localization.

Is MSCKF better than optimization-based VIO?

Neither dominates. MSCKF-style filters like OpenVINS offer lower, steadier compute and latency and excellent support for online calibration research, while sliding-window optimizers like VINS-Mono relinearize repeatedly and often achieve higher accuracy at higher cost. With First-Estimates Jacobians the accuracy gap narrows. Choose by latency budget, calibration needs and whether you need loop closure, then validate on your own data.

How does ORB-SLAM3 differ from VINS-Mono?

ORB-SLAM3 is a full SLAM system: ORB feature map, local bundle adjustment tightly coupled with IMU factors, MAP-based initialization, and a multi-map Atlas that survives tracking loss and merges on revisit. VINS-Mono is primarily a sliding-window odometry estimator with loop closure and a four-degree-of-freedom pose graph. ORB-SLAM3 is heavier and better for mapping; VINS-Mono is lighter and simpler to deploy for odometry.

Why does visual inertial odometry need an IMU?

A single camera cannot determine metric scale, and cameras fail briefly under blur, glare or lack of texture. The IMU provides acceleration, which after double integration yields metric displacement, and gives gravity so roll and pitch become observable. It also bridges short visual outages. The camera in turn bounds the IMU’s rapid drift. Together they make scale, velocity and biases observable under adequate motion.

What calibration does visual inertial odometry require?

You need camera intrinsics and distortion, the camera-to-IMU rigid transform, the time offset between sensor clocks, and IMU noise parameters. Kalibr estimates the first three from an AprilTag board recording; Allan variance of a static IMU log gives noise terms. Online estimation in VINS-Mono or OpenVINS can refine extrinsics and time offset, but needs a good initial guess and exciting motion.

Which datasets should I use to evaluate visual inertial odometry?

Use EuRoC MAV for drone-style motion with accurate ground truth, and TUM VI for fisheye, global-shutter and varied environments, remembering its ground truth covers only the start and end of most sequences. Add UZH-FPV for aggressive flight if relevant. Then record your own data. Report metrics consistently, with loop closure state, alignment type and repeated runs stated.