ONNX vs TFLite vs ExecuTorch vs Core ML: Picking an Edge Model Format in 2026

You trained a model that works. Now you have to make it run on a phone, a gateway, or a microcontroller you will never physically touch again, and the first real decision is not architecture or quantization scheme but format. The onnx vs tflite vs executorch question — plus Apple’s Core ML — is where most on-device projects quietly lose weeks, because the choice silently constrains which accelerators you can reach, how much accuracy you keep after quantization, and how many platforms one build can serve. The four runtimes overlap enough to feel interchangeable in a slide, and diverge enough in production that swapping one for another late is a rewrite, not a config change.

This is a practitioner’s comparison, not a feature grid copied from four homepages. The tension throughout is the same: portability across hardware you do not control versus peak utilization of hardware you do. Every runtime here sits somewhere on that axis, and the right pick is the one whose position matches your device fleet and your team’s source framework — not the one with the longest backend list.

What this covers: the four runtimes and their backend models, how each exports from PyTorch or TensorFlow, operator-coverage and quantization gaps, a decision matrix, the failure modes that bite in production, and concrete picks by scenario.

Context and Background

The uncomfortable truth of edge AI is that training frameworks and inference runtimes are different software with different priorities. PyTorch and TensorFlow optimize for flexible, eager, differentiable computation on fat GPUs. An on-device runtime optimizes for a fixed graph, a tiny binary, a battery budget, and an accelerator with a fraction of the operator coverage of a datacenter GPU. Between those two worlds sits a conversion step, and that step is where portability promises meet hardware reality.

For years the default was blunt: train in TensorFlow, convert to TFLite for mobile; train in PyTorch, export to ONNX and hope the operators survived. That worked because the models were small and the operator sets were stable. In 2026 the picture is messier and, in some ways, better. TFLite has been rebranded LiteRT and explicitly opened to models authored in PyTorch, JAX, and Keras, not just TensorFlow (Google’s announcement frames it as a multi-framework runtime). PyTorch shipped ExecuTorch as a native on-device path that skips ONNX entirely. ONNX Runtime matured its execution-provider system into a genuinely cross-platform dispatch layer. And Apple’s Core ML tooling learned to ingest PyTorch models directly through a unified conversion API, dropping the old ONNX middle step.

The result is four credible runtimes with real overlap and real edges. If you are architecting the surrounding system rather than just the model, our edge MLOps pipelines guide covers the versioning and rollout machinery these formats plug into. This post stays tight on the format decision itself.

One framing helps before we get into specifics. A model format is really three coupled things wearing one name: a serialized graph description, an operator set that the graph is allowed to reference, and a runtime that knows how to execute that operator set on real silicon. When people say “we use ONNX” or “we ship TFLite,” they are casually collapsing all three. The problems in this article almost always live in the seam between the operator set the format defines and the operators a specific accelerator backend actually implements. The graph converts cleanly, the file loads, and only at execution does the mismatch surface — as a CPU fallback, a refused conversion, or a quiet numerical divergence. Keeping those three layers distinct in your head is the single most useful mental model for reasoning about any of the four runtimes below, and it explains why two teams shipping “the same format” can get wildly different device performance.

The Four Runtimes and Their Backends

Short answer: pick ONNX Runtime for maximum hardware portability from any source framework, LiteRT when Android and iOS are the primary targets, ExecuTorch when you are a PyTorch shop that wants a native path with no lossy conversion, and Core ML when you are Apple-only and want the Neural Engine at full tilt. Each runtime reaches hardware through a pluggable backend abstraction — execution providers, delegates, or backends — and that abstraction, more than the file format, determines what you actually get on device.

Figure 1: The four export-and-deploy paths. A model authored in PyTorch, TensorFlow, or JAX is exported to a runtime-specific artifact — an ONNX graph, a LiteRT flatbuffer, an ExecuTorch .pte, or a Core ML .mlpackage — and each runtime then dispatches that artifact onto CPU, GPU, DSP, or NPU through its own backend layer. Long description: the diagram traces four parallel lanes from training frameworks on the left, through per-runtime export artifacts, into the four runtimes, and out to the hardware classes each one targets, showing that the divergence happens at the runtime and backend stage rather than at authoring.

ONNX Runtime: portability through execution providers

ONNX Runtime is the closest thing to a universal edge target, and its power comes from the execution-provider (EP) abstraction. An ONNX graph is hardware-agnostic; the runtime partitions it at load time and assigns each subgraph to the best available EP. On a Qualcomm device that is the QNN EP hitting the Hexagon NPU; on Apple silicon it is the CoreML EP reaching the Neural Engine; on Windows it is DirectML across any DirectX 12 GPU; on generic mobile and WebAssembly it is XNNPACK; and anything unassigned falls to the built-in CPU EP backed by MLAS. The current line is the 1.24.x series, and it is worth noting the churn even here — the ArmNN EP was removed, with Qualcomm users pushed toward QNN and Arm CPU users toward the KleidiAI-accelerated MLAS path.

The trade-off is that portability is only as deep as each EP’s operator coverage. The DirectML EP, for example, tracks up to a specific ONNX opset and leaves a handful of operators unimplemented; when your graph uses one, that node quietly reassigns to CPU. ONNX Runtime almost never fails outright — it degrades. That is a feature for shipping and a trap for performance, and it is the single most important thing to internalize about the runtime.

It is also worth being precise about how “one artifact everywhere” actually plays out. The ONNX file is portable, but the build of ONNX Runtime is not monolithic — you compile or select a package with the specific EPs you need, and the QNN, DirectML, and CoreML providers ship as separate builds tuned per platform. So the promise is one model graph, many platform-specific runtime binaries, each dispatching that graph differently. In practice this is a strong position for a mixed fleet, because your model-training and export pipeline stays single-source while your device builds specialize underneath. The cost is that you validate on every target build, since the same graph can hit different operator coverage on QNN than it does on DirectML or XNNPACK. Teams that treat “it passed on the CPU EP” as sufficient are the ones surprised in the field.

LiteRT (formerly TFLite): mobile-first with delegates

LiteRT is the 2024 rebrand of TensorFlow Lite, and the rename signals intent: it is no longer TensorFlow-only. It now accepts models authored in PyTorch, JAX, and Keras, converted into the same compact flatbuffer format that has shipped on billions of phones. Its backend abstraction is the delegate. The GPU delegate runs float32, float16, and 8-bit quantized models on both Android and iOS GPUs. The Core ML delegate routes to Apple’s Neural Engine on A12 and newer SoCs. Crucially, the older NNAPI and Hexagon delegates are now deprecated and no longer maintained — a meaningful shift, because a lot of legacy Android acceleration advice still points at NNAPI. On current Qualcomm silicon the recommended path is the vendor’s own LiteRT accelerator integration rather than the retired NNAPI route.

LiteRT’s strength is maturity on mobile: the converter is battle-tested, the runtime is tiny, and the tooling for post-training quantization is the most mature of the four. Its weakness is that it is still fundamentally a mobile and embedded runtime; it is not where you go for a Windows desktop GPU or a datacenter fallback.

ExecuTorch: PyTorch-native, ahead-of-time export

ExecuTorch is PyTorch’s own answer to on-device inference, and it reached its 1.0 release — a signal of production readiness after a long preview. Its pitch is directness: you export a PyTorch model ahead of time into a .pte program without an intermediate format like ONNX, so the operators you trained with are the operators you ship. The backend list is the broadest of the four by raw count — XNNPACK for CPU everywhere, plus Vulkan, Qualcomm, MediaTek, and Samsung Exynos on Android; Core ML and Metal on Apple; OpenVINO and experimental CUDA on Linux and Windows; and ARM Ethos-U, NXP, and Cadence DSP down at the microcontroller tier.

The catch, and it is a real one, is churn. ExecuTorch moved fast to get here, some backends are explicitly experimental, and a few (MPS on Apple, for instance) are already marked deprecated. The 1.0 label buys stability at the core, but the edges of the backend matrix still move release to release. For a PyTorch-first team the payoff — no conversion tax, no operator-translation surprises — is often worth managing that churn.

The architectural detail that distinguishes ExecuTorch from the others is that export happens ahead of time and produces a fully-specified program, not a graph the runtime interprets loosely. During export, ExecuTorch traces the model, lowers it through a defined intermediate representation, and delegates supported subgraphs to backends at that stage — so the partitioning decision that ONNX Runtime and LiteRT make at load time, ExecuTorch largely makes at build time. The upside is predictability: what runs on the accelerator versus the CPU is decided and inspectable before the app ever ships, which is exactly what you want for a device you cannot debug remotely. The downside is that a model with dynamic control flow or shapes the exporter cannot capture will fight you at export rather than degrading gracefully at runtime, and untangling those export errors is its own skill.

Core ML: Apple-only, but the Neural Engine at full depth

Core ML is the outlier: single-vendor, and that is the point. On Apple hardware nothing else reaches the Apple Neural Engine (ANE) as cleanly. The modern coremltools unified conversion API ingests PyTorch models directly — the old two-step dance through ONNX is gone — and produces an .mlpackage that the OS schedules across CPU, GPU, and ANE. The compute_units parameter lets you constrain that placement, defaulting to using all three. On iOS 18 and macOS 15 the tooling added an int8-activation, int8-weight (W8A8) path that exploits the faster int8 compute on A17 Pro and M4-class Neural Engines, plus per-block 4-bit weight quantization tuned for GPU-resident large models.

The price of that depth is total lock-in. A Core ML model runs on Apple platforms and nowhere else. If your fleet is iPhones and iPads and you want every last millisecond out of the ANE, Core ML wins decisively. If you also ship Android, you now maintain two model pipelines — which is exactly the tension the other three runtimes try to dissolve.

Choosing by Constraint

The runtime you can use is bounded first by your source framework and second by your target hardware. Those two constraints usually collapse the field to one or two real options before performance ever enters the conversation. Figure 2 shows how ONNX Runtime resolves the hardware half of that decision at load time, and Figure 3 turns the whole thing into a decision tree.

Figure 2: ONNX Runtime EP dispatch. At load, the graph partitioner walks each node against the ordered list of registered execution providers, assigns supported subgraphs to the preferred accelerator EP, and routes anything unsupported down the fallback chain to XNNPACK or the MLAS CPU EP. Long description: a top-down flow from a loaded ONNX model through the partitioner into a decision on whether the preferred EP supports each node, branching to an accelerator assignment or a fallback that ends at the default CPU path, with all fused subgraphs converging to run on device.

Export paths. From PyTorch you have all four doors open: torch.onnx.export to an ONNX graph, the LiteRT converter via the PyTorch-to-LiteRT flow, ExecuTorch’s ahead-of-time export to .pte, or coremltools straight to .mlpackage. From TensorFlow, Keras, or JAX the natural path is LiteRT, with ONNX available through community converters. This asymmetry matters: a PyTorch team can pick on merit, while a TensorFlow team is gently funneled toward LiteRT or ONNX. If you are weighing this for LLM-class models specifically, the calculus shifts again — see our breakdown of on-device LLM runtimes, where llama.cpp and MLC change the trade space.

Operator coverage gaps. This is where conversions die. Every runtime supports a core set of operators fully and a long tail partially or not at all. A custom attention variant, an exotic activation, a dynamic-shape control-flow block — any of these can hit a wall. The runtimes handle the wall differently: ONNX Runtime and LiteRT reassign the unsupported node to CPU and keep running; ExecuTorch, because it exports ahead of time, surfaces the gap at export as a partitioning boundary; Core ML may refuse the conversion outright or fall back to CPU at runtime. The direct-export runtimes fail earlier and louder, which is usually what you want.

Quantization. All four support INT8 post-training quantization (PTQ) and, with more work, quantization-aware training (QAT). The frontier is INT4 and mixed schemes. Core ML’s per-block 4-bit weight quantization and W8A8 activation path are the most hardware-specific, tuned to exact Apple silicon generations. LiteRT has the most mature and forgiving PTQ tooling. ONNX Runtime supports INT8 broadly but the accuracy you keep depends heavily on which EP executes the quantized ops. ExecuTorch inherits PyTorch’s quantization flows, which are powerful but still stabilizing. The rule holds across all four: quantization is where silent accuracy loss hides, and you validate on-device, not on the host.

Figure 3: Choosing by source framework and target OS. Starting from whether the model was authored in PyTorch or a TensorFlow-family framework, the tree branches on target device — Apple-only, PyTorch-native edge, cross-platform, or Android-and-iOS mobile — and lands on the runtime that fits each combination. Long description: a branching diagram beginning with the source-framework question, splitting into PyTorch and TF-or-JAX-or-Keras paths, each of which then splits by target device into Core ML, ExecuTorch, ONNX Runtime, or LiteRT before converging on shipping a runtime.

Decision matrix

Dimension	ONNX Runtime	LiteRT (TFLite)	ExecuTorch	Core ML
Source framework	Any (PyTorch, TF, JAX via converters)	TF, Keras, JAX, PyTorch	PyTorch native	PyTorch, TF via coremltools
Target OS / HW	Windows, Linux, Android, iOS, web, edge	Android, iOS, embedded	Android, iOS, desktop, MCU	Apple only
Backends / delegates	EPs: QNN, CoreML, DirectML, XNNPACK, CPU/MLAS	Delegates: GPU, Core ML (NNAPI/Hexagon deprecated)	XNNPACK, Vulkan, Qualcomm, MediaTek, Core ML, Ethos-U	CPU, GPU, ANE
Quantization	INT8 broad, INT4 EP-dependent	INT8/INT4 PTQ, most mature tooling	Inherits PyTorch PTQ/QAT	INT8, W8A8, per-block INT4 (iOS 18+)
Operator coverage	Broad, EP-dependent gaps	Broad on mobile ops	Follows PyTorch ops, partitions on gaps	Good, may refuse unsupported ops
Binary size	Moderate, trims per EP	Smallest, mobile-tuned	Small, tunable per backend	Bundled with OS frameworks
Portability	Highest	Mobile/embedded only	Broad, PyTorch-only source	None (Apple-locked)
Maturity	High, stable EP model	Very high on mobile	1.0, core stable, edges churn	Very high on Apple

The matrix rewards reading down the two rows that bind hardest for your project — usually target OS/HW and source framework — and only then comparing quantization and coverage among the survivors.

A subtlety the matrix flattens is maturity-per-backend rather than maturity overall. ONNX Runtime as a project is mature, but a given execution provider may be new; ExecuTorch is 1.0 at the core while specific backends are experimental; LiteRT is extremely mature on the GPU delegate but its story on retired delegates like NNAPI is now a liability, not an asset. So “maturity” is never a single number you can trust from a comparison table — it is a property of the exact backend-plus-operator-set combination your model will actually exercise. The disciplined move is to enumerate the specific operators your architecture uses, then check each against the maturity of the one backend you intend to run them on. That is more work than reading a row, and it is the work that separates a deployment that holds up from one that regresses the first time a device model or OS version shifts underneath it.

Trade-offs, Gotchas, and What Goes Wrong

The failure modes are more instructive than the feature lists, because every runtime here works in the demo and breaks in the same few places under real fleets.

Figure 4: The CPU-fallback failure path. When a model contains a custom or newly introduced operator the accelerator backend does not support, the graph partition breaks at that node, tensors are copied back to CPU, the op executes there, and results are copied back — and the memory transfers, not the op itself, are what spike latency. Long description: a flow showing an unsupported or low-precision operator causing either a partition break with round-trip CPU copies and latency spikes, or silent accuracy drift, both resolved by profiling and pinning ops to a single backend.

The most common and most expensive failure is the unsupported-operator fallback to CPU. Your model runs, the accuracy is fine, and the latency is three times what the benchmark promised — because one operator in the middle of the graph is not supported on the NPU, the partition breaks there, and every inference now pays for two tensor copies across the memory bus. The op itself is cheap; the round trip is not. This is invisible unless you profile per-node placement, and it is the number-one reason edge inference misses its latency budget.

Second is silent accuracy drift after quantization. INT8 and especially INT4 change numerics, and the loss is rarely uniform — it concentrates in specific layers and specific inputs. A model that scores 94% on your host validation set can quietly ship at 88% on device because the quantized path executes differently on the target accelerator than in your simulator. You catch this only by evaluating the actual on-device build against a held-out set, which most teams skip until a user complains.

Third are the runtime-specific traps. Core ML lock-in is total: the model is Apple-only, so a business decision to add Android is a from-scratch second pipeline. ExecuTorch churn means a backend that worked in one release can shift behavior in the next; pin your version and read the changelogs. Conversion brittleness haunts every ONNX and LiteRT path — a converter that succeeds today can break when you bump the source framework, because the operator set moved underneath it. The lesson across all three: treat conversion as versioned, tested infrastructure, not a one-time script.

Practical Recommendations

Pick by the constraint that binds hardest, not by the longest feature list.

Cross-platform, mixed fleet (Android + iOS + Windows + web): ONNX Runtime. One artifact, EP-based dispatch to whatever accelerator each device offers, graceful CPU fallback everywhere. Accept that you will profile per-node placement to avoid silent CPU spillover.
Android-first mobile, TensorFlow or Keras shop: LiteRT. Most mature converter and quantization tooling, smallest binary, GPU and Core ML delegates cover both mobile OSes. Do not rely on NNAPI or Hexagon — they are deprecated.
Apple-only, latency-critical: Core ML. Nothing else reaches the ANE as deeply; use the W8A8 and per-block INT4 paths on iOS 18+ hardware. Accept the lock-in as the cost of peak performance.
PyTorch-native team wanting no conversion tax: ExecuTorch. Direct ahead-of-time export, broadest backend count, MCU-to-desktop reach. Pin versions and budget for backend churn.

Pre-ship checklist: (1) confirm every operator lands on the accelerator, not CPU, via per-node profiling; (2) validate quantized accuracy on the actual device build, not the host; (3) pin the runtime and converter versions in CI; (4) measure cold-start and binary size on the lowest-end device in the fleet; (5) if you target more than Apple, avoid single-vendor formats. For microcontroller-class targets specifically, the TensorFlow Lite Micro on ESP32 guide walks the constrained end of this spectrum.

Frequently Asked Questions

Is ONNX Runtime always the safest cross-platform choice?

For breadth of hardware it usually is, because the execution-provider model lets one ONNX artifact dispatch to Qualcomm, Apple, DirectML, or CPU without re-exporting. The caveat is that portability is only as good as each EP’s operator coverage, and unsupported nodes silently fall back to CPU. If your fleet is genuinely one vendor — all Apple, say — a native runtime like Core ML will beat ONNX Runtime on peak performance. ONNX Runtime wins when you cannot predict or control which hardware each unit will have.

What changed when TFLite became LiteRT?

The rename in 2024 marked a shift from a TensorFlow-only runtime to a multi-framework one that accepts PyTorch, JAX, and Keras models alongside TensorFlow. The flatbuffer format and the tiny mobile runtime are the same proven code. The practical 2026 change to watch is that the NNAPI and Hexagon delegates are deprecated; current Android acceleration goes through the GPU delegate or vendor-specific integrations, so older tutorials pointing at NNAPI are now stale advice.

Should a PyTorch team use ExecuTorch or ONNX Runtime?

It depends on portability needs. ExecuTorch gives you a native path with no lossy conversion — the operators you trained are the operators you ship — and the broadest backend count, which is ideal if you are PyTorch-first and want tight control. ONNX Runtime adds a conversion step but buys the widest hardware reach and a more stable, longer-lived runtime. If you deploy to many vendors, ONNX Runtime; if you want zero conversion tax and can absorb some backend churn, ExecuTorch.

How much accuracy do I lose to INT8 or INT4 quantization?

There is no universal number — it depends on the model, the layers, and the target accelerator. INT8 post-training quantization often costs low single-digit percentage points; INT4 can cost more and concentrates loss in sensitive layers. Core ML’s per-block INT4 and LiteRT’s mature PTQ tooling mitigate this, but the only reliable answer comes from evaluating the quantized, on-device build against a held-out set. Never trust host-side quantization accuracy as a proxy for device behavior.

Can Core ML models run on Android at all?

No. Core ML is an Apple-only format and runtime; there is no supported path to run an .mlpackage on Android. If you need both platforms you either maintain two pipelines — Core ML for Apple, LiteRT or ONNX Runtime for Android — or you standardize on a cross-platform runtime and give up some Apple-specific ANE performance. This lock-in is the central trade-off of choosing Core ML: maximum depth on one platform, zero reach beyond it.

Which runtime has the smallest binary footprint?

LiteRT is the smallest by design, built from the start for mobile and embedded size budgets. ExecuTorch is also small and tunable, since you link only the backends you use. ONNX Runtime is moderate and trims down when you build with a subset of execution providers. Core ML carries no app-side runtime cost in the usual sense because the framework ships with the OS. For the tightest microcontroller targets, LiteRT for Microcontrollers and ExecuTorch’s MCU backends are the realistic contenders.

ONNX vs TFLite vs ExecuTorch vs Core ML (2026): Edge Format Pick