Ethos-U85 and Cortex-M85: How Arm’s TinyML NPU Actually Works

The Arm Ethos-U85 is the first microcontroller-class NPU that can execute transformer attention natively, and that single fact changes what “TinyML” can mean. Earlier Ethos-U parts were built around convolutions; anything resembling a matrix-matrix product or a transpose fell back to the host CPU, where a Cortex-M spent most of its cycles shuffling tensors. The U85 adds those operators in hardware, scales to 2,048 MACs, and can be attached to anything from a Cortex-M7 to a Cortex-A520.

That matters now because small language models and vision transformers are being squeezed into battery-powered devices, and the bottleneck there is rarely peak TOPS. It is memory placement, weight bandwidth, and whether the compiler can keep the whole graph on the accelerator. This post walks through the U85 microarchitecture, its pairing with the Cortex-M85 and Helium, the Vela compiler and memory modes, and a worked memory budget you can reuse.

What this covers: the Ethos-U55 to U85 lineage, MAC configurations and the arithmetic behind the TOPS claims, operator support, the Vela and TOSA toolchain with runnable commands, memory-mode trade-offs, shipping silicon, failure modes, and a decision matrix against Hailo, Coral and Jetson-class parts.

Context and Background

Arm’s Ethos-U family is a “microNPU”: a small neural accelerator designed to sit beside a Cortex-M core and share its memory system. The Ethos-U55 arrived with the Cortex-M55 and targeted always-on keyword spotting and small vision models. The Ethos-U65 extended the family to Cortex-A hosts and, per Arm, roughly doubled on-device ML performance. The U85 is the third generation, announced by Arm on 9 April 2024, and Arm describes it as its highest-performing and most energy-efficient Ethos NPU.

The headline numbers Arm publishes are consistent across its product page and newsroom: 128 to 2,048 MACs per cycle, 256 GOPS to 4 TOPS at 1 GHz, up to 4x the maximum performance of the U65, and 20% better energy efficiency than the U65. Arm also quotes up to 85% utilization on popular networks. Treat these as vendor claims: Arm does not publish the full benchmark set behind the utilization figure on the pages I reviewed, and no absolute power figure appears on the U85 product page.

The host side matters as much as the NPU. The Cortex-M85 is Arm’s highest-performance Cortex-M core: an Armv8.1-M processor that Arm says was the first Cortex-M to exceed 6 CoreMark/MHz and 3 DMIPS/MHz. With the optional Helium vector extension (also called M-Profile Vector Extension, MVE) Arm quotes roughly 4x the DSP and ML throughput of the Cortex-M7 and about 20% more vector performance than the Cortex-M55. Its memory system has ECC on the tightly coupled memories and the L1 cache, and it adds TrustZone and PACBTI (pointer authentication and branch target identification) for security.

Arm packages the pair in the Corstone-320 reference design: Cortex-M85, Ethos-U85, the DMA-350 controller and the Mali-C55 image signal processor, with firmware, drivers and example applications. For the wider picture of how this class of silicon compares to other options, see our TI MSPM0G5187 and TinyEngine NPU analysis and the TinyML with TFLite Micro, int8 and CMSIS-NN tutorial. The official Arm product page is the primary source for the numbers above: Arm Ethos-U85.

A caution on terminology. “Cortex-M85 plus Ethos-U85” is the Arm reference pairing, not a statement about which shipping chips use it. As the silicon section below shows, the first U85 microcontroller family I could verify pairs the NPU with Cortex-M55 cores, so the combination is cleaner as an architecture pattern than as a purchasable part number today.

Ethos-U85 Architecture: What Sits Between the MACs and the Memory

The Ethos-U85 is a command-stream-driven accelerator. A host core writes a pointer to a compiled command stream plus base addresses for weights, scratch and tensors; the NPU then fetches and executes the stream autonomously through its AXI ports, interrupting the host when the network finishes. Inside, a MAC array fed by a weight decoder produces accumulators, an elementwise and activation unit post-processes them, and a small local SRAM of 29 to 267 KB holds working data. Arm lists up to six 128-bit AXI5 interfaces for reaching system memory.

Figure 1: Ethos-U85 block structure and its attachment to a Cortex-M85 host. Unsupported operators run on the CPU through CMSIS-NN kernels.

Figure 1 shows the structure that matters for performance. Data flows from external memory through the weight decoder into the MAC array, results pass through the elementwise unit, and only then touch SRAM again. Every extra trip through SRAM or AXI is energy and latency, which is why the third generation focused less on adding MACs than on removing memory round trips.

MAC configurations and the arithmetic behind the TOPS

The U85 is a licensable IP block, so silicon vendors choose a MAC count between 128 and 2,048. The quoted throughput follows from simple arithmetic: a MAC counts as two operations (a multiply and an add), so 2,048 MACs at 1 GHz gives 2,048 x 2 x 10^9, about 4.1 TOPS, which Arm rounds to 4 TOPS. The smallest configuration, 128 MACs, gives 256 GOPS at the same clock. Arm separately states native support for 2:4 structured sparsity, which it says doubles throughput on networks pruned to that pattern.

Real products run far below 1 GHz. Alif’s Ensemble parts clock the U85 at 400 MHz and quote up to 204 GOPS. Working backwards, 204.8 GOPS / (2 x 0.4 GHz) is 256 MACs. Alif’s sources I reviewed do not state the MAC count, so treat 256 as my inference rather than a published specification. It illustrates the general rule: the headline 4 TOPS is a ceiling for a large, high-clocked, advanced-node configuration, while microcontroller silicon will sit one to two orders of magnitude lower.

Utilization is the other half. A 2,048-MAC array is only useful if the compiler can keep it fed. Depthwise convolutions, small-channel layers and attention with short sequences all leave MACs idle. Arm’s 85% utilization figure is a best case on popular networks; your model’s number could be far lower, and Vela’s performance report (shown later) is the way to find out before you have hardware.

Data types: int8 weights, int8 or int16 activations

The U85 executes integer networks only. Weights are int8; activations may be int8 or int16. The int16 activation path is the interesting one for audio and for transformer layers, where softmax inputs and layer-norm statistics have a dynamic range that int8 cannot hold without damaging accuracy. One reporting source says Arm is still evaluating shared-exponent formats for future versions; that is a stated direction, not a shipping feature.

The practical consequence is that there is no float fallback on the NPU. Anything you leave in float32 runs on the Cortex-M, where Helium helps but cannot approach NPU throughput. Quantization is therefore a prerequisite, and the choice of int8 versus int16 per tensor is a first-order accuracy decision. Our int4 vs int8 vs fp8 edge NPU quantization comparison covers the broader trade-offs.

The operators that make it a transformer NPU

Arm’s blog lists new operators relative to the U65: TRANSPOSE, GATHER, MATMUL, RESIZE BILINEAR and ARGMAX. MATMUL and TRANSPOSE are the pair that matter for attention, since computing Q times K-transpose and then multiplying by V needs a batched matrix product between two activation tensors, not just activation times a constant weight. GATHER covers embedding lookups. RESIZE BILINEAR is what segmentation decoders such as DeepLabV3 need, and Arm cites DeepLabV3 as newly supported in hardware.

Arm’s reported model ports go further. According to EE Times Asia’s coverage of the launch, Arm ported ViT-Tiny and TinyLlama-1.1B and estimated 8 to 10 tokens per second for TinyLlama depending on configuration. The Arm blog says TinyLlama mapped with no operator fallback to the CPU. These are launch-time Arm estimates on its own virtual platform, not measurements on shipping silicon, and I did not find an independent reproduction. The next section shows why the token rate is bounded by memory, not MACs.

Operator chaining and the weight decoder

Elementwise operator chaining lets the NPU fuse an elementwise operation (an add in a residual connection, say) with the operation before it, so the intermediate tensor never gets written to SRAM and read back. In a residual network, every skip connection saved this way removes one full tensor write and read. Arm lists chaining, a fast weight decoder, a more efficient MAC array and better elementwise throughput as the sources of the 20% energy gain over the U65.

The weight decoder reads compressed weights and expands them on the fly. Vela compresses weights offline, and the decoder inflates them between memory and the MACs, so compression reduces both flash footprint and bus traffic. The U85 has a standard and a fast decoder plus what Arm calls extended compression. Arm does not publish compression ratios; real ratios depend on how much structure and sparsity your trained weights have.

Host flexibility: from Cortex-M7 to Cortex-A

Earlier Ethos-U parts were driven by Cortex-M cores, with the U65 added for Cortex-A systems. Arm says the U85 works in the same system configurations as the U55 and U65 and can be driven directly from a Cortex-A host, with recommended hosts spanning Cortex-M7, M55 and M85 up to Cortex-A520 class cores. The practical benefit is software reuse: one command stream format and one compiler target across a range of SoCs, from always-on sensor hubs to application processors running Linux.

Cortex-M85 and Helium: The Host That Still Matters

An NPU does not remove the CPU from the critical path. The Cortex-M85 does the sensor acquisition, pre-processing (FFT, mel filterbanks, image resizing), post-processing (non-maximum suppression, decoding), the RTOS, networking, and every operator the NPU cannot run. Treating the host as an afterthought is the most common reason a “4 TOPS” design misses its latency budget.

What the M85 brings

The Cortex-M85 implements Armv8.1-M. Arm’s launch material claims it was the first Cortex-M to exceed 6 CoreMark/MHz and 3 DMIPS/MHz, and, with Helium, about 4x the DSP and ML processing of the Cortex-M7. Its memory interfaces matter for NPU co-design: four 32-bit data TCM (tightly coupled memory) interfaces and one 64-bit instruction TCM interface, all with ECC, plus an L1 cache with ECC and an AMBA 5 AXI main interface to external memory. An extra 32-bit AHB port lets an external DMA controller reach the TCMs concurrently with the core.

For security, Arm lists TrustZone for Armv8-M and PACBTI, which it positions as a route to PSA Certified Level 2. For an NPU deployment this matters twice: model weights are valuable intellectual property, and the NPU’s bus master must not become a way around the secure/non-secure split. How the NPU’s AXI master is firewalled is a system-integration decision made by the silicon vendor; the IP does not decide it for you.

Helium as the fallback engine

Helium (MVE) is a 128-bit vector extension with int8, int16 and int32 lanes, predication and low-overhead loops. CMSIS-NN uses it to accelerate convolution, fully connected, softmax and pooling kernels. Arm’s blog states that operators the U85 does not support can run on the Cortex-M through CMSIS-NN, and the Vela compiler partitions the graph accordingly.

The split is the design lever. Consider a vision transformer where Vela maps every attention block to the NPU but leaves a custom pre-processing step and a final post-processing step on the CPU. Those CPU segments run serially with NPU segments unless you pipeline frames, so a graph with many NPU/CPU boundaries pays a hand-off cost each time: cache maintenance, command-stream submission, and an interrupt. The fewer subgraphs Vela reports, the better.

Cache coherence and the hand-off

The M85 has an L1 data cache, and the NPU reads and writes memory over its own AXI master. If the CPU writes an input tensor into a cacheable region and starts the NPU without cleaning the cache, the NPU reads stale data. If the NPU writes an output and the CPU reads without invalidating, the CPU reads stale cache lines. Production firmware either places NPU buffers in non-cacheable regions, or performs explicit clean/invalidate (the standard CMSIS cache maintenance calls) around inference. This is a classic source of “works on the FVP, fails on silicon” bugs, because functional models are often coherent where hardware is not.

The Corstone-320 reference subsystem

Corstone-320 pairs the M85 with an optional Ethos-U85, the DMA-350, and the Mali-C55 ISP, and Arm supplies an SSE-320 example subsystem plus a Fixed Virtual Platform (FVP), a software model that lets you run firmware before silicon exists. The ExecuTorch Arm documentation names the Corstone SSE-320 FVP as the Cortex-M85 plus Ethos-U85 target and the SSE-300 FVP for Cortex-M55 plus Ethos-U55. Two caveats: Arm’s pages give no memory sizes for Corstone-320, and an FVP is functionally accurate but not cycle-accurate, so use it to validate correctness and to read Vela’s estimates, not to sign off latency.

The Software Stack: Vela, TOSA, TFLM and ExecuTorch

The Ethos-U85 is programmed through an offline compiler. The Vela compiler (PyPI package ethos-u-vela) takes a quantized network, decides which operators the NPU runs, schedules tiles through SRAM, packs and compresses weights, and emits a command stream. At the time of writing the latest release I could verify on PyPI is 5.2.0, uploaded 3 September 2026, and it supports the Ethos-U55, U65 and U85.

Figure 2: The Ethos-U85 compile and deploy flow. Vela emits a command stream plus packed weights, while unsupported operators stay on the CPU.

Vela accepts two inputs per its PyPI description: TFLite (now LiteRT) flatbuffers and TOSA flatbuffers. Its output is either a TFLite file in which accelerated regions are replaced by Custom operators, or a raw .npz containing the command stream and weight data. The network must have quantized activations and weights to be accelerated at all.

TOSA: the intermediate language

TOSA (Tensor Operator Set Architecture) is a small, well-specified operator set intended as a stable target between frameworks and accelerators. Arm lists TOSA support on the U85 product page. In the ExecuTorch Arm backend, the exported PyTorch graph is lowered to TOSA and serialized, and Vela then compiles the TOSA flatbuffer into an Ethos-U command stream. The benefit is precision: TOSA fixes the exact arithmetic (rounding, zero-point handling), so the framework-side quantized result and the NPU result can agree bit-for-bit more often than with ad-hoc operator mappings.

TFLM versus ExecuTorch

Two runtimes target the Ethos-U85. TensorFlow Lite for Microcontrollers (TFLM) is the established path: Vela rewrites the .tflite, and the Ethos-U kernel in TFLM hands the command stream to the Ethos-U core driver. ExecuTorch is PyTorch’s on-device runtime; its Arm Ethos-U backend is integer-only, supports post-training quantization (PT2E) and quantization-aware training, and is configured via an EthosUCompileSpec whose parameters include target (for example ethos-u55-128), system_config, memory_mode, and extra Vela flags. At launch Arm described ExecuTorch support as upcoming; the current ExecuTorch documentation now lists U55, U65 and U85 as supported targets. Our ExecuTorch on-device LLM serving post covers the runtime side in more depth.

Choosing between them is mostly about your training stack and operator coverage. TFLM wins on maturity and the amount of existing example code. ExecuTorch wins if your models start in PyTorch, especially transformers, because you avoid the PyTorch to ONNX to TFLite conversion chain where operator semantics drift.

Walk-through: Compiling a Model with Vela and Reading the Report

Installing and running Vela takes minutes, and its report is the most honest performance data you will get before silicon. The commands below use options documented on the Vela PyPI page: --accelerator-config selects the target, --config points to a configuration file such as Arm/vela.ini, --system-config and --memory-mode pick entries from that file, and --optimise chooses between inference time and memory size.

pip3 install ethos-u-vela            # needs Python 3.10 or later
vela --list-configs                  # inspect system configs and memory modes

# Compile an int8 TFLite model for a 256-MAC Ethos-U85
vela model_int8.tflite \
  --accelerator-config ethos-u85-256 \
  --config Arm/vela.ini \
  --system-config <name from --list-configs> \
  --memory-mode Shared_Sram \
  --optimise Performance \
  --output-dir out \
  --verbose-performance

The accelerator names follow the pattern ethos-u85-<macs>; verify the exact accepted values with vela --help for your installed version rather than trusting a blog post, including this one. The <name from --list-configs> placeholder is deliberate: system-config names are defined in the vela.ini you pass, and the bundled defaults describe generic reference systems, not your silicon.

Reading the summary

Vela prints a block that includes SRAM used, off-chip memory used, estimated cycles, and the estimated inference time, plus the fraction of operators mapped to the NPU versus the CPU. Three numbers deserve attention.

First, NPU operator count versus CPU operator count. Any CPU operator is a hand-off point and a latency risk. Second, the peak SRAM figure, because it tells you whether Sram_Only is feasible. Third, the per-operator cycle table from --verbose-performance, which separates compute-bound layers (MAC cycles dominate) from memory-bound ones (DMA cycles dominate). If the second group dominates, adding MACs buys nothing.

A minimal TFLM invocation

On the device, TFLM loads the Vela-processed .tflite, registers the Ethos-U custom operator, and allocates a tensor arena. Treat the following as a structural sketch; function names for the Ethos-U driver registration are in the TFLM and Ethos-U core driver repositories and may differ by version.

// Structural sketch - verify symbols against your TFLM / core-driver version
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"

alignas(16) static uint8_t tensor_arena[ARENA_BYTES]; // placed in NPU-reachable SRAM

extern const unsigned char g_model[];   // output of vela, in flash or DRAM

tflite::MicroMutableOpResolver<2> resolver;
resolver.AddEthosU();                   // routes ethos-u custom op to the driver
resolver.AddSoftmax();                  // any CPU fallback op you still need

tflite::MicroInterpreter interp(tflite::GetModel(g_model), resolver,
                                tensor_arena, ARENA_BYTES);
interp.AllocateTensors();
// fill interp.input(0)->data.int8, clean D-cache if the region is cacheable
interp.Invoke();                        // blocks until the NPU interrupt

Two details save days of debugging. The arena size must match what Vela reported for scratch, and the arena must live in memory the NPU can address over AXI; on many MCUs, DTCM is CPU-only. Second, the command stream and weights must be aligned as the driver requires, which is why linker-script placement of g_model is not optional.

Gating the build on the report

A useful CI step runs Vela on every new model checkpoint and fails the build if the CPU operator count rises, peak SRAM exceeds the budget, or estimated cycles regress beyond a threshold. Because Vela is deterministic, these checks are stable and cost seconds. They catch the common regression in which a harmless-looking model change (a new activation function, a reshape pattern) silently moves an operator off the NPU.

Memory Modes, Bandwidth and a Worked Budget

The U85’s compute is rarely the limit. Memory placement is, and Vela exposes it through three memory modes that the ExecuTorch Arm backend documents precisely.

Figure 3: The three Vela memory modes and where weights and scratch buffers are placed.

In Sram_Only, the scratch buffer and the network both live in on-chip SRAM: best performance, limited by SRAM size. In Shared_Sram, scratch lives in SRAM while the network (weights) sits in external memory such as flash or DRAM; it needs SRAM large enough for the largest intermediate tensor. In Dedicated_Sram, both scratch and network sit in external memory, with a dedicated on-chip SRAM acting as a fast scratch cache; per the ExecuTorch documentation it is available on U65 and U85 only, not U55.

A worked memory budget (illustrative numbers)

The model below is hypothetical, chosen to show the method. Suppose an int8 vision model has 3.2 MB of weights, a peak activation working set of 480 KB after Vela’s tiling, and a 224 x 224 x 3 int8 input of about 147 KB.

Sram_Only needs about 3.2 MB + 0.48 MB, around 3.7 MB of SRAM. That fits an Alif-class part with 8 MB of bulk SRAM but not a typical 1 MB MCU.
Shared_Sram needs about 0.5 MB of SRAM for scratch, with the 3.2 MB of weights streamed from MRAM or flash. That fits far more devices, and the cost is weight-fetch bandwidth.

Now estimate that bandwidth. If the network performs one inference per frame at 15 frames per second, the weights are read once per inference (assuming each weight is used once, which holds when tiling does not force re-reads). That is 3.2 MB x 15 = 48 MB/s of weight traffic, trivial for any bus. The picture changes completely for language models.

Why TinyLlama is bandwidth-bound

TinyLlama-1.1B has about 1.1 billion parameters. At int8 that is roughly 1.1 GB of weights, so it cannot fit in on-chip memory on any microcontroller; it needs external DRAM. Autoregressive decoding reads essentially all weights for each generated token. At Arm’s estimated 8 to 10 tokens per second, that implies about 1.1 GB x 8 to 10, or roughly 9 to 11 GB/s of sustained weight bandwidth, before counting the KV cache.

This is my arithmetic, not an Arm statement, but it explains the architecture. Arm’s list of six 128-bit AXI5 ports and its claim that the U85 tolerates high DRAM latency are the features that make the estimate plausible, and it explains why the weight decoder and compression matter: any compression directly cuts the bytes per token. It also shows why MAC count is almost irrelevant for this workload: 1.1 billion MACs per token at 10 tokens per second is about 11 GMAC/s, well within even a modest array. The bound is DRAM, which is why small language models on microcontroller-class parts typically use sub-billion-parameter models or aggressive quantization.

Bandwidth sanity checks you can run

For any memory mode, compute three bounds and take the largest. Compute-bound time is MACs per inference divided by (MAC count x clock x utilization). Weight-bound time is weight bytes divided by effective weight-bus bandwidth. Activation-bound time is bytes of tensors spilled out of SRAM divided by their bus bandwidth. If Vela’s cycle estimate is far above the compute bound, you are memory-limited and the fix is a bigger SRAM, better compression, or a smaller model, not more MACs.

Inference Lifecycle, Silicon, and How U85 Compares

What happens during one Invoke

Figure 4: One inference on the Ethos-U85. The CPU submits a command stream and sleeps or does other work until the completion interrupt.

The runtime submits the command stream and base pointers to the driver, which writes the NPU’s queue registers and starts it. The NPU fetches packed weights and the input tensor over AXI, runs fused operations in local SRAM, writes the output, and raises an interrupt. In this model the Cortex-M85 is free during NPU execution, so an RTOS can run other tasks, or you can pipeline frames: pre-process frame N+1 on Helium while the NPU runs frame N. Pipelining converts the CPU work from additive latency into overlapped throughput, but only if buffers are double-buffered and cache maintenance is correct.

Shipping silicon: what I could verify

On 12 August 2025, eeNews Europe reported that Alif Semiconductor had shipped what it called the first MCU with an Ethos-U85: the Ensemble E4, E6 and E8 family. Details from the reporting and Alif’s launch coverage:

The U85 runs at 400 MHz and delivers up to 204 GOPS in the shipping configuration. Alif mentions possible future configurations of 0.5, 2 and 3 TOPS, which are plans, not products.
The cores are Cortex-M55 (a 400 MHz high-performance core and a 160 MHz efficiency core), not Cortex-M85. The E6 adds one Cortex-A32 at 800 MHz and the E8 adds two. The E8 also carries two Ethos-U55 NPUs.
Memory is reported as 9.75 MB total SRAM (8 MB bulk plus TCMs) with MRAM of up to 5.5 MB in one source and 6 MB in another. The sources disagree, so check the datasheet.
Alif claims on-device small language model execution at 36 mW on the E4. I could not find the model, quantization or token rate behind that figure, so read it as a marketing data point.
Alif integrated the NPU by synthesizing the RTL directly, not by using Arm’s Corstone subsystem.

Arm’s own launch material named Alif and Infineon as early Ethos-U85 adopters. I could not verify any shipping silicon that pairs an Ethos-U85 with a Cortex-M85 as of this writing, though that is the Corstone-320 reference architecture and virtual platform. If you are choosing a part today, confirm the exact core, MAC count, clock and SRAM in the vendor datasheet. For a different class of embedded NPU, see our TI MSPM0G5187 NPU analysis.

U55, U65 and U85 side by side

Attribute	Ethos-U55	Ethos-U65	Ethos-U85
Generation	First	Second	Third
Typical host	Cortex-M	Cortex-M and Cortex-A	Cortex-M7 up to Cortex-A520 class
MAC range	Not verified here	Not verified here	128 to 2,048
Max throughput at 1 GHz	Not verified here	Not verified here	256 GOPS to 4 TOPS
Transformer MATMUL in hardware	No (per Arm launch claims)	No	Yes
Dedicated_Sram mode	No	Yes	Yes
Vela support	Yes	Yes	Yes

I left the U55 and U65 MAC and TOPS cells as “not verified” because the sources I could open this run did not state them in text. Arm’s relative claims, 4x peak performance and 20% better energy efficiency than the U65, are the verified comparison.

The decision matrix against other edge accelerators

The U85 competes less with Hailo, Coral or Jetson than with “do it on the CPU” and with other MCU NPUs. Still, engineers do ask. The matrix below is qualitative and based on class of device, not on benchmarks.

Need	Ethos-U85 class MCU	Hailo-class accelerator	Coral or Jetson class module
Battery life in months	Strong fit	Poor, needs a host SoC	Poor
Small transformers, SLMs under 1B parameters	Possible, bandwidth-bound	Better throughput, host required	Best throughput
Real-time hard determinism	Strong, MCU RTOS	Depends on host	Depends on OS
Software maturity	Vela, TFLM, ExecuTorch	Vendor toolchain	Vendor toolchain, CUDA on Jetson
BOM and board area	Single chip	Extra chip and memory	Module

For measured comparisons at the higher end, see our Hailo-10H vs Jetson Orin Nano benchmark. The honest framing: the U85 pushes MCU-class devices into models that previously needed an application processor, but throughput per watt gains do not make it a substitute for a 40 TOPS module.

Trade-offs, Gotchas, and What Goes Wrong

Vendor claims are ceilings. The 4 TOPS and 85% utilization figures apply to the largest configuration under favorable networks. Shipping MCUs run at 400 MHz with a fraction of the MACs. Request Vela-estimated cycles for your own model, not a peak table.

Silent CPU fallback. A single unsupported operator, such as an unusual activation or a dynamic reshape, forces a subgraph split. Each split adds a CPU segment and a hand-off. Watch the Vela operator counts in CI and prefer operators in TOSA’s set when designing the model.

Cache coherency bugs. As described earlier, stale data between the M85 L1 cache and the NPU’s AXI master produces nondeterministic wrong outputs that never appear on a coherent FVP. Test on hardware with the cache enabled.

Quantization accuracy cliffs. Int8 attention is fragile. Softmax, layer norm and GELU are the usual culprits, and int16 activations exist for a reason. Budget time for quantization-aware training or per-layer int16 on sensitive tensors. Do not assume that a model that works in float will survive PTQ.

Memory-mode lock-in. A model tuned for Sram_Only may not fit a lower-cost part with less SRAM, and Shared_Sram with slow flash can cut throughput several-fold depending on weight reuse. Decide the memory architecture before you pick the model, not after.

FVP versus silicon. The Fixed Virtual Platform validates function. It does not model your actual DRAM latency, bus contention with the camera or DMA, or thermal throttling. Latency sign-off needs a board.

Toolchain drift. Vela is on a rapid release cycle (4.5.0 in December 2025, 5.0.0 in March 2026, 5.1.0 in June 2026, 5.2.0 in September 2026 per PyPI). Pin the version, and recompile and revalidate when you upgrade, because scheduling changes can alter both memory use and cycle counts.

Practical Recommendations

Start from the model and the memory system, not from the TOPS number. Run your quantized model through Vela for two or three candidate MAC counts and memory modes, record the NPU/CPU operator split, peak SRAM and estimated cycles, and only then shortlist silicon. If the cycle estimate barely changes between 128 and 512 MACs, you are memory-bound and should spend die area and budget on SRAM and bandwidth instead.

Use the Cortex-M85 class host deliberately. Put sensor pre-processing on Helium, pipeline it against NPU execution, and keep NPU buffers in non-cacheable memory or wrap them in explicit cache maintenance. For language workloads, do the bandwidth arithmetic in this post first: parameters times bytes per weight times tokens per second gives the sustained DRAM bandwidth you need, and if the part cannot supply it, no compiler will help.

Treat the toolchain as part of the product. Pin the Vela version, gate CI on the report, and prefer TOSA-friendly operators.

Quantize to int8, with int16 activations on sensitive tensors, before measuring anything.
Compile with Vela for each memory mode and record SRAM, cycles and CPU operator count.
Compute weight-bandwidth and compute bounds; take the larger.
Verify cache coherency on real hardware with caches on.
Confirm core type, MAC count and clock in the vendor datasheet, not the IP brochure.
Validate latency and energy on a board, not on the FVP.

Frequently Asked Questions

What is the Arm Ethos-U85?

The Ethos-U85 is Arm’s third-generation microNPU, announced in April 2024. It scales from 128 to 2,048 MACs, giving 256 GOPS to 4 TOPS at 1 GHz, supports int8 weights with int8 or int16 activations, and adds hardware support for transformer operators such as MATMUL and TRANSPOSE. It can be driven by Cortex-M or Cortex-A hosts and is programmed through the Vela compiler.

How is the Ethos-U85 different from the Ethos-U55 and U65?

Arm claims up to 4x the maximum performance and 20% better energy efficiency than the U65, from operator chaining, a faster weight decoder and a better MAC array. The U85 adds MATMUL, TRANSPOSE, GATHER, RESIZE BILINEAR and ARGMAX for transformer and segmentation networks, supports 2:4 sparsity, and can be driven from a Cortex-A host. Dedicated_Sram mode is available on U65 and U85 but not U55.

Does the Ethos-U85 need a Cortex-M85?

No. Arm recommends hosts from Cortex-M7 and M55 through M85, and Cortex-A cores up to the A520 class. The Cortex-M85 pairing is Arm’s Corstone-320 reference design. Alif’s first shipping U85 microcontrollers, for example, use Cortex-M55 cores. The M85 is attractive because of its Helium vector unit and security features, but the NPU works with other hosts.

What does the Vela compiler do?

Vela is Arm’s offline compiler for Ethos-U NPUs. It takes a quantized TFLite or TOSA model, decides which operators run on the NPU, tiles tensors through SRAM, compresses weights, and emits a command stream plus packed weights. It targets the Ethos-U55, U65 and U85 and prints a performance and memory report that you can use for sizing before silicon exists.

Can the Ethos-U85 run a language model?

Small ones. Arm reported TinyLlama-1.1B mapped entirely to the NPU with an estimated 8 to 10 tokens per second, but that is an Arm estimate, and the weights need roughly 1.1 GB of external memory at int8, so decoding is bandwidth-bound at around 9 to 11 GB/s by my arithmetic. Alif claims small-language-model operation at 36 mW on its E4, without published model details.

Which frameworks support the Ethos-U85?

TensorFlow Lite for Microcontrollers is the long-standing path, using Vela-processed models. The ExecuTorch Arm Ethos-U backend now lists U55, U65 and U85, lowering PyTorch graphs through TOSA to Vela. Both require integer-quantized models; operators the NPU cannot run fall back to the CPU using CMSIS-NN kernels.