Keyword Spotting on Microcontrollers: Building a TinyML Wake-Word Pipeline with MFCC Features, a DS-CNN and INT8 Quantization

Keyword Spotting on Microcontrollers: Building a TinyML Wake-Word Pipeline with MFCC Features, a DS-CNN and INT8 Quantization

Keyword Spotting on Microcontrollers: Building a TinyML Wake-Word Pipeline with MFCC Features, a DS-CNN and INT8 Quantization

A 12-class keyword model that scores above 94% on a benchmark test set can still wake your device 20 times an hour in a kitchen. The model is rarely the problem. A keyword spotting microcontroller pipeline lives or dies on four things the accuracy number hides: an audio front end that matches training bit for bit, a streaming loop that reuses work between hops, posterior smoothing that turns noisy frame scores into decisions, and a power design that keeps the expensive parts asleep. This matters now because the toolchain moved. TensorFlow Lite for Microcontrollers is now LiteRT for Microcontrollers, the tf.lite Python module is being deprecated, and MLPerf Tiny added a streaming wake-word test in 2025. You will leave with runnable feature-extraction, training and INT8 conversion code, a C++ inference loop, verified reference numbers, and illustrative RAM, flash and power budgets you can adapt.

What this covers: Speech Commands v2, MFCC framing math, DS-CNN design, Keras training, full-integer quantization, LiteRT for Microcontrollers with CMSIS-NN and ESP-NN, streaming inference, false-accept tuning, memory budgets, MLPerf Tiny and duty-cycled always-on audio.

Context and Background

Keyword spotting (KWS) means detecting a small, fixed vocabulary of words in a continuous audio stream. A wake word such as “Hey device” is the extreme case: one positive class against everything else a microphone will ever hear. The task is old, but its microcontroller form crystallised in 2017 and 2018 around two papers.

The first is the Google Speech Commands dataset by Pete Warden, described in arXiv 1804.03209. Version 0.02 contains 105,829 one-second utterances from 2,618 speakers, recorded as 16 kHz, 16-bit mono PCM and released under CC BY 4.0. It covers 35 words: 20 core words (the digits plus yes, no, up, down, left, right, on, off, stop, go) and auxiliary words such as “marvin”, “bed” and “wow”. The paper’s own baseline reaches 88.2% top-one accuracy when training and testing on v2.

The second is “Hello Edge: Keyword Spotting on Microcontrollers” by Zhang, Suda, Lai and Chandra at Arm (arXiv 1711.07128). It compared DNN, CNN, CRNN, LSTM and depthwise separable CNN (DS-CNN) models under explicit memory and compute budgets. The small DS-CNN reached 94.4% test accuracy on 12 classes with 38.6 KB of memory and 5.4 million operations per inference. The authors also deployed a quantized model on an STM32F746G Cortex-M7 board using CMSIS-NN, the Arm library of optimised neural network kernels.

Those two artefacts became the MLPerf Tiny keyword spotting benchmark. The MLPerf Tiny paper uses a DS-CNN on Speech Commands v2 with a 49 x 10 input, a 52.5 KB TFLite model and a 90% top-one quality target. Its quantized reference reached 91.6% on the full test set.

The runtime layer also changed names. Google rebranded TensorFlow Lite as LiteRT in 2024, and its documentation now calls the embedded runtime LiteRT for Microcontrollers. The code still lives in the tflite-micro repository and still uses the tflite:: C++ namespace. If you have not used that runtime before, start with our TinyML on microcontrollers tutorial covering TFLite Micro, INT8 and CMSIS-NN, which covers the generic deployment path. This post focuses on what is specific to audio: framing, streaming, smoothing and always-on power.

The thesis is simple. Treat wake-word detection as a signal-processing system with a neural classifier inside it, not as a classifier with some audio glue. Most failures in the field come from the glue.

The Keyword Spotting Microcontroller Reference Architecture

A keyword spotting microcontroller pipeline captures 16 kHz audio by DMA, slices it into 30 ms windows every 20 ms, converts each window to 10 MFCC coefficients, keeps the last 49 frames in a ring buffer, and runs an INT8 DS-CNN over that one-second window several times per second. Smoothed class scores then pass a threshold and refractory gate.

Keyword spotting microcontroller reference architecture from microphone through MFCC front end and INT8 DS-CNN to wake event

Figure 1: The end-to-end keyword spotting pipeline on a microcontroller, from microphone DMA to a debounced wake event.

Figure 1 reads left to right. A PDM or I2S microphone feeds a double-buffered DMA ring, so the CPU only wakes when half a buffer is full. Each hop produces one feature frame: Hann window, 512-point FFT, 40-band mel filterbank, logarithm and a discrete cosine transform truncated to 10 coefficients. Frames accumulate in a 49 x 10 int8 ring. The DS-CNN reads that ring, emits 12 class scores, and a smoothing stage converts a jittery score stream into one clean event.

The audio front end is the real model input contract

The neural network never hears audio. It sees a 49 x 10 matrix, and every number in that matrix depends on front-end parameters. The MLPerf Tiny training code defaults to a 30 ms window, a 20 ms stride, 16 kHz audio, one-second clips and 10 MFCC coefficients. The arithmetic is worth doing once. A 30 ms window is 480 samples and a 20 ms hop is 320 samples. The number of frames is 1 + floor((16,000 − 480) / 320) = 1 + 48 = 49. That is where the famous 49 x 10 shape comes from.

Change any of those numbers and the model is invalid, even if the shape still matches. A different mel range, a natural log instead of log10, a different FFT scaling, or a missing pre-emphasis filter will shift the feature distribution. The network then sees inputs it was never trained on, and accuracy degrades quietly rather than failing loudly. This is why the most important artefact in a KWS project is not the model file but the front-end specification shared by the training script and the firmware.

Hello Edge itself shows how easy it is to get this wrong in documentation. Its main text describes 40 MFCC features per frame with a 40 ms frame and 20 ms stride, while its appendix tables for the DS-CNN use 10 MFCC features. Both are valid experiments, but they are not interchangeable. Pick one specification, write it down, and test it.

Google’s own micro_speech example takes a different route. Its current audio preprocessor is itself a small .tflite model built from Signal Library operations: Hann window, FFT, filterbank and per-channel energy normalisation (PCAN) gain control. It uses a 30 ms window, a 20 ms stride and 40 channels, giving a 49 x 40 input to a model of under 20 KB that recognises “yes” and “no”. Running the front end as a graph means the same operators execute in training and on the device, which removes a whole class of mismatch bugs.

Why the DS-CNN wins the budget

A standard convolution with a 3 x 3 kernel over 64 input and 64 output channels costs 3 x 3 x 64 x 64 = 36,864 multiply-accumulates (MACs) per output position. A depthwise separable convolution splits that into a 3 x 3 depthwise filter per channel (576 MACs) plus a 1 x 1 pointwise mix (4,096 MACs). That is 4,672 MACs, about 7.9 times fewer, with similar representational power for this task.

The MLPerf Tiny DS-CNN is small enough to count by hand. The input 49 x 10 x 1 passes through a 10 x 4 convolution with 64 filters and stride 2, giving a 25 x 5 x 64 tensor. That layer costs 25 x 5 x 64 x 40 = 320,000 MACs. Each of the four depthwise separable blocks then costs 72,000 MACs for the depthwise part and 512,000 for the pointwise part. Four blocks add 2,336,000 MACs. Average pooling and a 64-to-12 dense layer are almost free. The total is about 2.66 million MACs per inference, which matches the 2.657 million MACs reported by Silicon Labs’ MLTK profiler for its MLPerf-derived model.

Parameter count is similarly modest. Building the network in Keras gives roughly 22,000 convolution and dense weights, plus batch-normalisation parameters that fold into the convolutions at conversion time. In INT8 that is about 22 KB of weights. The rest of a roughly 50 KB .tflite file is per-channel quantization scales, int32 biases and flatbuffer metadata. Notice that the MLPerf paper describes the model as having “38.6K parameters”; my own Keras build counts fewer. The 38.6 figure matches Hello Edge’s 38.6 KB memory estimate, so treat parameter counts in secondary sources with care.

Streaming is a separate system from classification

Benchmarks score isolated one-second clips. Products never receive isolated clips. The device sees an endless stream, and a keyword can start at any of the 50 hop positions per second. Three design choices follow.

First, compute features incrementally. Each new hop needs exactly one new frame. Recomputing all 49 frames on every inference wastes 98% of the front-end work. Second, decide the inference cadence independently of the hop. Running the DS-CNN every hop (50 times per second) costs 133 million MACs per second. Running it every fifth hop (10 times per second) costs 26.6 million, which is what Hello Edge’s deployment did. Third, add a decision layer that averages scores over time and suppresses repeated triggers. That layer, not the network, sets your false-accept rate.

Walk-through: Building the Pipeline End to End

This section builds each stage in order. The Python code runs on TensorFlow 2.21, the current PyPI release as of November 2026, and was smoke-tested for shapes, parameter counts and INT8 conversion in a clean environment. The C++ code is a sketch against the LiteRT for Microcontrollers API; adapt it to your board support package.

Step 1: Dataset, labels and the split that prevents leakage

The standard 12-class setup uses ten command words, a “silence” class built from background-noise snippets, and an “unknown” class sampled from the other 25 words. The MLPerf Tiny defaults put 10% of training examples in silence and 10% in unknown. Those proportions matter later, because the unknown class is your only explicit training signal for “not the keyword”.

The split is deterministic. The TensorFlow input_data.py helper strips the _nohash_ suffix from each filename, hashes the remaining speaker-and-word stem with SHA-1, and maps it to a percentage. Every clip from one speaker for one word therefore lands in the same partition across dataset versions. The dataset also ships validation_list.txt and testing_list.txt. Use them. A random file-level split lets the same voice appear in train and test and inflates accuracy by an amount you cannot quantify afterwards.

Augmentation follows the reference recipe. Mix background noise from the _background_noise_ folder into about 80% of training clips at a volume of up to 0.1, and shift each clip randomly by up to 100 ms. Both defaults come from the MLPerf Tiny training script. For a real product, add recordings from your own enclosure, microphone and rooms. Speech Commands was recorded mostly on phones and laptops, not on a MEMS microphone behind a plastic grille.

Step 2: MFCC features that the device can reproduce

The feature extractor below uses tf.signal, so the same graph runs in training and can be ported line by line to CMSIS-DSP or ESP-DSP in firmware.

import tensorflow as tf

SR = 16000          # sample rate, Hz
WIN = 480           # 30 ms window
HOP = 320           # 20 ms hop
NFFT = 512          # next power of two above WIN
N_MEL = 40          # mel bands
N_MFCC = 10         # coefficients kept
F_LO, F_HI = 20.0, 4000.0

MEL_W = tf.signal.linear_to_mel_weight_matrix(
    num_mel_bins=N_MEL, num_spectrogram_bins=NFFT // 2 + 1,
    sample_rate=SR, lower_edge_hertz=F_LO, upper_edge_hertz=F_HI)

def mfcc(wav_int16):
    """wav_int16: 1-D tensor of 16,000 int16 samples -> (49, 10) float32."""
    x = tf.cast(wav_int16, tf.float32) / 32768.0
    stft = tf.signal.stft(x, frame_length=WIN, frame_step=HOP,
                          fft_length=NFFT, window_fn=tf.signal.hann_window)
    power = tf.abs(stft) ** 2                       # (49, 257)
    mel = tf.tensordot(power, MEL_W, 1)             # (49, 40)
    log_mel = tf.math.log(mel + 1e-6)
    return tf.signal.mfccs_from_log_mel_spectrograms(log_mel)[..., :N_MFCC]

def make_example(wav_int16, label):
    feats = mfcc(wav_int16)
    return feats[..., tf.newaxis], label            # (49, 10, 1)

A few choices deserve explanation. The 512-point FFT zero-pads the 480-sample window, which gives 257 frequency bins at 31.25 Hz spacing. The upper mel edge of 4 kHz discards high-frequency energy that carries little keyword information and much fan and hiss noise; Silicon Labs’ MLPerf-derived model uses the same 4 kHz limit. The small epsilon inside the logarithm prevents negative infinity on digital silence, which a real MEMS microphone produces after power-up.

On the device, the work per hop is modest. A 512-point real FFT costs on the order of tens of thousands of cycles on a Cortex-M4 with CMSIS-DSP, and a 40-band filterbank with sparse triangular weights costs a few thousand MACs. The DCT to 10 coefficients is a 10 x 40 matrix multiply, so 400 MACs. Even at 50 hops per second, the front end typically costs a few million cycles per second. That estimate is illustrative and depends on the FFT implementation and fixed-point format you choose.

The most common bug is scaling. Training code that normalises audio to the range −1 to 1 and firmware that feeds raw int16 samples produce features offset by a constant in the log domain. After the DCT, that offset lands almost entirely in coefficient zero. If your on-device accuracy collapses while offline accuracy looks fine, dump one second of device features over UART and compare them with the Python output for the same recording. That single test catches most front-end mismatches.

Step 3: Define and train the DS-CNN

The model below mirrors the MLPerf Tiny reference: one standard convolution, four depthwise separable blocks with 64 channels, average pooling, and a dense layer to 12 logits.

from tensorflow.keras import layers as L

def ds_cnn(input_shape=(49, 10, 1), n_classes=12, f=64, batch_size=None):
    inp = L.Input(input_shape, batch_size=batch_size)
    x = L.Conv2D(f, (10, 4), strides=(2, 2), padding="same", use_bias=False)(inp)
    x = L.BatchNormalization()(x)
    x = L.ReLU()(x)
    x = L.Dropout(0.2)(x)
    for _ in range(4):
        x = L.DepthwiseConv2D((3, 3), padding="same", use_bias=False)(x)
        x = L.BatchNormalization()(x)
        x = L.ReLU()(x)
        x = L.Conv2D(f, (1, 1), use_bias=False)(x)
        x = L.BatchNormalization()(x)
        x = L.ReLU()(x)
    x = L.Dropout(0.4)(x)
    x = L.AveragePooling2D(pool_size=(25, 5))(x)   # 25 x 5 -> 1 x 1
    x = L.Reshape((f,))(x)
    out = L.Dense(n_classes)(x)                    # logits
    return tf.keras.Model(inp, out)

model = ds_cnn()
model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-3),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
    metrics=["accuracy"])

# train_ds / val_ds: tf.data pipelines built from make_example() + augmentation
lr_schedule = tf.keras.callbacks.LearningRateScheduler(
    lambda epoch: 1e-3 if epoch < 12 else (5e-4 if epoch < 24 else 1e-4))
model.fit(train_ds, validation_data=val_ds, epochs=36, callbacks=[lr_schedule])
model.save("kws_dscnn.keras")

The 36-epoch budget and batch size of 100 are the MLPerf Tiny defaults; the step learning-rate schedule above is my choice and is not taken from that repository. Silicon Labs’ MLTK variant trains for 80 epochs at batch size 64 with a decaying schedule. Both are reasonable starting points.

Two details protect you during quantization. First, use_bias=False before batch normalisation avoids a redundant bias that the converter would fold anyway. Second, plain ReLU keeps activations non-negative and unbounded on one side only, which INT8 handles well. Avoid exotic activations: each operator you use must exist in the LiteRT for Microcontrollers kernel set and, ideally, in CMSIS-NN or ESP-NN.

I used AveragePooling2D with an explicit 25 x 5 pool rather than GlobalAveragePooling2D. In my test conversion, global pooling lowered to a MEAN operator, and a Flatten layer with an unknown batch size added SHAPE, STRIDED_SLICE and PACK operators. Rebuilding the model with a fixed batch of one for export, as in Step 4, leaves just four operator types: CONV_2D, DEPTHWISE_CONV_2D, AVERAGE_POOL_2D and FULLY_CONNECTED.

Step 4: Full-integer INT8 post-training quantization

Post-training quantization maps float32 weights and activations to 8-bit integers using a scale and zero point per tensor, or per output channel for convolution weights. The converter needs a representative dataset to observe activation ranges.

import numpy as np

def representative_data():
    # 300 validation clips spanning all 12 classes, including noise and unknown
    for feats, _ in val_ds.unbatch().take(300):
        yield [tf.expand_dims(feats, 0)]

# Rebuild with a fixed batch of 1 so the graph has no dynamic-shape ops
export = ds_cnn(batch_size=1)
export.set_weights(model.get_weights())

conv = tf.lite.TFLiteConverter.from_keras_model(export)
conv.optimizations = [tf.lite.Optimize.DEFAULT]
conv.representative_dataset = representative_data
conv.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
conv.inference_input_type = tf.int8
conv.inference_output_type = tf.int8
tflite_bytes = conv.convert()
open("kws_dscnn_int8.tflite", "wb").write(tflite_bytes)

# Read back the input quantization so firmware can quantize features identically
from ai_edge_litert.interpreter import Interpreter
it = Interpreter(model_content=tflite_bytes)
it.allocate_tensors()
scale, zero_point = it.get_input_details()[0]["quantization"]
print("input scale", scale, "zero point", zero_point)

In TensorFlow 2.21, tf.lite.TFLiteConverter still works. The Python interpreter class warns that it is deprecated in favour of the ai_edge_litert package, whose latest PyPI release is 2.3.0 from October 2026. TensorFlow’s August 2025 release notes say tf.lite will be removed from future TensorFlow packages, so pin your TensorFlow version in CI and watch the LiteRT migration guide.

DS-CNN keyword spotting training and INT8 quantization flow for microcontroller deployment

Figure 2: Training and full-integer quantization flow, with the quantization-aware training fallback when the INT8 accuracy drop exceeds budget.

Figure 2 shows the flow as a loop with one exit condition. The INT8 model is evaluated on the held-out test set, and only if the drop stays within your budget does it get exported as a C array. Otherwise you fall back to quantization-aware training (QAT), which simulates INT8 rounding during fine-tuning so the weights adapt to it.

For this architecture you will rarely need QAT. Hello Edge reported that 8-bit weights and activations matched or slightly beat float accuracy without retraining. Silicon Labs’ MLTK documentation lists 94.3% for both the float and quantized versions of its model. MLPerf Tiny’s quantized reference scored 91.6%. These are three different training recipes and feature front ends, so do not compare them with each other; compare your INT8 model only with your own float model.

The representative dataset is where most quantization regressions start. If it contains only clean keyword clips, the converter under-estimates the activation range for loud noise and clips it. Include background noise, unknown words and quiet speech. A few hundred examples are usually enough; MLPerf Tiny uses a fixed list of calibration indices from the validation set.

Finally, record the input scale and zero point. Firmware must quantize each MFCC value as q = round(x / scale) + zero_point, clamped to the range −128 to 127. A mismatch here produces the same quiet accuracy loss as a front-end bug.

Step 5: Deploy with LiteRT for Microcontrollers

Convert the .tflite file to a C array with xxd -i, align it to 16 bytes, and place it in flash. The interpreter then needs three things: an operator resolver listing only the kernels you use, a static tensor arena for activations and scratch buffers, and an input tensor you fill from the feature ring.

#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "tensorflow/lite/schema/schema_generated.h"
#include "kws_model_data.h"   // alignas(16) const unsigned char g_kws_model[]

namespace {
constexpr int kFrames = 49, kCoeffs = 10, kClasses = 12;
constexpr int kInferEveryNHops = 5;              // 10 inferences per second
constexpr int kArenaSize = 32 * 1024;            // trim after measuring
alignas(16) uint8_t tensor_arena[kArenaSize];

int8_t feature_ring[kFrames][kCoeffs];
int ring_head = 0, hop_count = 0;
tflite::MicroInterpreter* interpreter = nullptr;
}

void kws_init() {
  const tflite::Model* model = tflite::GetModel(g_kws_model);
  static tflite::MicroMutableOpResolver<5> resolver;
  resolver.AddConv2D();
  resolver.AddDepthwiseConv2D();
  resolver.AddAveragePool2D();
  resolver.AddReshape();
  resolver.AddFullyConnected();
  static tflite::MicroInterpreter interp(model, resolver, tensor_arena, kArenaSize);
  interpreter = &interp;
  if (interpreter->AllocateTensors() != kTfLiteOk) { /* halt and log */ }
  // log interpreter->arena_used_bytes() once, then shrink kArenaSize
}

// Called from the audio task every 20 ms with one new frame of MFCCs
void kws_on_hop(const float mfcc[kCoeffs], float in_scale, int in_zp) {
  for (int c = 0; c < kCoeffs; ++c) {
    int q = static_cast<int>(lrintf(mfcc[c] / in_scale)) + in_zp;
    feature_ring[ring_head][c] = static_cast<int8_t>(q < -128 ? -128 : (q > 127 ? 127 : q));
  }
  ring_head = (ring_head + 1) % kFrames;
  if (++hop_count % kInferEveryNHops != 0) return;

  TfLiteTensor* in = interpreter->input(0);
  for (int f = 0; f < kFrames; ++f) {            // unroll ring, oldest frame first
    const int src = (ring_head + f) % kFrames;
    memcpy(in->data.int8 + f * kCoeffs, feature_ring[src], kCoeffs);
  }
  if (interpreter->Invoke() != kTfLiteOk) return;

  const TfLiteTensor* out = interpreter->output(0);
  posterior_update(out->data.int8, out->params.scale, out->params.zero_point);
}

The resolver above registers five operators; RESHAPE is a cheap safety margin, since my fixed-batch conversion folded it away. Check your converted model with a viewer such as Netron or the interpreter’s op listing; a missing operator fails at AllocateTensors() time, not at link time. Note also the order of the copy loop. The ring head points at the oldest frame, so the model sees time flowing in the same direction as in training.

Kernel selection is a build-time decision. On Arm Cortex-M, building with OPTIMIZED_KERNEL_DIR=cmsis_nn routes convolution, depthwise and fully connected operators to CMSIS-NN. The CMSIS-NN repository describes three code paths: pure C for cores like Cortex-M0 and M3, SIMD DSP-extension code for cores like Cortex-M4 and M33, and M-profile Vector Extension (Helium) code for Cortex-M55 and M85. It states that its kernels are bit-exact with the TFLM reference kernels, which means your Python INT8 evaluation predicts on-device output exactly.

On Espressif parts, the esp-tflite-micro component pulls in ESP-NN. Its README says assembly-optimised versions are selected automatically on ESP32-S3, ESP32-P4 and ESP32-S31, with generic optimisations on ESP32 and ESP32-C3. Espressif’s own person-detection figure shows invoke time on an ESP32-S3 at 240 MHz dropping from 2,300 ms to 54 ms with ESP-NN. That is a vision model, not KWS, but it shows how much the kernel library matters relative to the clock rate. Our TensorFlow Lite Micro on ESP32 implementation guide covers the ESP-IDF build in detail.

If you are choosing silicon, the instruction set matters as much as the clock. Our comparison of RISC-V and Arm Cortex-M for IoT microcontrollers, including ESP32-C series extensions explains why a 160 MHz core without packed-SIMD instructions can lose to a slower core that has them.

Streaming keyword spotting inference sequence on a microcontroller with feature ring, LiteRT Micro and posterior smoothing

Figure 3: The streaming loop. One feature frame per 20 ms hop, one DS-CNN invocation every Nth hop, and a smoother that owns the final decision.

Figure 3 makes the timing explicit. The DMA interrupt fires every 20 ms, the front end pushes one frame, and every fifth hop the interpreter runs on the full 49-frame window. The smoother then decides, and the application starts a refractory period after any accepted event. Keeping these roles separate makes each one testable on recorded audio.

Step 6: Posterior smoothing, thresholds and false-accept tuning

A single inference is a weak decision. As a keyword slides through the one-second window, its score rises, peaks and falls across several consecutive inferences. Noise and similar-sounding words produce brief spikes. Smoothing exploits that asymmetry.

The TensorFlow streaming-accuracy tool ships a reference recognizer with sensible defaults: average scores over a 500 ms window, accept only above a 0.7 averaged confidence, require a minimum number of results in the window, and suppress a repeat detection for 500 ms. At 10 inferences per second, a 500 ms window averages five score vectors. A one-inference spike of 0.9 surrounded by scores of 0.1 averages to 0.26 and is rejected. A real keyword that scores above 0.8 for four of five inferences passes.

#define N_AVG 5          /* 500 ms at 10 inferences per second */
#define THRESH 0.70f
#define REFRACTORY_INFERENCES 15   /* 1.5 s */

static float hist[N_AVG][12];
static int h = 0, filled = 0, cooldown = 0;

void posterior_update(const int8_t *q, float scale, int zp) {
    float logits[12], m = -1e9f, sum = 0.f;
    for (int i = 0; i < 12; ++i) { logits[i] = (q[i] - zp) * scale; if (logits[i] > m) m = logits[i]; }
    for (int i = 0; i < 12; ++i) { logits[i] = expf(logits[i] - m); sum += logits[i]; }
    for (int i = 0; i < 12; ++i) hist[h][i] = logits[i] / sum;     /* softmax */
    h = (h + 1) % N_AVG; if (filled < N_AVG) filled++;
    if (cooldown > 0) { cooldown--; return; }
    if (filled < N_AVG) return;
    int best = -1; float best_p = 0.f;
    for (int c = 2; c < 12; ++c) {            /* skip silence (0) and unknown (1) */
        float p = 0.f;
        for (int k = 0; k < N_AVG; ++k) p += hist[k][c];
        p /= N_AVG;
        if (p > best_p) { best_p = p; best = c; }
    }
    if (best_p > THRESH) { on_keyword(best, best_p); cooldown = REFRACTORY_INFERENCES; }
}

Two metrics govern tuning. The false-reject rate (FRR) is the share of real keywords the device misses. The false-accept rate (FAR) is usually expressed as false accepts per hour of non-keyword audio. Sweep the threshold on long negative recordings and on positive clips, then plot FRR against false accepts per hour. That curve, often called a detection error trade-off (DET) curve, is the real specification of your product.

The arithmetic shows why per-hour rates matter. At 10 inferences per second, a device makes 36,000 decisions per hour. If raw single-inference false positives occurred at 1 in 10,000 decisions, that would be 3.6 spurious triggers per hour before smoothing. That figure is illustrative, and real errors are correlated, so smoothing helps less than an independence assumption would suggest. Measure on your own negative audio: television, music, conversation and the specific appliance noise of the target room.

Expect to need far more negative audio than positive audio. A benchmark test split contains a few thousand clips, which is under two hours of sound. A credible false-accept estimate at one per several hours needs tens of hours of realistic negatives. The MLPerf Tiny streaming test reflects this direction: its 20-minute synthesized stream contains 50 instances of “marvin” mixed with background noise, and scoring tracks both false negatives and false positives.

Step 7: RAM and flash budget

The table below is an illustrative budget for the MLPerf-style DS-CNN on a Cortex-M4F-class MCU. Measure your own numbers with arena_used_bytes() and the linker map.

Item Size estimate How it was derived
Model weights and metadata in flash about 48 to 54 KB 48.2 KB from my conversion; 52.5 KB in MLPerf Tiny; 53.7 KB in MLTK
Runtime and kernels in flash about 30 to 80 KB (illustrative) Core runtime “fits in 16 KB on an Arm Cortex M3” per Google; kernels add more
Tensor arena in RAM about 16 to 24 KB Two 25 x 5 x 64 int8 activations of 8,000 B each, plus input, output and scratch
Feature ring in RAM 490 B 49 x 10 int8
Audio DMA buffers in RAM about 1.3 KB two halves of 320 int16 samples, plus a 480-sample window
FFT and mel scratch in RAM about 2 to 4 KB (illustrative) 512-point real FFT in float32 or q15
Smoothing history 240 B 5 x 12 float32

The arena estimate deserves a cross-check. Silicon Labs’ MLTK reports 21,424 bytes of runtime memory in its model summary for a similar network, which falls inside the range. The dominant cost is the 25 x 5 x 64 activation tensor that every depthwise separable block reads and writes. The memory planner can reuse buffers across layers, but it cannot hold fewer than an input and an output of that size at once.

The conclusion is practical. The whole wake-word pipeline fits in roughly 30 KB of RAM and 100 to 150 KB of flash. That leaves room on a 256 KB RAM part for a BLE or Thread stack, which is usually the larger consumer.

Step 8: Reading real benchmark numbers

The table collects published figures, each from its primary source. They use different front ends, splits and hardware, so treat them as reference points, not a league table.

Source Model and setup Reported result
Speech Commands paper Baseline conv model, train v2, test v2 88.2% top-one
Hello Edge DS-CNN small, 12 classes 94.4% test, 38.6 KB, 5.4 M ops
Hello Edge DS-CNN large 95.4% test, 497.6 KB, 56.9 M ops
Hello Edge 8-bit DNN on STM32F746G with CMSIS-NN about 12 ms per inference including MFCC
MLPerf Tiny paper DS-CNN reference, INT8 91.6% on full test set, 90% target
Silicon Labs MLTK MLPerf-derived DS-CNN, 50 x 10 microfrontend input 94.3% float and INT8; 36.5 ms on EFR32xG24 at 78 MHz with its accelerator

Methodology caveats. Hello Edge’s 94.4% used Speech Commands v1 with roughly 65,000 clips and an 80:10:10 speaker-disjoint split, not the v2 lists. MLPerf Tiny measures inference only and excludes feature extraction. MLTK’s latency is for a part with a matrix accelerator and a different front end. The MLTK 36.5 ms figure is printed as “36.5m” in its profile and I read it as milliseconds, consistent with its reported 27.4 inferences per second.

The MLPerf Tiny v1.3 round, released on 17 September 2025, contained 70 results across five benchmarks from four submitters: Kai Jiang, Qualcomm, STMicroelectronics and Syntiant. It also introduced the streaming wake-word benchmark based on a one-dimensional DS-CNN. Syntiant reported that its NDP120 achieved the lowest energy per inference and highest throughput on that streaming test. I could not retrieve per-system latency or energy values in text form, so none are quoted here; read them in the MLCommons results sheet before citing any.

Power and Duty Cycling: Designing the Always-On Audio Path

Average power, not peak performance, decides whether a battery product ships. A wake-word device spends almost all of its life hearing nothing interesting. The design goal is to make that state as cheap as possible while keeping detection latency acceptable.

Duty-cycled always-on wake word detection MCU power architecture with VAD gating

Figure 4: A tiered always-on audio path. Cheap detectors gate the MCU, and the MCU gates the host processor or radio.

Figure 4 shows three tiers. The microphone, often in a low-power mode, feeds a hardware voice-activity detector (VAD) or a simple energy threshold. Only when speech is likely does the MCU leave deep sleep and run the front end and DS-CNN. Only when a keyword is accepted does the system wake the expensive component: an application processor, a radio link or a cloud speech recognizer.

The duty-cycle arithmetic

Start with compute. At 10 inferences per second, the DS-CNN needs about 26.6 million MACs per second. Assume, illustratively, that CMSIS-NN on a Cortex-M4 sustains around one MAC per cycle on these layers. The network then needs roughly 27 MHz of a core that may run at 80 to 120 MHz. Add a few MHz for the front end and the core is busy roughly a quarter to a third of the time.

Now the power, again illustrative. Suppose the MCU draws 10 mW when active and 50 µW in a retention sleep state, and the microphone draws 0.5 mW. Running KWS continuously at a 30% duty cycle averages about 0.3 × 10 + 0.7 × 0.05 + 0.5, roughly 3.5 mW. On a 2,000 mAh battery at 3 V, about 6 Wh, that lasts roughly 1,700 hours, or about ten weeks.

Gate the MCU with a VAD that reports speech 10% of the time, and the MCU term falls by a factor of ten. Average power drops towards 0.85 mW plus the VAD’s own draw, and battery life moves into the range of many months. Every number in this paragraph is illustrative, but the structure is real: microphone and VAD power become the floor once the processor is gated.

Our guide to energy harvesting power budgets for battery-free IoT sensors applies the same budgeting method to other sensors.

What the VAD costs you

Gating is not free. A VAD has its own miss rate, and a keyword it misses is a keyword the DS-CNN never sees. It also adds latency: the MCU wakes after speech onset, so the first syllable of the wake word may already be gone. The fix is a pre-roll buffer. The microphone path or a small always-on SRAM keeps the last 200 to 500 ms of audio, and the MCU processes that history when it wakes. Some codec and dedicated-processor designs include such a holding buffer; Syntiant’s NDP120 product brief, for instance, lists an input holding tank of up to ten seconds of audio.

The second cost is firmware complexity. Sleep, wake, clock switching and DMA restart must be glitch-free, or each wake introduces a click that the network has never heard. Scheduling matters here too. An async executor or an RTOS with tickless idle makes the “sleep until DMA half-complete” pattern natural; our comparison of Embassy async Rust, Zephyr and FreeRTOS for firmware scheduling and power walks through the trade-offs.

When to move off the general-purpose core

If your budget is below a milliwatt and the product must always listen, a general-purpose MCU core may not be the right engine. Dedicated neural processors, such as Syntiant’s NDP family, and MCUs with integrated NPUs, such as Cortex-M55 or M85 parts paired with an Arm Ethos-U, run the same DS-CNN at a fraction of the energy per inference. Our Arm Ethos-U85 and Cortex-M85 TinyML NPU architecture analysis explains how those accelerators schedule convolution layers. The software pipeline in this post still applies; only the kernel backend and the power tiers change.

Trade-offs, Gotchas, and What Goes Wrong

Benchmark accuracy is not field accuracy. The 12-class test split rewards distinguishing “up” from “off”. A product needs to reject an unbounded set of non-keywords. A model at 94% on the test split can still produce several false accepts per hour on television audio. Build your own negative corpus early.

The unknown class is a leaky bucket. Training “unknown” with 25 other short words teaches the model about those words, not about music, laughter or a dishwasher. Add diverse non-speech and long-form speech to unknown, and weight it in the loss if false accepts dominate.

Front-end drift is silent. A firmware update that changes the FFT library, the window table precision or the log approximation can cost several accuracy points without any error. Put a golden-feature test in CI: run a fixed WAV file through the firmware front end on a host build and compare against Python within a tolerance.

Microphone and enclosure dominate. MEMS microphones differ in sensitivity, low-frequency roll-off and noise floor. An enclosure adds resonances. Collect at least some training and calibration audio through the final mechanical design, and recheck the quantization representative set after any hardware change.

Speaker and accent bias. Speech Commands was crowd-sourced and is not balanced across accents, ages or speaking styles. Evaluate FRR per demographic group you care about, not only in aggregate.

Over-tight arenas fail late. Sizing the tensor arena to the exact measured value leaves no slack for a kernel library update that needs a larger scratch buffer. Keep a margin of 10 to 20% and assert on AllocateTensors() failure at boot.

INT8 is not the end of compression. CMSIS-NN supports int4 weights with int8 activations for several operators, and some NPUs prefer lower precision. For a 22 KB model the flash saving is small, so the accuracy risk is rarely worth it. Our analysis of INT4, INT8 and FP8 on edge NPUs covers when it does pay off.

Licensing and privacy still apply. Speech Commands is CC BY 4.0, which requires attribution. Any audio you collect from users is personal data in most jurisdictions, even if it never leaves the device during normal operation.

Practical Recommendations

Start from the reference and change one thing at a time. The MLPerf Tiny DS-CNN with a 30 ms window, 20 ms hop and 10 MFCCs is a well-understood baseline that fits on almost any 32-bit MCU with DSP instructions. Get it working end to end, with streaming and smoothing, before touching the architecture.

Spend data effort on negatives and on your hardware. Recording two hours of the target keyword through the real enclosure, plus tens of hours of realistic background audio, will move false accepts more than any architecture change. Treat the decision layer as a product parameter: product managers should choose the operating point on the DET curve, not the ML engineer alone.

Keep the toolchain pinned. LiteRT naming and packaging are still moving: the converter still lives in TensorFlow 2.21, the Python interpreter is moving to ai_edge_litert, and tflite-micro publishes rolling development builds rather than numbered releases. Lock versions and keep the converted .tflite file in version control with its front-end specification.

A short checklist before you call the pipeline done:

  • Front-end parameters documented and shared by training code and firmware.
  • Golden-feature test comparing device and Python features on the same WAV.
  • Speaker-disjoint split using the official validation and testing lists.
  • Representative dataset that includes noise, unknown words and quiet speech.
  • INT8 accuracy within your budget of the float model on the test set.
  • Arena size measured with arena_used_bytes() plus a 10 to 20% margin.
  • DET curve measured on at least tens of hours of realistic negative audio.
  • Duty-cycle and average-power measurement on the real board with a power analyser.

Frequently Asked Questions

What is keyword spotting on a microcontroller?

Keyword spotting on a microcontroller is the detection of a small set of spoken words, such as a wake word, directly on a low-power MCU without sending audio to the cloud. The device converts audio into compact features like MFCCs, runs a small neural network such as a DS-CNN, and smooths the scores over time to decide whether a keyword occurred. A typical INT8 model needs roughly 20 KB of RAM for activations and about 50 KB of flash for weights, so it fits comfortably on many Cortex-M4 and ESP32-class parts.

Why use MFCC features instead of raw audio for TinyML keyword spotting?

MFCCs compress one second of 16 kHz audio, which is 16,000 samples, into a 49 x 10 matrix of 490 values. That reduces the network’s input size by over 30 times and encodes the spectral envelope that distinguishes phonemes while discarding pitch and fine detail. Raw-waveform models exist, and MLPerf Tiny even provides time-domain inputs for its open division, but they need more compute in the early layers. On a microcontroller, a cheap fixed front end plus a small network is usually more efficient than learning the front end.

How accurate is a DS-CNN for keyword spotting?

On Speech Commands, the original Hello Edge paper reported 94.4% test accuracy for its small DS-CNN and 95.4% for its large variant across 12 classes. The MLPerf Tiny INT8 reference scored 91.6% against a 90% target, and Silicon Labs reports 94.3% for its derived model. These numbers come from different dataset versions, splits and front ends, so they are not directly comparable. More importantly, clip accuracy says little about false accepts per hour in streaming use, which you must measure on your own audio.

Does INT8 quantization reduce wake word detection accuracy?

For DS-CNN keyword spotting models, full-integer INT8 post-training quantization usually costs little or no accuracy. Hello Edge reported 8-bit results equal to or marginally better than float, and Silicon Labs lists identical 94.3% accuracy for its float and quantized models. Losses appear when the representative dataset is unrepresentative, for example when it contains only clean speech and the converter under-estimates activation ranges for noise. If the drop exceeds your budget, use quantization-aware training before changing the architecture.

Is TensorFlow Lite Micro the same as LiteRT for Microcontrollers?

Yes. Google renamed TensorFlow Lite to LiteRT in 2024, and the documentation now calls the embedded runtime LiteRT for Microcontrollers. The source code still lives in the tflite-micro repository and still uses the tflite:: C++ namespace, so existing firmware keeps compiling. On the Python side, the interpreter is moving to the ai_edge_litert package, and TensorFlow has said the tf.lite module will be removed from future TensorFlow releases. Pin your toolchain versions while that transition completes.

How much power does always-on wake word detection use on an MCU?

It depends mainly on duty cycle and gating. A DS-CNN at 10 inferences per second needs roughly 27 million MACs per second, which keeps a Cortex-M4-class core busy a fraction of each second. Illustratively, that can average a few milliwatts with the core running whenever audio arrives. Adding a hardware voice-activity detector that wakes the MCU only during speech can cut the processor’s share by an order of magnitude, leaving microphone and VAD power as the floor. Measure on your own board, because sleep currents vary widely across parts.

Further Reading

By Riju — about

Comments

No comments yet. Why don’t you start the discussion?

Leave a Reply

Your email address will not be published. Required fields are marked *