TinyML on Microcontrollers: TensorFlow Lite Micro, INT8 Quantization and CMSIS-NN Inference
TinyML explained: running neural networks on microcontrollers with TensorFlow Lite Micro, INT8 quantization, CMSIS-NN kernels, memory budgeting and.
Tencent Hy4 Explained: 770B MoE, Sherry Quantization and Open Weights
Tencent Hy4 preview explained: 770B total and 49B active MoE, Apache 2.0, Sherry quantization to about 214GB, and what it takes to run it locally.
ONNX Runtime 1.30 vs 1.29: What Changed for Edge AI in 2026
ONNX Runtime 1.30 adds Arm NEON/SVE LinearAttention, INT4 paged KV cache, Go bindings and FP16 fallback changes. What changed and how to upgrade safely.
OpenVINO 2026.4 vs 2025.4: What Changed for Edge LLMs and NPUs
OpenVINO 2026.4 vs 2025.4: EAGLE-3 and MTP speculative decoding, MoE disk offload, FP8 in NNCF, NPU changes, and the APIs removed in 2026.
MLPerf Edge Agentic Inference: How TensorRT Edge-LLM Beat llama.cpp 6.4x
MLPerf Inference v6.1 added an Edge Agentic benchmark. NVFP4, FP8 KV cache, 96% cache reuse and tree MTP explain the 6.4x gap over the llama.cpp reference.
Which Small Language Models Actually Run on CPU in 2026: 11 Models Compared
Eleven small language models compared for CPU-only inference in 2026: real Q4 file sizes, official GGUF availability, licences and the memory-bandwidth ceiling.
INT4 vs INT8 vs FP8 on Edge NPUs: The 2026 Quantization Trade-off
INT4, INT8, and FP8 quantization compared on edge NPUs in 2026: accuracy loss, latency, and memory trade-offs for vision and LLM workloads.
Small Language Models on Device: Edge Inference Architecture (2026)
How to run small language models (SLMs) on-device: model sizing, distillation, quantization, NPU acceleration, memory budgets, and when a 1-8B SLM beats a cloud LLM.
AI Inference Cost Optimization: GPU FinOps in 2026
An AI inference cost optimization decision record: continuous batching, KV-cache, quantization, speculative decoding, spot GPUs, and autoscaling the inference path.
Edge AI Inference at Scale: NVIDIA Jetson, Intel, and Arm NPUs (Updated 2026)
Edge AI inference at scale, updated for 2026: NVIDIA Jetson Thor, Hailo and Arm Ethos NPUs, INT4/FP8 quantization, runtimes, and how to pick edge accelerators by TOPS-per-watt.