INT4 vs INT8 vs FP8 on Edge NPUs: The 2026 Quantization Trade-off
INT4, INT8, and FP8 quantization compared on edge NPUs in 2026: accuracy loss, latency, and memory trade-offs for vision and LLM workloads.
Small Language Models on Device: Edge Inference Architecture (2026)
How to run small language models (SLMs) on-device: model sizing, distillation, quantization, NPU acceleration, memory budgets, and when a 1-8B SLM beats a cloud LLM.