SGLang 0.5.18 vs 0.5.15: What Changed and How to Upgrade
SGLang 0.5.16-0.5.18 add DSpark speculative decoding, a Rust server and Breakable CUDA Graph. Breaking flags, cache moves and a step-by-step upgrade path.
OpenVINO 2026.4 vs 2025.4: What Changed for Edge LLMs and NPUs
OpenVINO 2026.4 vs 2025.4: EAGLE-3 and MTP speculative decoding, MoE disk offload, FP8 in NNCF, NPU changes, and the APIs removed in 2026.
What a ChatGPT Query Actually Costs in Energy and Water: Every Number, Traced to Source
The 500ml and 2.9 Wh figures come from real papers that said something narrower. Here is every published measurement of AI energy and water per query, with its scope.
MLPerf Edge Agentic Inference: How TensorRT Edge-LLM Beat llama.cpp 6.4x
MLPerf Inference v6.1 added an Edge Agentic benchmark. NVFP4, FP8 KV cache, 96% cache reuse and tree MTP explain the 6.4x gap over the llama.cpp reference.
Flink 2.2 ML_PREDICT and VECTOR_SEARCH: Running Model Inference Inside a Stream
Flink 2.2 adds VECTOR_SEARCH and Table API ML_PREDICT for in-stream inference. How they work, where they break, and when to keep inference outside.
ExecuTorch 1.5 On-Device LLM Serving (2026): Batched Scheduling, Cancellation and Off-Graph KV Cache
ExecuTorch 1.5 shipped multi-method export, batched request scheduling, bounded cancellation and off-graph KV cache. What it changes for on-device LLM apps.
vLLM vs SGLang vs TensorRT-LLM in 2026: The Serving Engine Pick
vLLM, SGLang, and TensorRT-LLM compared for 2026 production LLM serving: throughput, KV-cache handling, and which to pick by workload.
Diffusion LLMs: How Text Diffusion Models Work (2026)
Diffusion LLMs explained: how text diffusion models replace next-token prediction with iterative denoising and parallel decoding - masked discrete diffusion, remasking, throughput trade-offs vs autoregressive models in 2026.
Constrained Decoding: Architecture for Guaranteed-Valid LLM Output (2026)
How constrained decoding guarantees valid LLM output: grammars, FSAs, token masking, JSON-schema enforcement, and where structured generation breaks in production.
On-Device SLM Inference: A 2026 Edge GPU Benchmark
A 2026 benchmark methodology for small language models on edge GPUs — latency, tokens/sec, memory, and cost for Phi, Gemma, and Qwen on Jetson-class hardware.