LLM inference - IoT Digital Twin PLM

Native Sparse Attention and DeepSeek Sparse Attention: Long-Context LLMs Without the Quadratic Cost (2026)

By MPRAUTO MPRAUTO October 30, 2026AINo Comments

Sparse Attention LLM Design: Native Sparse Attention and DeepSeek Sparse Attention A 128,000-token prompt looks innocent in an API call. Inside the model it means every new token must be…

Tensor vs Expert vs Data Parallelism for LLM Inference: MoE Serving Design (2026)

By MPRAUTO MPRAUTO October 29, 2026AI2 Comments

Expert Parallelism LLM Inference: Tensor vs Expert vs Data Parallelism for MoE Serving Most teams that move from a dense model to a Mixture-of-Experts (MoE) model keep the parallelism recipe…

Constrained Decoding for LLMs: Grammar-Guided Generation with XGrammar, Outlines and llguidance

By MPRAUTO MPRAUTO October 10, 2026AI1 Comment

Constrained decoding for LLMs explained: token masking, finite-state machines, pushdown grammars, XGrammar, Outlines and llguidance, with overhead.

Speculative Decoding Explained: EAGLE-3 vs Medusa vs Lookahead for LLM Inference Speedup

By MPRAUTO MPRAUTO October 4, 2026AINo Comments

Speculative decoding explained: draft-and-verify math, EAGLE-3 vs Medusa vs lookahead, acceptance rates, vLLM and SGLang support, and when it hurts.

GPU Memory Physics: DRAM vs SRAM, Kernel Fusion and FlashAttention (Series Part 3)

By MPRAUTO MPRAUTO October 2, 2026AI2 Comments

GPU memory physics: DRAM vs SRAM latency and bandwidth, tiling, kernel fusion, and why FlashAttention made LLM inference faster. Series finale.

llm-d on Kubernetes: Prefill/Decode Disaggregation for LLM Inference

By MPRAUTO MPRAUTO September 29, 2026Kubernetes1 Comment

llm-d explained: disaggregated prefill and decode, KV-cache-aware routing and vLLM on Kubernetes, with cost math, an install path and where it beats a plain vLLM deployment.

SGLang 0.5.18 vs 0.5.15: What Changed and How to Upgrade

By MPRAUTO MPRAUTO September 24, 2026TechNo Comments

SGLang 0.5.16-0.5.18 add DSpark speculative decoding, a Rust server and Breakable CUDA Graph. Breaking flags, cache moves and a step-by-step upgrade path.

OpenVINO 2026.4 vs 2025.4: What Changed for Edge LLMs and NPUs

By MPRAUTO MPRAUTO September 23, 2026Tech1 Comment

OpenVINO 2026.4 vs 2025.4: EAGLE-3 and MTP speculative decoding, MoE disk offload, FP8 in NNCF, NPU changes, and the APIs removed in 2026.

What a ChatGPT Query Actually Costs in Energy and Water: Every Number, Traced to Source

By MPRAUTO MPRAUTO September 22, 2026TechNo Comments

The 500ml and 2.9 Wh figures come from real papers that said something narrower. Here is every published measurement of AI energy and water per query, with its scope.

MLPerf Edge Agentic Inference: How TensorRT Edge-LLM Beat llama.cpp 6.4x