Chunked Prefill and Prefill-Decode Interference: How Sarathi-Serve, vLLM and SGLang Schedule LLM Requests (2026)
Chunked Prefill and Prefill-Decode Interference in LLM Serving A single 8,000-token prompt can freeze every other user on a GPU for a third of a second. Nothing is broken: the…