SGLang 0.5.18 vs 0.5.15: What Changed and How to Upgrade

Last Updated: September 24, 2026

If you pinned SGLang at 0.5.15 in early summer and now bump the image tag, there is a good chance the server will not even start. Two flags were renamed in v0.5.16 with no alias, so an old launch script dies with unrecognized arguments before a single weight is loaded. That is the easy failure. The hard ones come later: a default radix tree that behaves differently for sliding-window models, a PyTorch jump to 2.13.0, and every compiled-kernel cache moving to a new directory.

An SGLang 0.5.18 upgrade spans three releases in four weeks — v0.5.16, v0.5.17 and v0.5.18 — carrying 1,866 merged pull requests between them. The project has since shipped v0.5.20, which makes waiting even more expensive.

This guide separates the changes that fail loudly from the ones that fail silently, explains the three headline mechanisms in enough depth to reason about them, and gives you a staged upgrade plan with rollback points.

What this covers: a release-by-release change map, a breaking-change and flag-rename table, how DSpark speculative decoding sizes its verify windows, why breakable CUDA graphs matter for prefill, what the Rust server actually moves out of Python, the new SGLANG_CACHE_DIR layout, and a step-by-step rollout checklist.

Context and Background

SGLang is one of the three engines most teams shortlist for self-hosted LLM serving, alongside vLLM and TensorRT-LLM. If you are still choosing between them, our vLLM vs SGLang vs TensorRT-LLM comparison covers the decision. This post assumes the decision is made and you need to keep an SGLang fleet current.

The release cadence is the first thing to internalise. According to the GitHub release pages, the three releases covered here landed two weeks apart. The table also shows v0.5.20, the current latest release, for reference:

Release	Date	Merged PRs	Contributors
v0.5.16	25 July 2026	574	169
v0.5.17	8 August 2026	582	194
v0.5.18	22 August 2026	710	212
v0.5.20	18 September 2026	713	237

The v0.5.18 release notes are long, and the most important operational details are buried under “Breaking” and “Known issues” headings. That is normal for an engine moving this fast, but it means release-note skimming is not a migration strategy.

Three structural facts frame this upgrade window.

The platform floor moved. v0.5.17 still ran PyTorch 2.11.0 on a CUDA 13.0.1 base image. v0.5.18 moved to torch 2.13.0, triton 3.7.1, torchvision 0.28.0 and torchcodec 0.15.0, with ahead-of-time kernels rebuilt. vLLM moved to the same torch 2.13.0 base in its 0.27 release and still pins it in 0.30, as our vLLM 0.30 Model Runner V2 migration guide notes. If you run both engines, you can finally converge their base images.

Kernel backends were pruned. v0.5.16 deleted the QServe W4A8 and FBGEMM FP8 quantization paths, removed CUTLASS FP8 blockwise kernels for SM90 and SM100, and made FlashInfer mandatory for NVFP4 GEMM. v0.5.18 removed the torchao integration entirely. If your checkpoint depends on any of these paths, the upgrade is a re-quantization project, not a version bump.

The serving stack is being split across languages. v0.5.17 introduced an initial Rust frontend that takes over network ingress up to the tokenized handoff to the GPU scheduler. That is the first step in a larger architectural shift, and it changes where you should look when latency regresses.

What Actually Changed Between 0.5.15 and 0.5.18

The short answer: v0.5.16 added DSpark speculative decoding and renamed two flags with no alias; v0.5.17 added the Rust frontend and much faster MoE weight loading; v0.5.18 moved to torch 2.13, consolidated all compiled-kernel caches under SGLANG_CACHE_DIR, and removed torchao. Several silent default changes span all three.

Figure 1: The three-release path from v0.5.15 to v0.5.18, with the headline features and the breaking changes that land at each step, plus where v0.5.20 now sits.

Read the figure left to right as your upgrade path. Each release node branches into what it adds and what it breaks. The breaking branches are front-loaded at v0.5.16 (CLI renames) and v0.5.18 (platform and cache layout). v0.5.17 is the “feature” release: it adds a lot and breaks little. The final node shows that the upstream frontier has already moved to v0.5.20.

The loud failures: flags that no longer parse

The renames in v0.5.16 have no compatibility alias. A launch script containing the old name fails at argument parsing. That is annoying, but it is the best kind of failure: it happens at start-up, in CI, before traffic.

Change	Release	Old	New / action	Failure mode
Waterfill flag renamed	v0.5.16	--enable-deepep-waterfill	--enable-waterfill	unrecognized arguments, no alias (#27350)
Prefill retry flag renamed	v0.5.16	--optimistic-prefill-retries	--optimistic-prefill-attempts	unrecognized arguments, no alias (#30951)
FP4 GEMM backend	v0.5.16	--fp4-gemm-backend cutlass	auto (FlashInfer CuTe DSL on SM100, FlashInfer CUTLASS on SM120)	Removed option (#30448)
QServe W4A8, FBGEMM FP8	v0.5.16	Supported	Removed	Checkpoint cannot load (#31109)
CUTLASS FP8 blockwise	v0.5.16	SM90 / SM100 kernels	Deleted; SM120 moves to JIT	Backend unavailable (#30438)
Spec-decode runner field	v0.5.16	num_tokens_per_bs	num_tokens_per_req	Custom code breaks (#30977)
Kernel namespace	v0.5.16	Old sglang.kernels paths	Relocated	ImportError in forks (#30044, #31582)
Diffusion RL rollout	v0.5.16	JSON, tensor_to_base64	application/msgpack, tensor_to_bytes	Client decode errors (#31565)
torchao	v0.5.18	--torchao-config	Removed	Flag and quant path gone (#34304)
DeepEP install	v0.5.18	Built from source	sgl-deep-ep release wheels	Custom builds diverge
CUDA stack	v0.5.18	torch 2.11.0	torch 2.13.0, triton 3.7.1	Custom extensions must rebuild
Kernel caches	v0.5.18	~/.triton, ~/.cache/flashinfer, others	Under SGLANG_CACHE_DIR	One-time recompile on first launch (#32434)

One rename deserves a second look. Moving from “retries” to “attempts” usually signals a change in what is being counted: an attempt count typically includes the first try, while a retry count does not. Do not copy your old number blindly. Check PR #30951 for the exact semantics and decide whether your value needs to change by one.

The quiet failures: defaults that changed underneath you

Silent default changes do not break start-up. They change behaviour under load, which is why they cause the incidents that make it into post-mortems. Across the three releases, these defaults flipped:

• UnifiedRadixTree is the default for sliding-window attention (SWA), Mamba and DeepSeek Sparse Attention (DSA) models (v0.5.16). Prefix-cache hit rates and memory accounting can shift for hybrid models.
• Chunked input-logprob processing is on by default (v0.5.16). If you request prompt logprobs for evaluation or RL, verify outputs are unchanged.
• FA3 sparse mask kernels are off by default (v0.5.16).
• MoE deferred finalize is on for NVFP4 with the flashinfer_trtllm backend on DeepSeek-V3-family models (v0.5.18). Revert with SGLANG_ENABLE_MOE_DEFERRED_FINALIZE=False.
• Out-of-window SWA slot freeing is on in the unified cache (v0.5.18). Revert with SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=False.
• Remote media downloads are capped at 64 MiB, with redirect validation and an opt-in hostname allowlist (v0.5.18, #34892). Multimodal clients that pass large video URLs will now get errors.
• The engine freezes Python garbage collection after warm-up (v0.5.18, #34999).
• The DeepSeek-V4 fused MHC post-plus-pre path is on by default, including SM12x (v0.5.18).

Each revert switch exists for a reason. Keep a list of them in your runbook so on-call engineers can bisect a regression without reading source code.

The platform re-base

The third category is the least visible and the most expensive. The torch 2.13 move means any custom CUDA extension, custom attention kernel or out-of-tree plugin must be rebuilt against the new ABI. The dependency pins moved in every release: flashinfer 0.6.14 in v0.5.16 (a 0.6.15 bump landed and was reverted), 0.6.15.post1 in v0.5.17, and 0.6.17 in v0.5.18. CuTe DSL went from 4.6.0 to 4.6.2, and sgl-kernel from 0.4.5 to 0.4.6.post1.

The CuTe DSL 4.6.2 bump matters beyond bookkeeping: the release notes say it fixes an FA4 start-up regression on Blackwell. If you tried an intermediate build on B200 or GB300 and gave up on FA4, retest on v0.5.18.

Here is my thesis for this upgrade. SGLang 0.5.15 to 0.5.18 is a platform re-base with features attached, not a feature release with some breaking changes. Plan it as three independent migrations — CLI surface, kernel and quantization backends, compile-cache layout — plus one separate evaluation track for the opt-in features. Teams that treat it as “bump the tag and try DSpark” will debug four things at once.

Deeper Analysis: The Mechanisms Behind the Headlines

The three headline features — DSpark, breakable CUDA graphs and the Rust server — each change a different part of the serving loop. You need to understand the mechanism to predict whether it helps your workload.

DSpark speculative decoding: confidence-sized verify windows

Speculative decoding runs a cheap draft to propose several tokens, then has the large target model verify them in one forward pass. Tokens are accepted up to the first mismatch, plus one “bonus” token the target produces anyway. Our speculative decoding architecture deep dive covers the maths of acceptance rates and why the technique is lossless.

Classic EAGLE-style and multi-token prediction (MTP) setups use a fixed draft length k. Every step, the target verifies k draft tokens regardless of how likely they are to be accepted. When the draft is confident, k is too small and you leave speed on the table. When the draft is guessing, the target spends compute verifying tokens that will be thrown away.

Figure 4: DSpark drafts a block of tokens, cuts the verify window where the draft’s own confidence drops, and has the target verify a ragged window in one pass. Fixed-length drafting always verifies k tokens.

The sequence shows one DSpark step. The draft head produces a block semi-autoregressively, along with a confidence signal per token. The scheduler cuts the verify window where confidence falls, so each request in the batch can verify a different number of tokens. The target then verifies these ragged windows together and returns the accepted prefix plus a bonus token.

That ragged shape is why DSpark needs SGLANG_RAGGED_VERIFY_MODE=compact alongside --speculative-algorithm DSPARK. The verify batch is no longer a rectangle of batch size times k. It is a packed set of variable-length windows, and the kernels must handle that layout. Block size is tunable with --speculative-dspark-block-size.

The project-reported headline number is 383.7 tokens per second at an accept length of about 5, on DeepSeek-V4-Pro with TP8 on B300 at batch size 1 (PRs #30261 and #31434, with an LMSYS blog post dated 6 July). We can derive a rough step time from that. My estimate: 383.7 ÷ 5 ≈ 76.7 verify steps per second, so each draft-plus-verify step takes about 1,000 ÷ 76.7 ≈ 13 ms. That is a derived figure, not a measurement, and it assumes the reported accept length includes the bonus token.

The practical lesson is that DSpark shines at low batch sizes, where decode is memory-bandwidth bound and verifying extra tokens is nearly free. At high batch sizes the GPU becomes compute bound, and every speculative token competes with real requests. Benchmark at your production batch distribution, not at batch 1.

There is also a correctness footnote that should shape your upgrade target. v0.5.18 fixed silent KV-cache corruption on DeepSeek-V4 when draft tokens exceed 4 (#34189). If you run speculative decoding on DeepSeek-V4 and planned to stop at v0.5.16 or v0.5.17, do not. Go to v0.5.18 or later. v0.5.18 also added DSpark logprobs and MegaMoE for DSpark under data-parallel attention.

Breakable CUDA graph: capturing what torch.compile cannot

A CUDA graph records a sequence of GPU kernel launches once and replays it with a single launch. That removes CPU launch overhead, which dominates at small batch sizes. Decode has used CUDA graphs for years because its shapes are small and regular. Prefill is harder: shapes vary with prompt length, and many models contain operations that cannot be captured, such as data-dependent control flow or host synchronisation.

The traditional workaround is piecewise capture through torch.compile (tc_piecewise in SGLang’s terminology). The compiler traces the model, splits the graph at incompatible operations, and captures each piece. It works, but it is slow to build and heavy to maintain.

Figure 3: Three prefill capture strategies. Breakable CUDA graph captures segments directly and runs incompatible operations eagerly between them, with fixed boundary buffers connecting the segments.

The diagram shows the choice SGLang makes at prefill. Full capture puts the whole pass in one graph but works only with FA4 and FlashInfer backends and is still experimental. Piecewise capture relies on torch.compile to split the graph. Breakable CUDA graph (BCG) cuts the graph during capture at incompatible operations, with no compiler involved, runs those operations eagerly, and joins segments through fixed boundary buffers.

The sglang.io breakable CUDA graph deep dive (27 August) gives the numbers. All are project-reported on TP4 with 4× GB300:

Metric	tc_piecewise	BCG	Full capture
Cold-start graph build, GLM-5.2, 42 shapes	183.1 s	35.2 s (5.2×)	n/a
Cold-start graph build, Qwen3-235B-A22B	106.6 s	27.7 s (3.8×)	n/a
Prefill speed-up vs eager, gpt-oss-120b	1.45×	1.70×	1.93×
Prefill speed-up vs eager, GLM-5.2	cannot capture	1.60×	cannot capture
Implementation size	1,771 lines	521 lines	n/a

Two things stand out. First, torch.compile accounted for 78–86% of preparation time in the piecewise path, so BCG’s cold-start advantage comes from removing the compiler. Second, on GLM-5.2 only BCG can capture at all. For architectures with awkward operations, BCG is not faster than the alternatives; it is the only graph option.

Developers mark regions that must run eagerly with the @eager_on_graph decorator. For debugging, --debug-cuda-graph wraps the decode forward in a single eager break, which lets you step through code that would otherwise be hidden inside a replay.

BCG is not new in this window. It first landed on the decode path (#19102), and the blog’s timeline puts the prefill-default PR (#22218) at 24 April. The experimental full prefill graph (#27988) is dated 7 July in that timeline, only days before v0.5.15 shipped, so check your own build rather than assuming it is new to you. What v0.5.16–v0.5.18 add is coverage: breakable prefill CUDA graph under DP attention (v0.5.16, #30898), full MLA support and prefill BCG for Kimi architectures (v0.5.18). If you run MLA models on 0.5.15, you may pick up graph capture where you previously had none.

Graph memory is resident, but BCG works hard to keep it small. Segments of a captured shape share one CUDA graph pool, tensors at eager breaks are held by weak reference, and all capture sizes share one maximum-sized output buffer. The blog reports that 42 captured shapes on the 78-layer GLM-5.2 add 2.4 GB of graph memory. Because of that sharing, do not treat the cost as linear per shape; the average works out to roughly 57 MB, but adding or removing a bucket will not move memory by exactly that amount.

The more important finding is about the capture ceiling. Capturing all the way up to chunked_prefill_size removes the worst-case eager activation peak: from 0.56 GB to 0.001 GB on gpt-oss-120b and from 1.55 GB to 0.35 GB on GLM-5.2. The blog reports that this puts total prefill memory 0.51 GB and 1.10 GB below the no-graph baseline. Stop capture short of the chunk size, and large prefills fall back to eager with the peak intact while the smaller shapes still hold resident memory, so you pay for both.

The Rust server: moving ingress out of Python

v0.5.17 migrated network ingress, up to the tokenized handoff to the GPU scheduler, from Python to multi-threaded Rust (#29799). The Rust side now covers the tokenizer manager, ingress validation and egress, the OpenAI-compatible API server, and prefill-decode (PD) disaggregation support. It ships as prebuilt release artifacts.

Figure 2: Request path with the Rust frontend. Everything up to the tokenized handoff runs in multi-threaded Rust; the GPU scheduler and model runner stay in Python.

Follow a request through the diagram. The client hits the Rust HTTP server, which validates the request, tokenizes it and hands token IDs to the Python scheduler. The scheduler drives the model runner, and results flow back out through the Rust egress path to the client. The legacy Python API server still feeds the same handoff point, and a PD router can sit in front of the Rust server.

Why does this matter? Python’s global interpreter lock makes CPU-heavy frontend work — JSON parsing, chat templating, tokenization of long prompts — a serial bottleneck. Under high request rates with short outputs, the frontend can starve the GPU. Moving that work to multi-threaded Rust removes the lock from the hot path. It does nothing for GPU-bound workloads.

The feature is called “initial” for a reason. v0.5.18 added native multimodal support for Qwen VL in the Rust server (#32365), and v0.5.20 kept adding router policies, health gating on warm-up completion and DP-attention port fixes. Treat v0.5.18’s Rust server as something to canary behind the Python path, not a default. The fact pack I worked from does not name the exact switch that selects it, so check the v0.5.17 notes and current docs for your install method rather than guessing.

There is one more reason to care. A split frontend changes debugging. A request that fails validation now fails in Rust, with Rust error messages, before Python sees it. Update your log parsing and alerting before you enable it.

The unified compiled-kernel cache

v0.5.18 put every compiled-kernel cache under one directory controlled by SGLANG_CACHE_DIR (#32434). The mapping is:

Old location	New location
~/.triton	{SGLANG_CACHE_DIR}/triton
~/.cache/flashinfer	{SGLANG_CACHE_DIR}/.cache/flashinfer
~/.cache/deep_gemm	{SGLANG_CACHE_DIR}/deep_gemm
/tmp/torchinductor_$USER	{SGLANG_CACHE_DIR}/inductor
~/.nv/ComputeCache	{SGLANG_CACHE_DIR}/nv

Setting TRITON_CACHE_DIR, FLASHINFER_WORKSPACE_BASE, SGLANG_DG_CACHE_DIR, TORCHINDUCTOR_CACHE_DIR or CUDA_CACHE_PATH pins that one cache in place. The first launch after the upgrade recompiles once.

Here is the part that trips up container fleets: if you pre-warm or volume-mount caches, your mount point is now wrong. The upstream release notes advise copying or symlinking the old directories into the new layout, and they single out deep_gemm and flashinfer as the expensive ones to rebuild. Follow that advice, but keep expectations realistic for this particular jump. The same release moves triton to 3.7.1, torch to 2.13.0 and flashinfer to 0.6.17, so many entries compiled under the old toolchain will be keyed to old versions and simply recompiled. Copying is cheap insurance; the step that actually matters is pointing SGLANG_CACHE_DIR at a persistent volume and pre-warming it once per image and GPU type.

This is a real win for containers. Before v0.5.18, making compiled kernels survive a pod restart meant mounting five separate paths, one of them under /tmp. Now it is one volume.

The MoE and memory work you get for free

Several v0.5.17 and v0.5.18 changes need no configuration or only a single opt-in, and they change your operating costs. All numbers here are project-reported.

• Faster large-MoE loading: DeepSeek-V4-Pro at TP8 went from about 35 minutes to 6 minutes 20 seconds for a full load (5.6×). GPT-OSS-20B BF16 went from 545 s to 70 s. Opt in with SGLANG_MOE_COPY_WEIGHT_VIEWS_BEFORE_H2D (#32315).
• Overlapped checkpoint staging: with --startup-weight-load-mode overlap, Qwen3-32B on H100 starts in 35.6 s instead of 84.8 s under the plain default (2.38×), and 8.6–11.7% faster than serial loading with prefetch (#32017).
• Weight-cache daemon for faster engine recovery (#27139). The release notes frame restarts as costing 3–6+ minutes, about 6.5 minutes for Qwen3-235B FP8 on four GPUs.
• DWDP for MoE prefill (early development): prefetches peer expert weights over NVLink P2P instead of doing all-to-all expert-parallel dispatch. On 4× B200 running gpt-oss-120b prefill-only, 506K vs 329K tokens per second at saturation (1.54×). Flag --dwdp-size (#29778).
• Session-aware radix cache for agentic and RL workloads: requests carry a session_id, released through /close_session, opt-in with --enable-session-radix-cache (#29173).
• GLM-5.2 DSA cache layer split under prefill context parallelism: per-rank KV memory drops about 74%, from 0.77 to 0.20 GB per rank at 8,192 tokens. It needs --enable-dsa-cache-layer-split with --enable-prefill-cp --cp-strategy interleave.
• ReplaySSM ring spec-verify for gated-delta-net models: speculative scratch memory drops from 11.5 GB to 1.8 GB per GPU on Qwen3.5-35B-A3B. Opt-in --enable-gdn-replayssm-spec.

Loading time is where the business case sits. A worked estimate: if you run 12 DeepSeek-V4-Pro replicas and roll them one at a time, a 35-minute load means a rolling restart takes about 12 × 35 = 420 minutes, or seven hours. At about 6.3 minutes per load it is about 12 × 6.3 ≈ 76 minutes. That estimate ignores graph capture and health-check time, but it shows why faster loading changes how often you can afford to deploy.

For long-context DeepSeek-MLA decode, v0.5.17 also made the decode context parallelism (DCP) communication backend pluggable (--dcp-comm-backend with ag_rs, a2a or fi_a2a) and added --dcp-replicate-q-proj, which projects full-head Q locally and skips a per-layer all-gather. DCP is a decode-side parallelism choice, not disaggregation itself, but it often appears alongside it. Our prefill-decode disaggregation architecture guide explains how the prefill and decode tiers split and where their communication costs arise.

Step-by-Step Upgrade Plan

The plan below takes a v0.5.15 fleet to v0.5.18 in stages. Each stage has an explicit pass condition. Do not combine stages; the point is that when something breaks, you know which change caused it.

Stage 0: inventory

Before touching an image, grep every launch script, Helm chart and systemd unit for the removed and renamed flags. List every quantization format in use. List every environment variable you set, including the five cache-path variables above.

grep -rnE -- '--enable-deepep-waterfill|--optimistic-prefill-retries|--fp4-gemm-backend cutlass|--torchao-config' deploy/ helm/ scripts/
grep -rnE 'TRITON_CACHE_DIR|FLASHINFER_WORKSPACE_BASE|SGLANG_DG_CACHE_DIR|TORCHINDUCTOR_CACHE_DIR|CUDA_CACHE_PATH' deploy/ helm/

Pass condition: you have a written list of hits, and each has an owner and a replacement.

Stage 1: fix the launch command on the old version where possible

Some fixes can be made before the upgrade. Removing --fp4-gemm-backend cutlass in favour of auto is one. Retiring torchao-quantized checkpoints is another. Make those changes on 0.5.15 first so they ship separately from the version bump.

Here is an illustrative before and after for a DeepSeek-style MoE deployment. The model path and numeric values are placeholders; keep your own.

# Before: v0.5.15
python -m sglang.launch_server \
  --model-path "$MODEL" --tp 8 \
  --enable-deepep-waterfill \
  --optimistic-prefill-retries 2 \
  --fp4-gemm-backend cutlass

# After: v0.5.18
export SGLANG_CACHE_DIR=/var/cache/sglang   # persistent volume
python -m sglang.launch_server \
  --model-path "$MODEL" --tp 8 \
  --enable-waterfill \
  --optimistic-prefill-attempts 2 \
  --fp4-gemm-backend auto \
  --startup-weight-load-mode overlap

Remember the retries-versus-attempts caveat: confirm whether 2 should become 3.

Pass condition: CI parses the new command against a v0.5.18 image without unrecognized arguments.

Stage 2: rebuild and pre-warm the image

Build the v0.5.18 image with torch 2.13.0. Rebuild any custom CUDA extension against it. Install DeepEP from the sgl-deep-ep wheels instead of building from source.

Then pre-warm the compile cache on one node of each GPU type:

export SGLANG_CACHE_DIR=/var/cache/sglang
mkdir -p "$SGLANG_CACHE_DIR"
# Start once, wait for readiness, send a short warm-up covering your common shapes, then stop.
python -m sglang.launch_server --model-path "$MODEL" --tp 8 --port 30000 &
until curl -sf localhost:30000/health; do sleep 10; done
# ...send warm-up requests, then:
kill %1
du -sh "$SGLANG_CACHE_DIR"/*

Pass condition: a second start-up with the same volume is noticeably faster than the first, and the cache directory contains triton, inductor and other expected subdirectories.

Stage 3: correctness gate

Run your evaluation set at temperature 0 against v0.5.15 and v0.5.18 and diff the outputs. Expect small drift from kernel changes; investigate anything systematic. Pay special attention to prompt logprobs because chunked input-logprob processing is now on by default.

There is a known issue to plan around. v0.5.16 reported temperature-0 nondeterminism under DP attention with breakable prefill CUDA graph on the DeepSeek-V4-Flash FP4 recipe, and its guarding test was disabled (#31125). If you run that combination, your determinism tests may fail for reasons unrelated to your change. The release notes say that not enabling breakable prefill CUDA graph avoids the path.

Pass condition: evaluation scores within your tolerance, no systematic logprob drift, and a documented explanation for any non-determinism.

Stage 4: performance gate

Replay production traffic, not a synthetic benchmark. Compare time to first token (TTFT), time per output token (TPOT), throughput at your target concurrency, and memory headroom. Watch prefix-cache hit rate on hybrid SWA or Mamba models, because UnifiedRadixTree changed the default there.

Pass condition: no regression beyond budget on TTFT p95, TPOT p95 or throughput, and KV-cache headroom unchanged or better.

Stage 5: canary, then promote

Send a small share of traffic to v0.5.18. Watch error rates, especially 4xx responses from multimodal clients hitting the 64 MiB media cap. Promote in waves.

Only after the fleet is stable on v0.5.18 should you start evaluating DSpark, the Rust server, DWDP or session radix cache. Each is a separate experiment with its own canary.

Where v0.5.19 and v0.5.20 Fit

This post compares v0.5.18 with v0.5.15, but it would be misleading to stop there. The GitHub releases page now lists v0.5.20, released on 18 September 2026 with 713 PRs from 237 contributors. I could not read the v0.5.19 release notes for this post, so I will not characterise them beyond what the v0.5.20 notes say.

The v0.5.20 release notes contain several items that should shape your target version:

• The CUDA 12 lane is retired. v0.5.19 was the last release with -cu12x wheels and images (#38404). If your hosts cannot run CUDA 13, v0.5.19 is your ceiling until drivers move.
• Prefill context parallelism v1 is removed. The strategy-based implementation is the only path, and prefill CP API names were canonicalised (#36228, #36229). If you use the GLM-5.2 DSA cache split with --enable-prefill-cp, recheck those flags.
• Deprecated flags with replacements were cleared (#38375). Any alias you still rely on after v0.5.18 is likely gone.
• Responses API storage is opt-in. /v1/responses no longer keeps results unless the server starts with --enable-response-store; without it, previous_response_id chaining returns 400, and PD deployments cannot enable it (#39122).
• DSpark works under PD with decode context parallelism, verified on 8× B300 up to 256K input (#37709).

My recommendation is to land on v0.5.18 first, following the plan above, and then treat v0.5.20 as a second, smaller hop. The v0.5.18 re-base — torch, caches, backend pruning — is the expensive part. Mixing it with the CUDA 12 retirement and prefill CP rewrite turns one bisectable change into two tangled ones.

Trade-offs, Gotchas, and What Goes Wrong

Intermediate stops are a trap. It is tempting to go 0.5.15 → 0.5.16, pause, then continue. For DeepSeek-V4 with speculative decoding, that parks you on a release with the silent KV-corruption bug fixed in v0.5.18. v0.5.16 also flip-flopped flashinfer (a 0.6.15 bump landed and was reverted). Unless you need a specific intermediate for hardware reasons, go straight to v0.5.18.

DSpark does not help every workload. Its gains come from spare compute during memory-bound decode. At high concurrency, verifying extra tokens steals compute from real requests. The ragged verify mode also adds another kernel path to debug. Start with low-concurrency, latency-sensitive endpoints.

BCG memory depends on the capture ceiling. Graph memory is resident, 2.4 GB for 42 shapes on GLM-5.2 per the SGLang blog. Capture all the way to chunked_prefill_size and the removed activation peak more than pays for it; the blog reports net savings. Cap the capture below the chunk size and you keep the eager peak and pay the resident cost, which comes out of KV-cache capacity. Check your capture ceiling, then measure KV headroom before and after.

The Rust server is young. It moves CPU work off the GIL, which helps when the frontend is the bottleneck. It also changes error messages, logging and validation behaviour. v0.5.18 reverted v0.5.17’s gRPC parallel request lifecycle tracking (#34160), a sign that this area is still settling.

Vendor numbers are vendor numbers. Every benchmark quoted here comes from SGLang release notes or the SGLang blog, measured on specific hardware such as B300, GB300 and B200. None has been independently reproduced for this post. Treat them as direction and order of magnitude, not a capacity plan.

Reverts signal instability. v0.5.18’s known issues include a Kimi K3 MLA gate-projection fusion that landed and was reverted, and an AMD GLM-5.2 fused shared-expert append that was reverted. If you run those models, read the known-issues list before promoting.

Quantization removal is the hardest blocker. If production depends on QServe W4A8, FBGEMM FP8 or torchao, there is no flag to fix it. You must re-quantize to a supported format and re-run quality evaluation. Budget that time explicitly; it is usually the critical path.

Media caps break clients quietly. The 64 MiB default for remote media is a sensible security fix, but multimodal applications passing large video URLs will see new failures. Tell client teams before the canary, not after.

Practical Recommendations

Treat the SGLang 0.5.18 upgrade as a platform migration with a fixed order: CLI surface first, then backends and quantization, then the compile cache and image, then correctness, then performance. Only after the fleet is stable should you turn on new features, one experiment at a time.

If you run DeepSeek-V4 with speculative decoding, go directly to v0.5.18 or later. If you run hybrid SWA or Mamba models, budget extra time to validate prefix-cache behaviour under UnifiedRadixTree. If you run multimodal endpoints, check media sizes against the new cap. If your fleet cannot move to CUDA 13, note that v0.5.19 is the last CUDA 12 release.

For a broader view of how SGLang’s KV-cache and radix-tree design compare with other engines, see our KV-cache optimization guide for LLM inference.

Upgrade checklist:

• [ ] Grep deploy configs for --enable-deepep-waterfill, --optimistic-prefill-retries, --fp4-gemm-backend cutlass and --torchao-config.
• [ ] Confirm retries-to-attempts semantics in #30951 before setting the new value.
• [ ] Inventory quantization formats; re-quantize anything on QServe, FBGEMM FP8 or torchao.
• [ ] Rebuild custom CUDA extensions against torch 2.13.0 and triton 3.7.1.
• [ ] Set SGLANG_CACHE_DIR to a persistent volume; copy or symlink old deep_gemm and flashinfer caches as upstream advises, but expect a recompile.
• [ ] Pre-warm the cache once per image and GPU type.
• [ ] Diff temperature-0 outputs and prompt logprobs against v0.5.15.
• [ ] Replay production traffic; compare TTFT p95, TPOT p95, throughput and KV headroom.
• [ ] Document the revert env vars for deferred finalize and SWA slot freeing.
• [ ] Warn multimodal clients about the 64 MiB media cap.
• [ ] Canary, promote in waves, then evaluate DSpark and the Rust server separately.
• [ ] Plan the v0.5.20 hop as a separate change.

Frequently Asked Questions

What is the latest SGLang version in September 2026?

As of 24 September 2026, the GitHub releases page marks v0.5.20 as the latest release, published on 18 September with 713 merged PRs from 237 contributors. v0.5.18, released on 22 August, is the version this guide targets because it carries the torch 2.13 and compile-cache changes that make the jump from 0.5.15 expensive. The v0.5.20 notes say v0.5.19 was the last release with CUDA 12 wheels. Land on v0.5.18 first, then plan v0.5.20 as a smaller follow-up hop.

Why does my SGLang launch command fail with “unrecognized arguments” after upgrading?

v0.5.16 renamed two flags without a compatibility alias. --enable-deepep-waterfill became --enable-waterfill, and --optimistic-prefill-retries became --optimistic-prefill-attempts. Any script using the old names fails at argument parsing, before model loading. Also check for --fp4-gemm-backend cutlass, removed in v0.5.16, and --torchao-config, removed in v0.5.18. Replace them and re-run the parser in CI. Because “attempts” may count the first try, confirm whether your numeric value needs adjusting.

What is DSpark speculative decoding in SGLang?

DSpark is a confidence-driven speculative decoding algorithm added in v0.5.16. Its draft proposes tokens semi-autoregressively in blocks, then sizes each request’s verify window from the draft’s own confidence instead of using a fixed draft length. The target model verifies these ragged windows in one pass. Enable it with --speculative-algorithm DSPARK plus SGLANG_RAGGED_VERIFY_MODE=compact, and tune --speculative-dspark-block-size. The project reports 383.7 tokens per second on DeepSeek-V4-Pro at batch size 1 on B300.

What is a breakable CUDA graph and why does it matter?

A breakable CUDA graph captures GPU kernels into graph segments and cuts the graph at operations that cannot be captured, running those eagerly in between. Unlike torch.compile piecewise capture, it needs no compiler. SGLang reports cold-start prefill graph builds about 3.8–5.2× faster and prefill 1.70× faster than eager on gpt-oss-120b. For some models, such as GLM-5.2, it is the only way to capture prefill graphs at all. Captured graphs hold resident GPU memory, but if you capture up to the chunked-prefill size, the SGLang blog reports total prefill memory ends up below the no-graph baseline.

Do I need to migrate my Triton and FlashInfer caches to SGLANG_CACHE_DIR?

Only if you pre-warm or volume-mount them. v0.5.18 moves every compiled-kernel cache under SGLANG_CACHE_DIR, and the release notes advise copying or symlinking the old directories in that case, calling out deep_gemm and flashinfer as the expensive ones. Because the same release moves to torch 2.13.0, triton 3.7.1 and flashinfer 0.6.17, expect much of the old cache to be recompiled anyway. The essential step is pointing SGLANG_CACHE_DIR at a persistent volume. If you need one cache to stay elsewhere, setting its native variable, such as TRITON_CACHE_DIR, pins it in place. Pre-warm once per image and GPU type.

Should I enable the SGLang Rust server in production?

Not as part of the version upgrade itself. The Rust frontend arrived in v0.5.17 as an initial implementation that moves ingress, validation, tokenization and the OpenAI-compatible API into multi-threaded Rust. It helps most when Python frontend work bottlenecks the GPU, such as high request rates with short outputs. It also changes error messages and logging. Stabilise on v0.5.18 with the Python path first, then canary the Rust server as a separate experiment.

Further Reading

• vLLM 0.28 to 0.30 migration: Model Runner V2 default and breaking changes — the sibling guide for vLLM fleets on the same torch 2.13 base.
• vLLM vs SGLang vs TensorRT-LLM in 2026 — how the three serving engines compare on architecture and workload fit.
• Speculative decoding architecture for LLM inference — acceptance rates, draft models and when speculation pays off.
• Prefill-decode disaggregation for LLM serving — the architecture behind SGLang’s PD and context-parallel options.
• SGLang v0.5.18 release notes on GitHub — the primary source for breaking changes and known issues.
• Breakable CUDA Graph in SGLang — the SGLang team’s deep dive on BCG design and benchmarks.

By Riju — about