vLLM 0.28 to 0.30 Migration: Model Runner V2 Default, Breaking Changes

Last Updated: September 23, 2026

On 9 September 2026, vLLM 0.29.0 changed the component that runs every forward pass on your GPUs, and many operators did not notice. vLLM Model Runner V2 (MRV2) became the default for all models. Model Runner V1 was declared deprecated, with removal targeted for v0.32. Thirteen days later, vLLM 0.30.0 shipped several more changes:

scale-out endpoints moved behind a new --enable-scale-out flag
GPTQ activation ordering was deleted
a batch of environment variables was removed
YaRN handling was re-aligned with Transformers, which quietly shrinks the default context length on some models

If your vLLM fleet is pinned at 0.27, the jump to 0.30 spans three minor releases, about six weeks of upstream work and roughly 1,900 commits. This guide lists what actually changed in the engine loop, which changes fail loudly and which fail silently, and how to roll the upgrade out behind a canary so you are not debugging a context-length regression in production.

What this covers: what Model Runner V2 is and why V1 is going away, a release-by-release change table for 0.28, 0.29 and 0.30, every breaking change with symptom and fix, the silent max_model_len drop and how to detect it, KV-cache tiering changes, Fast Start restarts, and a canary rollout plan with explicit promote and rollback gates.

Context and Background

vLLM now ships a minor release roughly every two weeks. PyPI upload timestamps show:

Version	Uploaded
0.25.0	11 July
0.26.0	25 July
0.27.0	10 August
0.28.0	26 August
0.29.0	9 September
0.30.0	22 September

These releases are large: 0.25 to 0.27 each carried 411 to 561 commits, and 0.28 to 0.30 each carried 584 to 762 commits from 270 to 315 contributors. At that pace, “we’ll upgrade next quarter” means skipping six releases. It also means walking through several deprecate-then-remove cycles in one go, because vLLM typically removes an item one minor release after deprecating it.

This is not a comparison post. If you are still choosing an engine, our vLLM vs SGLang vs TensorRT-LLM comparison covers that decision. This post assumes you already run vLLM and need to keep running it. The people it is written for maintain OpenAI-compatible endpoints on H100, H200, B200 or MI300-class hardware, pin a version in a container image, and carry some local configuration: env vars, a quantised checkpoint, maybe a custom logits processor or a speculative decoding setup.

Three structural facts frame this upgrade window.

The model runner is being replaced. The model runner is the worker-side component that turns a scheduler decision into GPU tensors, runs the forward pass, samples tokens and hands results back. Model Runner V2 was introduced as an opt-in rewrite in the spring. The vLLM MRV2 announcement described it as a “ground-up re-implementation” with no user-facing API changes. It became the default for pooling models first (#48290) and then for everything in 0.29 (#53183).

The platform floor moved in 0.27. That release upgraded PyTorch to 2.13.0 with torchvision 0.28.0 and Triton 3.7.1 (#48155). The 0.30 requirements/cuda.txt still pins torch==2.13.0. If your fleet is on 0.26 or older, the torch jump is a bigger risk than anything in this post.

The default wheel targets CUDA 13.0. The PyPI wheel and the default vllm/vllm-openai:v0.30.0 image are built for CUDA 13.0, and -cu129 Docker tags exist for hosts that have not moved their drivers. ROCm wheels come from a separate index (wheels.vllm.ai/rocm/0.30.0/rocm723).

The Upgrade at a Glance: 0.28, 0.29 and 0.30

Direct answer: upgrading vLLM from 0.28 to 0.30 means three things:

accepting Model Runner V2 as the default engine runner
fixing about a dozen removed flags, env vars and kernels
re-checking three behavioural defaults: batch token budget, FlashInfer all-reduce and YaRN-derived context length

Most breaks fail at startup. The YaRN change can fail silently at request time.

The table below groups every operator-relevant change by release. Model additions and kernel-level performance work are left out unless they change a default.

Area	0.28.0 (26 Aug)	0.29.0 (9 Sep)	0.30.0 (22 Sep)
Model runner	MRV2 gains E/P/D disaggregation (#38390), weight offloading (#51413), `thinking_token_budget` (#46727)	MRV2 default for all models (#53183); MRV1 deprecated, removal targeted v0.32	MRV2 gains DBO (opt-in), MTP and EAGLE3/DFlash/DSpark under pipeline parallelism, faster graph capture (#54646)
KV cache	Tiered offloading: disk tier (#49644), out-of-tree tier managers via `module_path` (#51007), tiering metrics (#48798)	CUDA graph memory profiling for KV auto-sizing (#53306); deterministic `NONE_HASH` (#51875)	HiSparse host-resident KV tier for sparse-MLA decode (#53781); KVCR secondary-tier adapter (#53624)
Scheduler and admission	`max_num_batched_tokens` default raised on large GPUs (#51726)	`--max-num-queued-reqs` / `--max-num-queued-tokens` (#49445); `--prefix-cache-retention-interval` (#52216)	`vllm:request_num_preemptions` histogram (#49984)
API surface	`reasoning_content` output removal flagged as breaking for clients (#50624)	`--per-request-spec-decode-metrics` (#48915); `api_server` module entrypoint deprecated (#52131)	`/v1/responses/render` (#50195); `reasoning_tokens` in usage (#54982); scale-out endpoints need `--enable-scale-out`
Removals	bitsandbytes moved out of tree (#43529); `calculate_kv_scales` (#49389); `override_attention_dtype` (#48684)	Ten architectures (#53608); PyAV video decoder (#54231); two env vars	GPTQ `g_idx` (#54809); 0.29-deprecated env vars and aliases (#55353); `VLLM_ENABLE_SCALE_OUT_ENDPOINTS`
Silent changes	Prefix caching on by default for Mamba (#50991)	FlashInfer all-reduce on for TP (#52998)	YaRN aliases no longer rescale `max_model_len` (#56446)

Figure 1: The upgrade decision path from any pre-0.30 vLLM version to a pinned 0.30.0 canary.

Figure 1 orders the checks so that the most expensive surprises come first. If you are older than 0.27, the torch 2.13 and CUDA 13 rebuild comes before anything else, because it touches every custom kernel and extension in your image. After that, each branch is a yes-or-no question about your own deployment:

Do you serve bitsandbytes checkpoints?
Do your GPTQ checkpoints use activation ordering?
Does anything call the scale-out endpoints?
Do you serve a model whose config uses a vendor YaRN alias?

Every “yes” has a specific fix. Every path ends in the same place: pin the exact release and canary it.

What vLLM Model Runner V2 Actually Changes in the Engine Loop

Direct answer: Model Runner V2 is a rewrite of the vLLM worker component that prepares inputs, runs the model and samples tokens. It keeps a stable per-request state table on the GPU and builds per-step inputs with Triton kernels instead of CPU tensor operations. It is designed so that async scheduling and speculative decoding run with zero CPU-GPU synchronisation.

The problem with V1: persistent state doubled as model input

vLLM V1 introduced persistent batching. Consecutive decode steps usually contain almost the same requests, so it is cheaper to update cached tensors incrementally than to rebuild them each step. The trouble was that V1 used that persistent state directly as the model and sampler input. Request order in the batch was coupled to the block-table layout. Adding or removing a request forced tensor-wide reordering and bookkeeping, plus a backup structure (CachedRequestState) to survive the shuffles.

Over the V1 runner’s lifetime, features were added one at a time: async scheduling, speculative decoding, structured outputs, multimodal encoders. According to the MRV2 announcement, the single V1 runner file grew past 6,700 lines. Async scheduling had been retrofitted onto it, so combining async with speculative decoding required CPU-GPU synchronisation points. Those points exist precisely because the CPU needed to know how many draft tokens were accepted before it could prepare the next step.

What V2 does instead

MRV2 separates state from inputs. Each live request gets a stable row in a fixed-size state table for its whole lifetime. Each step, a gather operation builds the correctly ordered per-step tensors from that table: input_ids, positions, query_start_loc, seq_lens and the block table. These gathers run as Triton kernels on the GPU. Three consequences matter for operators.

Host overhead shrinks. Python-side tensor manipulation per step falls sharply. This matters most when the model is small relative to the GPU, because host time is then a large share of each step.
Speculative decoding and async scheduling stop fighting. GPU-side preparation kernels can consume rejection-sampling results directly. The CPU no longer has to wait for “how many tokens were accepted” before launching step N+1. Outputs are copied back on a separate CUDA stream. If you run EAGLE or MTP drafts, this is the headline change; see our speculative decoding architecture guide for why that sync point was so costly.
Sampling is rebuilt. A Triton-native sampler uses Gumbel-max sampling without materialising a softmax, computes logprobs only for top-k candidates, and chunks prompt logprobs more finely. 0.29 added batch-sharded sampling (#50465), which cuts per-step logits memory by a factor of the tensor-parallel degree. In the 0.30 source it is opt-in (enable_batch_sharded_sampling defaults to off). On large-vocabulary models at high TP, that is real memory returned to the KV cache.

Model-specific logic moves behind a ModelState interface: multimodal embeddings, extra inputs, attention metadata and graph capture. The common runner path stays small.

The only published MRV2-versus-V1 numbers are the project’s own stress tests from the announcement, chosen to expose host overhead. One ran Qwen3-0.6B on a single GB200; another measured TPOT with MTP on GLM-4.7-FP8 on 4×GB200. Neither tells you what your fleet will see, so treat any end-to-end gain as something your own canary has to measure.

Why V1 is being removed rather than kept

The 0.29 release notes say it plainly. MRV1 is deprecated, removal is targeted for v0.32, and the project will not accept further MRV1-specific improvements or optimisations. The reasoning is maintenance cost. Two runners mean two implementations of every new feature, and new work is increasingly going only to V2.

In the 0.30 source, the V1 runner already refuses several V2-only features:

prefill context parallelism
DSpark drafting
adaptive draft verification
DFlash2 drafts
batch-sharded sampling
diffusion models

That asymmetry is the real deprecation signal. Staying on V1 is not neutral. It locks you out of most of the performance work landing in 0.28 through 0.30.

When vLLM still falls back to V1

MRV2 is the default, not a guarantee. Figure 2 shows the selection logic as implemented in vllm/config/vllm.py at v0.30.0.

Figure 2: How vLLM 0.30 decides between Model Runner V2 and the deprecated V1.

The order matters.

HiSparse forces V2 and errors if you try to disable it. Watermarking also forces V2, overriding an explicit opt-out.
The VLLM_USE_V2_MODEL_RUNNER environment variable wins next:
– 0 forces V1.
– 1 forces V2 and validates the configuration. If you have configured an unsupported feature, startup fails with “Model Runner V2 does not yet support: …” instead of quietly downgrading.
With the variable unset, vLLM applies two automatic fallbacks:
– On ROCm, three architectures default to V1: DeepseekV32ForCausalLM, DeepseekV4ForCausalLM and GlmMoeDsaForCausalLM.
– On any platform, V1 is used if an unsupported feature is configured.

In 0.30, that unsupported-feature list contains:

sequence parallelism with TP greater than 1
elastic expert parallelism
custom logits processors, including any package that registers a vllm.logits_processors entry point
mamba_cache_mode='all'
stock torch.compile mode
pipeline parallelism under the external_launcher backend
the n-gram, draft-model, suffix, Medusa, MLP-speculator and custom-class speculative methods
EAGLE parallel drafting
the absence of Triton

There is one subtle entry. Dual-batch overlap (DBO) gained an MRV2 implementation in 0.30, but the code only uses it when VLLM_USE_V2_MODEL_RUNNER is set explicitly. With the variable unset, DBO still falls back to V1. The logits-processor check also counts plugins that are merely installed in the environment. A stray package in your image can silently put you on the deprecated runner.

To find out which runner you are on, read the startup log. When the automatic fallback triggers, vLLM logs a warning of the form Model Runner V2 does not yet support <features>; using the V1 model runner instead. On ROCm the message is Defaulting to V1 model runner on ROCm for model architectures: …. Make these strings a deploy-time alert. The upstream tracking issue for remaining gaps is #47172, “Model Runner V2 Remaining TODOs”. Check it before assuming any specific gap is closed.

Breaking Changes, One by One: Symptom, Cause, Fix

Direct answer: the 0.28 to 0.30 window removes the in-tree bitsandbytes integration, GPTQ activation ordering, a handful of environment variables and aliases, ten model architectures and the PyAV video decoder. It also moves the scale-out endpoints behind --enable-scale-out. Almost all of these fail at startup or on first request, so a pre-production smoke test catches them.

Old	New	Release	Action
bitsandbytes built in	Out-of-tree plugin	0.28 (#43529)	Install the plugin in your image, or move to AWQ/FP8
`calculate_kv_scales`	Removed	0.28 (#49389)	Use calibrated KV scales in the checkpoint
`override_attention_dtype`	Removed	0.28 (#48684)	Delete from config
`reasoning_content` in output	Removal documented as breaking	0.28 (#50624)	Update clients that parse it
`kv_offload_tiering_block_*` metrics	`kv_offload_tiering_chunk_*`	0.28 (#52812)	Update dashboards and alerts
`python -m vllm.entrypoints.openai.api_server`	`vllm serve`	0.29 deprecated (#52131)	Change container entrypoint
PyAV video decoder	OpenCV or Torchcodec	0.29 (#54231)	Change video backend
`VLLM_TEST_FORCE_FP8_MARLIN`	`--linear-backend` / `--moe-backend`	0.29 (#52182)	Move to CLI flags
`VLLM_ROCM_USE_AITER_FP4_ASM_GEMM`	Removed	0.29 (#53141)	Delete
Ten architectures (Arctic, MPT, GritLM and others)	Removed	0.29 (#53608)	Stay on 0.28 for these or migrate models
`VLLM_ENABLE_SCALE_OUT_ENDPOINTS`	`--enable-scale-out`	0.30 (#54579, #55176)	Add the flag
GPTQ `g_idx` activation ordering	Ignored, kernels removed	0.30 (#54809)	Requantize without act-order
`VLLM_PREFIX_CACHE_RETENTION_INTERVAL`	`--prefix-cache-retention-interval`	0.30 (#55353)	Move to CLI or config
`VLLM_MM_HASHER_ALGORITHM`	Config field	0.30 (#55353)	Move to config
`use_fp4_indexer_cache`	`indexer_kv_dtype`	0.30 (#55353)	Rename
ROCm `CUDA_VISIBLE_DEVICES` fallback	`HIP_VISIBLE_DEVICES`	0.30 (#55353)	Set the HIP variable
`python -m vllm.entrypoints.grpc_server`	`vllm serve --grpc`	0.30 deprecated (#56746)	Change entrypoint
`mamba_cache_mode='all'`	Deprecated, forces MRV1	0.30 (#55041)	Move to `align` or the default `none`

bitsandbytes moved out of tree

Symptom: a bitsandbytes-quantised checkpoint that loaded on 0.27 fails to load on 0.28+ with a quantisation-method error.

Cause: 0.28 moved bitsandbytes support to an out-of-tree plugin (#43529). The code still exists, but it is no longer part of the core wheel.

Fix: add the plugin package to your image and confirm that it registers at import time. Also ask whether you still want bitsandbytes at all. For serving (as opposed to QLoRA fine-tuning), AWQ, FP8 and NVFP4 paths get far more kernel attention upstream, and 0.30 made FlashInfer CuTeDSL NVFP4 W4A16 the default over Marlin on SM100/103.

GPTQ activation ordering is gone

Symptom: after upgrading to 0.30, a GPTQ model loads without error but produces degraded or incoherent output.

Cause: 0.30 removed GPTQ group and dynamic activation ordering (#54809). The g_idx tensor is now ignored, and the Marlin, GPTQ, CPU and RDNA3 kernels that honoured it are gone.

This is the most dangerous item in the list because the load succeeds. Activation ordering (“act-order” or desc_act=True in most GPTQ tooling) permutes input channels so that the most salient ones are quantised first. g_idx records which quantisation group each input channel belongs to. If you ignore it, a permuted checkpoint’s weights are paired with the wrong scales.

Fix: before upgrading, inventory every GPTQ checkpoint and inspect quantize_config.json or config.json for desc_act: true. Requantize those models without activation ordering, or move them to a different quantisation method. Then run an output-quality eval, not just a load test.

Scale-out endpoints now need `--enable-scale-out`

Symptom: after upgrading to 0.30, a disaggregated or router-fronted deployment starts returning 404 on /render, /derender or /inference/v1/generate.

Cause: these endpoints are no longer registered on a plain vllm serve (#54579, #55176). VLLM_ENABLE_SCALE_OUT_ENDPOINTS was removed, so the old environment variable does nothing. The server logs Scale-out endpoints are disabled. Set --enable-scale-out to enable them. Two modes still register them automatically: vllm launch render and vllm serve --tokens-only.

Fix: add the flag to the serve command. Shrinking the default API surface is a sensible security change. Endpoints that accept pre-tokenised input or return raw render output should not be exposed on a public-facing server by accident.

# vLLM 0.30: scale-out endpoints are opt-in
vllm serve meta-llama/Llama-3.3-70B-Instruct \
  --tensor-parallel-size 4 \
  --enable-scale-out

If you run prefill-decode disaggregation with a router that calls these endpoints between tiers, this is the change most likely to take you down. The upgrade itself is clean, but the router’s first call after rollout fails.

Environment variables and aliases removed in 0.30

0.29 deprecated a set of env vars and aliases in favour of config fields, and 0.30 removed them (#55353). Removed env vars generally fail open: vLLM ignores the variable instead of erroring. As a result, your tuned setting silently reverts to the default.

VLLM_PREFIX_CACHE_RETENTION_INTERVAL becomes --prefix-cache-retention-interval.
VLLM_MM_HASHER_ALGORITHM becomes the corresponding multimodal config field.
The use_fp4_indexer_cache alias is replaced by indexer_kv_dtype.
On ROCm, CUDA_VISIBLE_DEVICES is no longer honoured as a fallback, so set HIP_VISIBLE_DEVICES. On a shared node, getting this wrong can put two engines on the same GPU.
Plugin and custom-backend authors should note that the seq_lens_cpu and num_computed_tokens_cpu attention metadata properties were also removed.

A simple pre-flight check is to grep your deployment manifests for every removed name and fail the build if any appear.

# Pre-flight: fail CI if removed vLLM env vars are still set (illustrative script)
grep -RInE 'VLLM_(PREFIX_CACHE_RETENTION_INTERVAL|MM_HASHER_ALGORITHM|ENABLE_SCALE_OUT_ENDPOINTS|TEST_FORCE_FP8_MARLIN|ROCM_USE_AITER_FP4_ASM_GEMM)' \
  deploy/ helm/ && { echo "Removed vLLM env var found"; exit 1; } || echo "clean"

Entrypoint deprecations

python -m vllm.entrypoints.openai.api_server was deprecated in 0.29 in favour of vllm serve (#52131). python -m vllm.entrypoints.grpc_server was deprecated in 0.30 in favour of vllm serve --grpc (#56746). Both still work, so nothing breaks today. However, vLLM’s pattern is to remove deprecated items one or two releases later. Changing a container ENTRYPOINT is a five-minute job now and an outage later.

Model and media removals

0.29 removed ten deprecated architectures: Arctic, Chameleon, Cheers, Fairseq2Llama, FireRedLID, GritLM, HCXVision, MPT, the RWForCausalLM and StableLMEpochForCausalLM aliases, and PrithviGeoSpatialMAE (#53608). FlexOlmo, Olmo3 and Hunyuan V1/VL moved to the Transformers modeling backend (#53615). They still load, but through a different code path with different performance characteristics.

The PyAV video decoder was removed in 0.29, and 0.30 switched the default audio resampler from PyAV to torchaudio. Multimodal pipelines should expect small numerical differences in preprocessed audio.

Smaller 0.30 breaks worth a line each

Attention backends must now declare decode context parallel (DCP) support. DCP with ROCm standard attention, Triton, FlexAttention or TurboQuant fails at backend selection (#55780).
MoRI-IO connector WRITE mode with hybrid KV cache groups now needs --disable-hybrid-kv-cache-manager (#53721).
sse_keep_alive moved to vllm.entrypoints.serve.utils (#56369). This affects anyone importing it.

The Silent max_model_len Drop in 0.30

Direct answer: vLLM 0.30 aligned its YaRN handling with Transformers. Vendor YaRN aliases no longer multiply max_position_embeddings by the scaling factor a second time, so the derived default max_model_len drops for some models. Release notes give two examples: TeleChat3-36B-Thinking falls from 131,072 to 32,768 tokens, and sarvam-105b falls from 5,242,880 to 131,072 (#56446).

Callout — this one does not fail at startup. If you rely on vLLM’s derived context length (you do not pass --max-model-len), 0.30 starts cleanly with a smaller window. Nothing in the startup path errors. The first sign is usually a rise in HTTP 400 responses for long prompts, or agents and RAG pipelines that suddenly truncate context.

Why it happens

For YaRN, the Transformers convention is that max_position_embeddings in config.json is already the extended length. The scaling factor describes how RoPE frequencies were stretched to reach it; it is not an instruction to multiply again. Some vendors shipped custom rope_type aliases (for example telechat3-yarn). Older vLLM treated those as non-YaRN and multiplied max_position_embeddings by factor, which produced a window larger than the model was trained for.

In 0.30, YaRN variants (yarn, deepseek_yarn) are excluded from rescaling, and the config loader rewrites the telechat3-yarn alias to plain yarn, logging Replacing rope_type 'telechat3-yarn' with 'yarn'. Other models change only through the rescaling rule, with no such log line. The same change makes vLLM ignore attn_factor and extrapolation_factor in favour of mscale and attention_factor.

In one sense the new number is more correct. The old one advertised positions the model never saw in training, which is a known source of quality collapse at long range. Your users still experience the change as a regression.

Two failure shapes

You did not set --max-model-len. The window shrinks silently. Requests that fit before now exceed the limit and are rejected with a 400. Clients that pre-truncate to a stale limit keep working, but clients that trusted the old window break.
You did set --max-model-len to the old value. Startup fails with a ValueError saying the user-specified max_model_len is greater than the derived one. It will only proceed if you set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. That variable exists for exceptional cases, and vLLM’s own error text warns that RoPE positions beyond the derived length can produce NaNs. Do not set it to make the upgrade “go green”.

How to detect it before users do

Record the effective context length per model on the old version, then compare on the canary. The OpenAI-compatible /v1/models endpoint returns a max_model_len field for each served model.

# Capture effective context length on old and new versions, then diff
curl -s http://old-vllm:8000/v1/models | jq -r '.data[] | "\(.id) \(.max_model_len)"' > before.txt
curl -s http://canary-vllm:8000/v1/models | jq -r '.data[] | "\(.id) \(.max_model_len)"' > after.txt
diff before.txt after.txt && echo "no context-length change"

Also scan the startup log for Replacing rope_type lines, and inspect rope_scaling / rope_parameters in each model’s config.json for non-standard rope_type values. Any model that changes needs a product decision. You can accept the smaller window, or you can validate a longer window with a long-context eval before pinning --max-model-len explicitly.

New Defaults That Change Behaviour Without Errors

These do not break anything, but they shift latency, memory and determinism. Your baselines will move, and you should know why.

Batch token budget (0.28, #51726). The release notes describe max_num_batched_tokens rising from 8,192 to 16,384. In the 0.30 source, the default is tiered by GPU memory and usage context:

GPU class	LLM class	OpenAI server
160 GB or more (B200 class)	16,384	16,384
70 GB or more, excluding A100 (H100/H200)	16,384	8,192
Smaller GPUs	8,192	2,048

A larger chunked-prefill budget lets long prompts prefill in fewer steps. The cost is that each mixed step takes longer, which pushes decode inter-token latency up for concurrent requests. If your SLO is on inter-token latency, pin --max-num-batched-tokens explicitly so that a hardware change cannot move it.

CUDA graph capture size on Blackwell (0.28, #49390). The default capture ceiling rose to 1,024 on Blackwell. More batch sizes run as captured graphs, at the cost of longer capture and some graph memory. 0.29 added CUDA graph memory profiling to KV auto-sizing (#53306), so graph memory is now subtracted before KV blocks are allocated. On the same --gpu-memory-utilization, you may see slightly fewer KV blocks than on 0.28. That is expected, and it is more honest accounting.

FlashInfer all-reduce for tensor parallelism (0.29, #52998). This is on by default for TP CUDA groups. Opt out with VLLM_ALLREDUCE_USE_FLASHINFER=0 if you see hangs or regressions on unusual topologies. 0.30 added an opt-in PCIe IPC variant for boxes without NVLink (#53576).

Deterministic prefix-cache NONE_HASH (0.29, #51875). Distributed KV-cache users no longer need to pin PYTHONHASHSEED across nodes for prefix hashes to agree. Remove the workaround only after every node runs 0.29 or later. In a mixed-version fleet, the hash roots differ and cross-node prefix hits drop to zero.

Prefix caching for Mamba and hybrid models. 0.28 turned prefix caching on by default for Mamba models (#50991). 0.29 added internal prefill checkpoints, with a 9–25% TTFT improvement reported in the release notes (#52789), and exposed --prefix-cache-retention-interval. For sliding-window and Mamba cache groups, the default changed from dense retention to 0. At 0, vLLM keeps only semantic checkpoints: the latest replay boundary and shared-prefix junctions. A positive value adds periodic checkpoints (it must be a multiple of the block size), and None restores dense retention. Hybrid models using EAGLE or MTP automatically get dense retention back.

Admission control (0.29, #49445). --max-num-queued-reqs and --max-num-queued-tokens let the server reject work beyond a queue bound instead of accepting unbounded backlog. In 0.30, the request cap is shared across API server processes (#54746). These are opt-in, but they are the cleanest way to turn overload into fast failures that your load balancer can route around.

KV-Cache Tiering and Memory: What Moved

Direct answer: 0.28 turned KV offloading into a real tiered hierarchy (GPU, CPU, disk, plus pluggable secondary tiers). 0.29 made KV sizing account for CUDA graph memory. 0.30 added HiSparse, a host-resident tier for sparse-MLA decode that requires Model Runner V2.

The tiering work matters because KV cache, not weights, is what limits concurrency on long-context serving. If you need the fundamentals of paging, quantised KV and eviction, start with our KV cache optimization guide. The version-specific changes are the following.

0.28: disk tier and pluggable managers. SimpleCPUOffloadConnector gained disk offloading (#49644). A secondary tier can be implemented out of tree and loaded via module_path (#51007), and partial secondary-tier load results are now handled (#50321). A canonical CPU layout makes offloaded KV agnostic to the parallelism it was produced under (#48414). That matters when a KV block written by a TP=4 engine is read by a differently sharded one.

Tiering metrics were added (#48798) and then renamed within the same release, from kv_offload_tiering_block_{queries,hits} to ..._chunk_... (#52812). In 0.30 the Prometheus names are vllm:kv_offload_tiering_chunk_queries and vllm:kv_offload_tiering_chunk_hits. Dashboards built against a 0.28 pre-release will show flat zeroes.

Basic CPU offloading is controlled by --kv-offloading-size (GiB, summed across TP ranks) and --kv-offloading-backend (native or lmcache). Offloading stays off until a size is set.

0.29: honest memory accounting. MRV2’s CUDA graph memory profiling (#53306) reserves graph memory before auto-sizing the KV pool. Batch-sharded sampling (#50465), if you enable it, shrinks the transient logits buffer by 1/TP. For a model with a 150k-token vocabulary at TP=8, the per-step logits tensor is an eighth of what it was. On a tight memory budget, that can be the margin between a stable pool and a pool that thrashes under preemption.

0.30: HiSparse and more secondary tiers. HiSparse (#53781) is enabled through HiSparseConnector. Under GPU memory pressure it spills KV pages for sparse-MLA models to pinned host memory, and it serves top-k misses from a per-request GPU hot buffer. The host cache can be shared across TP ranks (#56629). It is V2-only: the runner selection code raises an error if you combine it with VLLM_USE_V2_MODEL_RUNNER=0. 0.30 also added a KVCR secondary-tier adapter (#53624), P2P tier timeouts, and a long list of offload correctness fixes, including async lookups, sliding-window reachability, disk alignment and a DiskBackend buffer race.

The operational point is that 0.30 is materially more robust for offloading than 0.28. If you tried CPU or disk tiers on 0.28 and backed out, the fix list alone justifies a retest.

Fast Start: Restarting Engines Without Reloading Weights

Direct answer: Fast Start in vLLM 0.30 is a persistent per-GPU daemon that holds post-quantisation, tensor-parallel-sharded weights in GPU memory. A restarting engine maps those weights over CUDA IPC with --load-format ipc_cache instead of reading them from storage.

Figure 3: Fast Start restart sequence versus a cold load from model storage.

On a large model, a cold restart is dominated by three steps: reading hundreds of gigabytes of shards, quantising or repacking them, and capturing CUDA graphs. Fast Start removes the first two. You launch one daemon per GPU, which loads and quantises once. The engine then connects over a Unix domain socket, receives CUDA IPC handles, and maps the weights zero-copy.

# From the vLLM 0.30 weight_cache daemon docstring
python -m vllm.model_executor.model_loader.weight_cache.daemon \
  --model /models/my-model --tensor-parallel-size 4

vllm serve /models/my-model --tensor-parallel-size 4 --load-format ipc_cache

Coverage in 0.30 includes FP4 checkpoints (#55465) and multi-node TP (#55468). For multi-node, you run one daemon launcher per node with the same --nnodes, --node-rank and --master-addr you pass the engine, plus a distinct --weight-cache-master-port. Only tensor and expert parallelism are supported; pipeline and data parallelism are rejected at launch.

Graph capture also got faster. Freezing Python’s garbage collector during capture cut capture from 12 s to 2 s, and engine init from 28.9 s to 8.2 s, on H200 (#54646). These are upstream-reported figures for a specific configuration.

The trade-off is memory residency. The daemon’s copy of the weights lives in GPU memory alongside the engine, and it is shared zero-copy rather than duplicated. The daemon therefore becomes a long-lived process you must supervise, version and restart when the checkpoint changes. It fits best in fleets that restart often, such as RL rollout workers that sleep and wake, config-tuning loops, or blue-green deploys on the same nodes. It fits worst where GPUs are shared across models.

A Canary Rollout Plan for the 0.30 Upgrade

Direct answer: roll vLLM 0.30 out through four gates. First, a startup gate that checks runner selection and context length. Then shadow traffic that compares usage fields and speculative-decoding acceptance. Then a small live canary that watches preemptions, latency and long-prompt errors. Then wave-based promotion. Keep the previous image pinned and warm until the last wave settles.

Figure 4: Canary rollout gates for promoting vLLM 0.30 across an inference fleet.

Gate 0: build and pin

Pin by image digest, not tag. Pick the CUDA 13.0 default image or the -cu129 variant to match your node drivers, and rebuild any custom extensions against torch 2.13.0. Record the full vllm serve command, including every env var, as a versioned artefact.

Gate 1: startup checks

Start the canary with production config and no traffic. Check four things:

Runner selection. Grep the log for using the V1 model runner instead and Defaulting to V1 model runner on ROCm. A V1 fallback is not fatal, but it must be a deliberate choice, because it disappears in v0.32.
Context length. Diff /v1/models max_model_len against the old version.
Warnings. Look for Replacing rope_type and Scale-out endpoints are disabled.
KV capacity. Compare the logged KV cache size against the old version. Expect a modest drop from graph-memory accounting. Investigate a large one.

Gate 2: shadow traffic

Mirror a sample of real requests to the canary, discard its responses, and compare them offline.

Usage accounting. In 0.30, reasoning parsers count reasoning tokens so that usage reports reasoning_tokens (#54982). Billing or quota code that assumed the field was absent should be tested against it. Note that the 0.28 notes already flagged the reasoning_content output removal as a breaking client change (#50624).
Speculative decoding. Enable --per-request-spec-decode-metrics (values none, summary, detailed) and compare per-request acceptance against your baseline. MRV2 changes how drafts are verified and supports adaptive verification (#52228). A drop in acceptance length is the earliest sign of a drafter or tokenizer mismatch.
Output quality. For GPTQ or other quantised models, run a fixed eval set through both versions and compare.

Gate 3: live canary

Route around 5% of traffic. Watch four signals:

vllm:request_num_preemptions (new in 0.30). Rising preemptions mean KV pressure, which ties back to graph-memory accounting or a changed batch budget.
TTFT and inter-token latency at p50 and p99.
The HTTP 400 rate on long prompts.
The error rate on any router-called endpoints.

Define rollback thresholds before you start, not after.

Gate 4: waves

Promote in waves (25%, 50%, 100%) with a soak period between each. Remember that deterministic NONE_HASH changes cross-node prefix hashing. During the mixed-version window, distributed prefix-cache hit rates will dip. Do not misread that dip as a regression.

Trade-offs, Gotchas, and What Goes Wrong

Forcing V1 is a loan, not a fix. VLLM_USE_V2_MODEL_RUNNER=0 will get a stubborn workload through 0.30. It also removes you from DSpark, adaptive verification, prefill context parallelism, batch-sharded sampling and HiSparse. With removal targeted for v0.32, it gives you perhaps a month of runway at the current cadence. Use it to buy time while you file or track the gap in #47172, not as a steady state.

Forcing V2 turns silent fallbacks into hard failures. Setting VLLM_USE_V2_MODEL_RUNNER=1 is a good CI setting: it proves your configuration runs on the runner that will survive. In production, remember that a later config change adding a logits processor then fails at startup instead of degrading.

Installed plugins count. The custom-logits-processor check includes entry-point plugins that are installed but unused. A base image carrying an extra package can pin you to V1 without any config change.

DBO on V2 is opt-in and partial. Even with the env var set, 0.30 excludes DBO combined with LoRA, speculative decoding or pipeline parallelism on V2.

GPTQ act-order breaks quality, not loading. Only output evals catch it.

Default drift is invisible in diffs. A config file that does not mention max_num_batched_tokens, retention interval or FlashInfer all-reduce still changed behaviour. Pin the values you care about.

Mixed-version fleets break shared state. Prefix hashing, KV connector layouts and tiering metric names all changed. Plan the upgrade per disaggregated group, where prefill and decode tiers move together, not per node.

Out-of-tree plugins lag. Custom attention backends, KV connectors and quantisation plugins written against 0.28 internals may reference removed metadata properties or undeclared DCP support.

Two-week cadence cuts both ways. Staying current means a small upgrade every two weeks. Falling behind means doing this whole post at once. Neither is free, but the first is cheaper to canary.

Practical Recommendations

Treat 0.30 as the upgrade target, not 0.29. 0.29 is where the runner switch happened, but 0.30 closes several MRV2 gaps and carries the offload fixes. The 0.29-deprecated items are removed in 0.30 anyway, so stopping at 0.29 only postpones the same work.

Plan for v0.32 now. Inventory every configuration that falls back to V1, because each of those is a future breaking change on a two-to-four-week horizon. Run CI with VLLM_USE_V2_MODEL_RUNNER=1 so that fallbacks become build failures you can see.

Treat context length and quantisation as quality risks, not load risks. The YaRN change and the GPTQ g_idx removal both pass a smoke test.

Checklist:

[ ] Rebuild the image against torch 2.13.0 and the correct CUDA variant (13.0 default, or -cu129)
[ ] Install the bitsandbytes plugin, or migrate those checkpoints
[ ] Audit GPTQ checkpoints for desc_act: true and requantize them
[ ] Add --enable-scale-out where routers call /render, /derender or /inference/v1/generate
[ ] Remove the deleted env vars, and move settings to CLI flags or config fields
[ ] Switch entrypoints to vllm serve and vllm serve --grpc
[ ] Set HIP_VISIBLE_DEVICES on ROCm
[ ] Diff /v1/models max_model_len for every model
[ ] Grep startup logs for V1 fallback and Replacing rope_type
[ ] Pin --max-num-batched-tokens explicitly
[ ] Update dashboards for kv_offload_tiering_chunk_* and request_num_preemptions
[ ] Canary with shadow traffic, spec-decode acceptance and long-prompt 400 monitoring
[ ] Keep the previous image digest warm until the final wave settles

Frequently Asked Questions

What is vLLM Model Runner V2?

Model Runner V2 is a rewrite of vLLM’s worker-side model runner, the component that turns scheduler decisions into GPU inputs, runs the forward pass and samples tokens. It keeps per-request state in a stable GPU table, builds step inputs with Triton kernels, and is designed for async scheduling with zero CPU-GPU synchronisation, including under speculative decoding. It became the default for all models in vLLM 0.29.0. No API changes are needed; your vllm serve commands and client code stay the same.

When will vLLM remove Model Runner V1?

The vLLM 0.29.0 release notes declare MRV1 deprecated and say the project is targeting v0.32 for its removal. They also say no further MRV1-specific improvements will be accepted. Given the roughly two-week release cadence, v0.32 is plausibly weeks away, but the project has published no firm date. Treat any configuration that currently falls back to V1 as an open migration item, and track the upstream issue #47172 for which remaining gaps have closed.

How do I force vLLM to use Model Runner V1 or V2?

Set the environment variable VLLM_USE_V2_MODEL_RUNNER. A value of 0 forces V1, 1 forces V2, and leaving it unset lets vLLM choose automatically. Forcing V2 validates your configuration and fails at startup if you use a feature V2 does not yet support. Some features override the variable: HiSparse requires V2 and errors if you set it to 0, and watermarking also forces V2. Forcing V1 disables V2-only features such as DSpark and adaptive verification.

Why did my model’s context length shrink after upgrading to vLLM 0.30?

vLLM 0.30 aligned YaRN with Transformers. Vendor YaRN aliases no longer multiply max_position_embeddings by the scaling factor a second time, so the derived max_model_len drops for affected models. The release notes cite TeleChat3-36B-Thinking going from 131,072 to 32,768. If you did not set --max-model-len, the change is silent until long prompts get rejected. Compare /v1/models output before and after upgrading, and validate any longer window with a long-context eval before pinning it.

What does –enable-scale-out do in vllm serve?

In vLLM 0.30, --enable-scale-out registers the scale-out endpoints /render, /derender and /inference/v1/generate. Plain vllm serve no longer exposes them. The flag replaces the removed VLLM_ENABLE_SCALE_OUT_ENDPOINTS environment variable. vllm launch render and vllm serve --tokens-only still register these endpoints automatically. If a router or disaggregated serving layer calls them and you upgrade without the flag, those calls will fail.

Is it safe to skip vLLM 0.29 and go straight from 0.28 to 0.30?

Yes, and it is usually the better choice. 0.30 contains everything 0.29 introduced, including the Model Runner V2 default, and then removes the items 0.29 deprecated. Going to 0.29 first only means doing the env-var cleanup twice. The risk is concentration: you absorb the runner change, the removals and the YaRN change in one step. That is why a staged canary with context-length diffs and output-quality evals matters more than which release you stop at.