live wire
▸Red Hat ships important MicroShift 4.16.73 security updateRed Hat Product Errata▸AI · Red Hat maps local AI Inference serving and load tests on RHEL (Oct. 15)Red Hat Emerging Technologies▸PLATFORM · IBM guide maps Secure Execution VMs onto OpenShift Virtualization for LinuxONEIBM Community▸AI · vLLM maps Nemotron 3.5 Super VL across Hopper, Blackwell and Red Hat DSpark (Oct. 15)vLLM Recipes▸AI · AWS contributes push-based KV transfer to vLLM, reporting 3x better TTFT under loadAWS Machine Learning Blog▸AI · AWS contributes bidirectional KV transfer to vLLM, cutting multi-turn TTFT up to 3.3xAWS Machine Learning Blog▸AI · vLLM Semantic Router adds d3-turbo with box and video-tracking decisionsvLLM Semantic Router▸IDENTITY · Keycloak DevDay 2027 opens call for deep technical talks through Oct. 24Keycloak▸AI · vLLM Recipes maps Muse Spark Open to single-node Blackwell and AMD deployment (Oct. 16)vLLM▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security▸Red Hat ships important MicroShift 4.16.73 security updateRed Hat Product Errata▸AI · Red Hat maps local AI Inference serving and load tests on RHEL (Oct. 15)Red Hat Emerging Technologies▸PLATFORM · IBM guide maps Secure Execution VMs onto OpenShift Virtualization for LinuxONEIBM Community▸AI · vLLM maps Nemotron 3.5 Super VL across Hopper, Blackwell and Red Hat DSpark (Oct. 15)vLLM Recipes▸AI · AWS contributes push-based KV transfer to vLLM, reporting 3x better TTFT under loadAWS Machine Learning Blog▸AI · AWS contributes bidirectional KV transfer to vLLM, cutting multi-turn TTFT up to 3.3xAWS Machine Learning Blog▸AI · vLLM Semantic Router adds d3-turbo with box and video-tracking decisionsvLLM Semantic Router▸IDENTITY · Keycloak DevDay 2027 opens call for deep technical talks through Oct. 24Keycloak▸AI · vLLM Recipes maps Muse Spark Open to single-node Blackwell and AMD deployment (Oct. 16)vLLM▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security
upstreambeat.ai
releaseAI

vLLM 0.31 turns fast restarts and request trust into upgrade decisions

The release adds GPU-resident weight preloading and broader distributed-serving controls while changing multimodal request handling and several operator-facing flags.

Before and after vLLM 0.31 upgrade changes
Timeline: dates from the story
By The Release Desk· Oct 5, 2026the quick take — two AI hosts go live when you do

vLLM 0.31.0 adds a GPU-resident restart path, expands large-scale serving controls, and changes several request and configuration defaults. The combination makes this more than a routine package refresh for platform teams operating shared inference services. The upstream release was published October 5, 2026.

What changed

The new vllm preload command starts a weight-cache daemon that keeps post-quantized model weights in GPU memory across engine restarts. The release extends that path to data parallelism and multi-token-prediction draft models, and adds health and readiness checks. An experimental snapshot workflow also uses CRIU to restore a fully initialized tensor-parallel-one engine.

Distributed-serving changes include a MoonEP all-to-all backend, prefill context parallelism with data parallelism, DeepEPv2 sequence parallelism, and back-pressure detection for KV-cache offloading. The scheduler also gains --max-num-active-seqs, separating active admission from the existing maximum sequence setting.

The security boundary for multimodal requests is stricter. Per-request mm_processor_kwargs and media_io_kwargs are rejected unless the server starts with --trust-request-mm-kwargs. The release also changes prefix-cache hashing to distinguish extra-key sources and include the LoRA path.

Who it affects

Operators that restart engines during model rollouts can evaluate preloading as a way to avoid reloading post-quantized weights. Teams running disaggregated or expert-parallel serving have new controls, but should benchmark them against their current topology rather than assume an automatic gain.

API owners need to review any client that sends per-request multimodal processor or media I/O options. Those requests now fail by default unless the deployment deliberately opts into trusting them.

What to do

Test 0.31.0 in staging with production request shapes and the same accelerator topology used in service. Audit clients for the newly gated multimodal fields before rollout.

The upgrade also requires a configuration review: tokenizer_mode="slow" is removed; --enable-mamba-fine-grained-prefix-cache is renamed to --enable-mamba-shared-prefix-checkpoint; XPU graphs are enabled by default; and several quantization and backend options changed or disappeared. Pin the release, compare startup and steady-state behavior, and update manifests before promoting it into a shared serving environment.

sources

Filed by The Release Desk. Corrections: desk@upstreambeat.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.

    more from the desk