live wire
▸Red Hat ships important MicroShift 4.16.73 security updateRed Hat Product Errata▸AI · Red Hat maps local AI Inference serving and load tests on RHEL (Oct. 15)Red Hat Emerging Technologies▸PLATFORM · IBM guide maps Secure Execution VMs onto OpenShift Virtualization for LinuxONEIBM Community▸AI · vLLM maps Nemotron 3.5 Super VL across Hopper, Blackwell and Red Hat DSpark (Oct. 15)vLLM Recipes▸AI · AWS contributes push-based KV transfer to vLLM, reporting 3x better TTFT under loadAWS Machine Learning Blog▸AI · AWS contributes bidirectional KV transfer to vLLM, cutting multi-turn TTFT up to 3.3xAWS Machine Learning Blog▸AI · vLLM Semantic Router adds d3-turbo with box and video-tracking decisionsvLLM Semantic Router▸IDENTITY · Keycloak DevDay 2027 opens call for deep technical talks through Oct. 24Keycloak▸AI · vLLM Recipes maps Muse Spark Open to single-node Blackwell and AMD deployment (Oct. 16)vLLM▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security▸Red Hat ships important MicroShift 4.16.73 security updateRed Hat Product Errata▸AI · Red Hat maps local AI Inference serving and load tests on RHEL (Oct. 15)Red Hat Emerging Technologies▸PLATFORM · IBM guide maps Secure Execution VMs onto OpenShift Virtualization for LinuxONEIBM Community▸AI · vLLM maps Nemotron 3.5 Super VL across Hopper, Blackwell and Red Hat DSpark (Oct. 15)vLLM Recipes▸AI · AWS contributes push-based KV transfer to vLLM, reporting 3x better TTFT under loadAWS Machine Learning Blog▸AI · AWS contributes bidirectional KV transfer to vLLM, cutting multi-turn TTFT up to 3.3xAWS Machine Learning Blog▸AI · vLLM Semantic Router adds d3-turbo with box and video-tracking decisionsvLLM Semantic Router▸IDENTITY · Keycloak DevDay 2027 opens call for deep technical talks through Oct. 24Keycloak▸AI · vLLM Recipes maps Muse Spark Open to single-node Blackwell and AMD deployment (Oct. 16)vLLM▸PLATFORM · Clang 23 bytecode interpreter cuts allocations and expands constant-expression supportRed Hat Developer▸PLATFORM · RHEL becomes first-party on OCI; OpenShift subscriptions reach Oracle MarketplaceRed Hat▸AI · vLLM-Omni merges native MOSS-Image serving with parallelism and offloadvLLM▸AI · vLLM adds System One endpoint for Decision 3.0 multimodal routing models (Oct. 15)vLLM▸SECURITY · Red Hat rates OpenShift AI 3.4.5 update Critical; advisory lists hundreds of CVEsRed Hat Product Security
upstreambeat.ai
guideAI

vLLM maps Muse Spark Open’s 816B model to one Blackwell or AMD node

The recipe separates low-latency, throughput and million-token serving paths—and shows where NVIDIA and AMD support diverge.

By The News Desk· Oct 19, 2026the quick take — two AI hosts go live when you do

vLLM’s deployment recipe for Muse Spark Open turns an unusually large sparse mixture-of-experts model into a set of concrete hardware choices. The model has 816 billion total parameters, activates 42 billion per token and supports a one-million-token context, but the recommended serving path still fits on a single eight-GPU node when operators use the model’s 4-bit variants.

What the recipe establishes

For NVIDIA Blackwell, the primary path is the NVFP4 checkpoint on eight GPUs with tensor parallelism across all eight devices plus expert parallelism. The generated command enables vLLM’s experimental Rust frontend and, by default, Inferact’s DSpark draft model for speculative decoding. The recipe says tensor parallel size must match the GPU count to avoid running out of memory from replicated dense components.

AMD’s validated path is narrower. The recipe lists MI350X and MI355X for Quark MXFP4, load-time BF16-to-MXFP4 conversion and BF16 serving. Those commands are text-only, use the V2 model runner, disable prefix caching and enable chunked prefill. AMD supports tensor parallelism and tensor-plus-expert parallelism in this recipe, but not its data-parallel or prefill/decode-disaggregation choices.

That difference matters because this is not one portable command with two accelerator names swapped. It is a deployment matrix with distinct attention kernels, graph settings, modality support and concurrency ceilings.

Choose for the workload, not the headline context

The recipe recommends plain tensor parallelism for latency-bound AMD serving with only a handful of concurrent requests. Tensor plus expert parallelism becomes the throughput choice around 32 or more concurrent requests for MXFP4, while BF16 should stay on plain tensor parallelism because its Triton experts are slower under expert parallelism.

DSpark changes that boundary. On AMD it helps most at low concurrency and can remain ahead through roughly 64 concurrent requests when paired with expert parallelism. Beyond that, the recipe points operators back to expert parallelism without the draft model. Adaptive DSpark requires Triton attention because AITER’s attention backend does not support the variable-length decode graphs it captures.

The default AMD commands cap the model at 32,768 tokens. Reaching the advertised one-million-token context is a separate validated configuration: MXFP4, plain tensor parallelism, a 1,048,576-token model limit and no more than eight sequences, with either the default or FP8 KV cache.

What teams should validate

Capacity planning should start with the selected checkpoint, concurrency and modality requirements. The recipe estimates roughly 1.64 TB of BF16 weights, about 509 GB for Quark MXFP4 and about 533 GB for NVFP4 before KV cache. Teams should also treat the Rust frontend as an optimization rather than a requirement and switch to Python if an unsupported feature appears.

Most importantly, AMD’s documented commands do not provide the model’s native image, audio and video path. Teams that need multimodal serving, data parallelism or prefill/decode disaggregation should evaluate the NVIDIA path or wait for equivalent validation rather than infer support from the model card alone.

sources

Filed by The News Desk. Corrections: desk@upstreambeat.ai · Our standards →

comments · 0

    Comments are moderated before they appear. Your email is used once to confirm it is you — never shown, never sold. Corrections and questions get an answer from the desk when we have one.

    more from the desk