vLLM’s Beam recipe exposes the hardware and serving trade-offs behind a 501B MoE

Reflection AI’s Beam is a 501-billion-parameter mixture-of-experts model, but only 23 billion parameters are active for each token. A new vLLM Recipes deployment guide turns that architecture into concrete serving choices across NVIDIA Hopper and Blackwell systems.

What the recipe makes deployable

The guide covers three checkpoints: a 294 GB NVFP4 variant, a 503 GB FP8 variant and a 1,003 GB BF16 variant. vLLM recommends the NVFP4 checkpoint on four B200 or B300 GPUs, or eight H100s; FP8 targets eight H200s; BF16 needs eight B200 or B300 GPUs. The model combines sliding-window and full-attention layers, supports a 1,048,576-token context window, and uses Harmony-format reasoning and tool calls.

For Blackwell, the default NVFP4 command uses tensor parallelism across four GPUs. The guide says that layout consumes about 73 GB of weights per GPU, leaving enough capacity for long contexts and allowing two replicas on an eight-GPU node. On H100, the same checkpoint uses eight GPUs and explicitly selects the Marlin MoE backend because the recipe’s measurements found it faster than vLLM’s default choice.

The important boundary

Beam support has not shipped in a regular vLLM release. The recipe requires the project’s vllm/vllm-openai:reflection-beam image and says Beam’s kernels, including its expert-parallel custom all-reduce, are CUDA-only. Teams should treat the recipe as an ahead-of-release integration path rather than evidence of portable, generally released support.

The first Blackwell start can also take about 10 minutes while FlashInfer autotunes its MoE kernel. That sits close to vLLM’s default 600-second engine-ready timeout, so the generated Docker command raises the timeout to 3,600 seconds.

How to choose a serving layout

For a straightforward single-node deployment, the guide recommends tensor parallelism. For prefill-heavy traffic at high concurrency, it documents data-parallel attention with expert parallelism and FlashInfer’s CuTe DSL NVFP4 MegaMoE kernel. That path is limited to Blackwell and does not support tensor-plus-expert parallelism.

The optional DFlash2 speculative drafter is aimed at higher-throughput decoding under load. It adds about 1.7 GB per GPU, can propose up to 15 tokens and defaults to seven in the recipe. vLLM cautions that acceptance depends on natural-language input, so operators should benchmark with representative text rather than random-token datasets.

The practical takeaway is not merely that Beam fits. The recipe identifies which quantization, backend and parallelism choices make it fit—and where the current integration remains experimental.