vLLM Semantic Router adds d3-turbo for box and video-tracking decisions

vLLM Semantic Router has published d3-turbo, a 26.09-billion-parameter multimodal decision model that adds object boxes and video tracking to the Decision 3.0 family. The Apache-2.0 checkpoint accepts text or JSON plus images and video, then returns typed answers and probabilities rather than generated prose, according to the model card.

What changed

The earlier Decision 3.0 models could classify shared inputs with choice, yes-or-no and score questions. d3-turbo adds two spatial outputs: box locates one target in an image or at a specified moment in a video, while track returns a sequence of boxes and visibility probabilities across a clip. Images can also appear inside individual questions or answer options, so they do not have to be exposed to every question in the request.

The model reads the state once and reuses it across all questions. That design targets routing and evaluation jobs that need several decisions about the same document, image or clip. The published checkpoint supports one to 255 questions when run locally and a 131,072-token context per question when long-context mode is enabled; its hosted test API imposes lower request limits.

What the project reports

On one AMD Instinct MI325X, the project reports that d3-turbo answered 64 questions about one photo in a median 867 milliseconds, compared with 8.26 seconds for d3-flash, which rereads the image for each question. d3-flash remained faster for a single question, at 154 milliseconds against 302 milliseconds, while d3-turbo became faster from four questions onward. At 255 questions, the project reports 2.12 seconds for d3-turbo versus 32.94 seconds for d3-flash.

The model card also reports 0.90 box accuracy at an intersection-over-union threshold of 0.5, 0.96 confidence AUROC and 0.97 presence F1 for video tracking at both one and two updates per second. Its 70.2 text and 72.9 vision index scores are explicitly labeled as the project’s own estimates rather than official board results. These source-native measurements should therefore be treated as claims to reproduce, not independent validation.

What developers should test

The checkpoint is available through Transformers using custom model code, with the quickstart pinning Transformers 5.17.0. Teams evaluating it should compare both decision quality and latency against the number of questions they actually ask: d3-flash may still be the better fit for one lightweight classification, while d3-turbo’s shared-state design is aimed at batches of related decisions.

The box interface always returns exactly one box, even if the requested object is absent. The project therefore recommends pairing localization with a yes-or-no presence question when absence is possible. For video, operators should also test their own clip lengths and sampling rates rather than generalizing from the published hummingbird examples and single-GPU measurements.