vLLM Semantic Router adds d3-turbo for box and video-tracking decisions
The new 26-billion-parameter checkpoint reads shared context once, answers multiple typed questions and adds object localization and tracking to the Decision 3.0 family.
The new 26-billion-parameter checkpoint reads shared context once, answers multiple typed questions and adds object localization and tracking to the Decision 3.0 family.
The merged path adds OpenAI-style image generation, parallel execution and layerwise offload, but its published performance evidence covers a narrower configuration.
The recipe separates low-latency, throughput and million-token serving paths—and shows where NVIDIA and AMD support diverge.
An AWS-contributed connector overlaps destination setup with prefill work, trading later routing flexibility for lower latency under load.
An AWS-contributed path lets disaggregated vLLM deployments reuse generated context across turns instead of repeatedly rebuilding it.
Red Hat’s latest engineering update shows the experimental constant-expression engine covering more C++ semantics while changing how it represents pointers and integers.