Distributed LLM Inference Gets a Standard
llm-d 1.0 brings a stable API, multi-tenant QoS, and agent-aware session routing to distributed LLM inference on Kubernetes, with IBM, Red Hat, Google, and NVIDIA behind it. It's not a model server — it's the routing and scheduling layer on top of vLLM or SGLang, aimed at the gap between "it works in a demo" and "it works in production."
Read more →
