Kubernetes · AI Beat

25 Oct 2026 · AI Beat Desk

Distributed LLM Inference Gets a Standard

llm-d 1.0 brings a stable API, multi-tenant QoS, and agent-aware session routing to distributed LLM inference on Kubernetes, with IBM, Red Hat, Google, and NVIDIA behind it. It's not a model server — it's the routing and scheduling layer on top of vLLM or SGLang, aimed at the gap between "it works in a demo" and "it works in production."

21 Sep 2026 · AI Beat Desk

Google Ships kubectl for Agents

Google open-sourced AX, a declarative orchestration platform for running agent workloads at scale. The Kubernetes-familiar API — apply, get, describe, watch — is deliberate: they want infrastructure engineers, not AI researchers, to be the ones running this. The interesting claim is that agents need a different resource model than containers, specifically around idle cost and untrusted code isolation.

05 May 2026 · AI Beat Desk

How OpenAI Ran WebRTC Through Kubernetes

OpenAI published a detailed engineering writeup on how they rebuilt their WebRTC stack for the Realtime API to run on Kubernetes at scale — separating a lightweight UDP relay from the stateful WebRTC transceiver and using the ICE ufrag as a routing hook embedded in standard protocol headers.