Distributed LLM Inference Gets a Standard

llm-d 1.0 brings a stable API, multi-tenant QoS, and agent-aware session routing to distributed LLM inference on Kubernetes, with IBM, Red Hat, Google, and NVIDIA behind it. It's not a model server — it's the routing and scheduling layer on top of vLLM or SGLang, aimed at the gap between "it works in a demo" and "it works in production."

Read more →

LLMs Know the Raft Paper. They Don't Know Etcd.

SysMoBench, a new benchmark from the Specula team, tests whether LLMs can produce TLA+ formal specifications that accurately model the behavior of real distributed system implementations. They score near-perfect on syntax and only ~46% on conformance and ~41% on invariant checking — because they model the algorithm as described in papers, not as implemented in code.

Read more →