Agentic AI news, research notes, and industry analysis.
Tag
6 stories in this tag.
Ray 2.58.0, released August 23, completes the KV-cache-aware request routing work previewed in 2.57: tokenization now happens on the LLMRouter ingress replica, tokens are passed out-of-band to the engine, KV lifecycle events broadcast to all routers, and CPU-offloaded cache blocks count toward cache hit scoring.
Read more →Stripe has agreed to acquire OpenRouter for over $7 billion — a company whose CEO described it as "the equivalent of Stripe for AI." The deal puts a single payment processor in control of the routing layer that sits between developers and 400+ AI models, raising real questions about what a non-neutral router means for the model access market.
Read more →Databricks talked to engineering leaders at Stripe, Coinbase, Uber, and Ramp and wrote up what they're doing about AI coding costs at scale. The playbook looks a lot like cloud cost management circa 2013: smart routing, caching, vendor-neutral abstraction layers, and progressive controls instead of hard caps. 30% cost reduction from routing; ~50% token reduction from compaction. The infrastructure is now real enough to need its own infrastructure.
Read more →TracerML's Echo, in public alpha today, routes each request across a coordinated fleet of open-weight models and claims Fable-comparable results at one-third the inference cost. The core insight isn't just "use cheaper models when you can" — it's that models are complementary, and a weaker model overall can still outperform a stronger one on specific problem types.
Read more →Cactus Hybrid adds a confidence probe to Gemma 4 that reads internal activations to score each completion 0–1 and routes low-confidence queries to a cloud model. 65–85% of queries stay on-device; overall accuracy matches Gemini 3.1 Flash-Lite. The probe generalizes to audio (0.79–0.88 AUROC) despite no audio training data.
Read more →OpenRouter raised $113M in a Series B led by CapitalG, with participation from NVIDIA, Databricks, Snowflake, ServiceNow, and MongoDB. The platform grew from 5 trillion to 25 trillion weekly tokens in six months. The round signals that model routing — the layer that sits between applications and the expanding zoo of frontier models — is now considered infrastructure worth owning.
Read more →