Agentic AI news, research notes, and industry analysis.
Tag
3 stories in this tag.
Boyd Kane's essay argues that inference engines — complex parsing stacks supporting hundreds of model architectures — represent an underappreciated attack surface where a malicious model's output could be mistaken for code to execute. CVE-2025-9141 in vLLM, which passed Qwen3 Coder tool-call arguments directly to eval(), shows this isn't theoretical.
Read more →Ray 2.58.0, released August 23, completes the KV-cache-aware request routing work previewed in 2.57: tokenization now happens on the LLMRouter ingress replica, tokens are passed out-of-band to the engine, KV lifecycle events broadcast to all routers, and CPU-offloaded cache blocks count toward cache hit scoring.
Read more →Two papers published this week attack the KV cache memory bottleneck from opposite directions: one proposes sharing key and value projections at training time for a 50% cache reduction with 3.1% perplexity cost, the other quantizes stored cache values to 4-bit keys and 2-bit values with no calibration required and throughput above FP16. Together they suggest the cache is far more compressible than inference engineers typically assume.
Read more →