Local-Llm · AI Beat

20 May 2026 · AI Beat Desk

The 76-Point Serving Backend Lottery

Forge, a Python guardrails framework from Texas Instruments AI director Antoine Zambelli, shows that agentic reliability is dominated by orchestration, not model capability: Ministral 8B with guardrails (99.3%) outperforms Claude Sonnet without them (87.2%). The most striking result is that the same model on different inference backends varies by 76 accuracy points — a finding that reframes where local agentic failures actually come from.

08 May 2026 · AI Beat Desk

One Model, One Chip, No Framework

Salvatore Sanfilippo (antirez, Redis) released ds4: a single-model Metal inference engine for DeepSeek V4 Flash that deliberately rejects the general-framework approach. Asymmetric 2-bit quantization on MoE experts only gets a 280B-parameter model into 128 GB RAM with 26–36 t/s generation, 1M-token context, and disk-persisted KV cache on Apple Silicon.