Agentic AI news, research notes, and industry analysis.
Tag
10 stories in this tag.
LFM2.5-2.6B and Needle2 arrived this week at opposite ends of the weight-class spectrum — one trimmed but architecturally orthodox, the other stripped of its feed-forward layers entirely — and together they define the two credible paths to running a real tool-calling agent on constrained hardware.
Read more →Meta's Muse Glimmer is a 30B open-weight model distilled from Muse Spark 1.2, targeting local agent workflows on a single consumer GPU. It slots in above LFM2.5-2.6B and Needle2 in the increasingly crowded on-device agent tier, with Apache 2.0 licensing, hybrid attention, and speculative decoding via a dedicated drafter model.
Read more →DeepGrove's Maple-Preview is a 20B ternary-weight MoE reasoning model trained from scratch at {-1, 0, +1} precision — not quantized down from float. The 5.31 GB checkpoint runs at 218 tokens/second on an M4 Mac mini and 120 tok/s on iPhone, with competitive AIME and GPQA-D scores and an MIT license.
Read more →Swiftlet is a new Swift+Metal runtime that runs 80B-parameter Qwen3-Next in 4.3 GB of RAM on an M5 Mac — not through compression magic, but by exploiting a structural property of MoE models: they activate only about 3B parameters per token regardless of total size. The missing piece is fast enough SSD I/O to stream expert weights on demand, which Apple silicon happens to provide.
Read more →Cactus Hybrid adds a confidence probe to Gemma 4 that reads internal activations to score each completion 0–1 and routes low-confidence queries to a cloud model. 65–85% of queries stay on-device; overall accuracy matches Gemini 3.1 Flash-Lite. The probe generalizes to audio (0.79–0.88 AUROC) despite no audio training data.
Read more →PrismML released Bonsai 27B on July 14: 1-bit binary and ternary builds of Qwen3.6-27B that fit in 3.9 GB and 5.9 GB respectively, run at 11 tok/s on an iPhone 17 Pro, and retain over 90% and 95% of full-precision benchmark performance. The compression factor is around 14× versus FP16, and the models are available under Apache 2.0.
Read more →Inscribe's benchmark of Apple's new SpeechAnalyzer API on macOS 26.5.1 finds it achieves 2.12% word error rate versus Whisper Small's 3.74%, while running three times faster — at the cost of covering roughly 30 languages instead of 100+.
Read more →FUTO released the models behind their swipe keyboard — a three-component stack totalling 2.5 million parameters that achieves 26% fewer errors than Gboard on their benchmark. It trains on one workstation GPU, runs on low-end Android devices in milliseconds, and is the first freely licensed open swipe-typing model. It's a reminder that model scale is a tool, not an objective.
Read more →NVIDIA's RTX Spark puts a Blackwell GPU and full CUDA stack inside a laptop SoC — enough to run a 120B-parameter model locally with 1M-token context. At roughly the same moment, Perplexity shipped a hybrid inference orchestrator that uses a compact on-device model to automatically decide which tasks stay local and which escalate to the cloud. Together they sketch what a local-AI platform actually looks like in hardware and software.
Read more →Cactus Compute released Needle, a 26M-parameter MIT-licensed model for on-device function calling that strips out all feed-forward networks from the transformer. The architectural choice is a thesis: tool calling is retrieval-and-routing, not reasoning, and attention is the right primitive for it. The numbers are striking — 6000 tok/s prefill on consumer hardware — even if the playground has rough edges.
Read more →