Agentic AI news, research notes, and industry analysis.
Tag
7 stories in this tag.
Cactus Compute released Whistle, a 16.9MB speech-to-text model that beats Whisper base on most benchmarks at less than a ninth of the size. It runs on CPU with no dependencies, targets 17 platforms including microcontrollers and WASI, and shares its runtime engine with the company's Needle LLM.
Read more →Cactus Compute's Needle 3 ships as an 8–29 MB on-device model with a novel Laddered Simple Attention Network architecture, beating models 10× its size on mobile tool-calling benchmarks. Released today under Apache 2.0, it's a clear marker of how far specialized small models have come for automation tasks.
Read more →FreeToken, from a team at UC Berkeley and MIT, proposes a serving stack that treats a personal machine's CPU, GPU, and RAM as a single elastic compute surface, adapting to what's actually available rather than committing to a fixed offloading strategy. The result: a 35B model on an 8GB laptop GPU, 284B on a gaming desktop, and the 753B GLM-5.2 on a single workstation with one high-end GPU.
Read more →LFM2.5-2.6B and Needle2 arrived this week at opposite ends of the weight-class spectrum — one trimmed but architecturally orthodox, the other stripped of its feed-forward layers entirely — and together they define the two credible paths to running a real tool-calling agent on constrained hardware.
Read more →OpenCV 5.0 ships a ground-up rewrite of its DNN engine: ONNX operator coverage jumps from 22% to 80%+, and native LLM/VLM support lands in a library already deployed across embedded systems, medical devices, and industrial hardware that can't run PyTorch.
Read more →PrismML's Bonsai Image 4B applies 1-bit and ternary quantization to a FLUX.2 Klein diffusion transformer, compressing it 8.3× to 0.93 GB — small enough to generate images on an iPhone in under 10 seconds. It's the first demonstration that extreme quantization techniques developed for language models transfer cleanly to diffusion architectures.
Read more →Liquid AI ships LFM2.5-8B-A1B, a 38T-token trained hybrid model where 18 of 24 layers are gated convolution blocks rather than attention — and it reaches 253 tokens/second on an M5 Max CPU with under 6 GB of memory.
Read more →