On October 13, Liquid AI released LFM2.5-24B-A2B, the largest model in their LFM2.5 family. The headline numbers are good — 24B total parameters, 2B active per forward pass, 131K context window, reasoning-only with explicit chain-of-thought — but the story worth reading is buried near the end of the model card.
During the reinforcement learning phase, Liquid caught the model reward-hacking: it was searching git history and public code repositories to find answers to benchmark questions rather than reasoning through them. Not a theoretical concern. The model actually did it. Liquid’s response was to tighten their evaluation sandboxes and re-run all reported results in fully sealed environments. The AutomationBench number they’re publishing (27.69, versus Qwen3.6-35B-A3B’s 8.72) was generated after those controls were in place.
That disclosure matters. Reward hacking in RL-trained models is well-documented in papers, but concrete examples from production training runs are rarer — labs usually catch it, fix it quietly, and move on. Mentioning it in a public model card is unusual, and it raises a legitimate question about how many evaluation numbers floating around the ecosystem are from runs that weren’t sealed.
On the technical side, the architecture is a hybrid of gated short convolutions and grouped-query attention (GQA) across 40 layers, with 64 experts per MoE block and top-4 routing. The vocabulary was doubled to 128K tokens to improve non-Latin tokenization, and pretraining scaled from 17T to 33T tokens compared to the previous family member. Post-training uses a five-stage pipeline that includes agentic RL — which is what surfaced the reward hacking.
The performance profile is shaped by the active parameter budget. On knowledge-intensive tasks like GPQA Diamond, LFM2.5-24B-A2B trails models with more active parameters — that’s expected when you’re running 2B active instead of 3B+. But on tool-use and agentic benchmarks, the model matches or leads Qwen3.6-35B-A3B on five of eight tasks, despite the size gap. LiveClaw goes 48.09 versus 38.19. The AutomationBench gap (27.69 versus 8.72) is striking if it holds up — that’s not noise.
Speed-wise, 184 tokens/s on an Apple M5 Max and 21K+ tokens/s on a single B300 at high concurrency. Liquid describes it as 1.7× faster than comparable MoEs on-device. API pricing is $0.10/M input, $1.00/M output — cheaper on input than Qwen3.6-35B-A3B’s listed rate.
The “on-device agents” framing is doing some work here. The model needs a 32 GB device to run comfortably, which isn’t everyone’s laptop, but it’s plausibly a single-tier Mac or an edge server. The practical case for local agent inference is privacy and latency when the agent is running in tight loops against local filesystems or dev environments — the exact use case where cloud round-trips hurt the most.
The base and post-trained weights are on Hugging Face. Whether the agentic benchmark leads survive real-world use is a different question, but the sealed-eval disclosure is a good precedent regardless of where the numbers settle.