There are two things worth paying attention to in Qwen3.6-35B-A3B, released yesterday by Alibaba’s Qwen team under Apache 2.0. One is the practical story — a model with roughly frontier-level coding performance that runs on a single machine. The other is the architecture, which is unusual enough to deserve a closer look.
The model has 35 billion total parameters but only 3 billion active per token. That’s an aggressive mixture-of-experts setup: 256 total experts, with 8 routed plus 1 shared expert active at inference time. The effective compute footprint during a forward pass is much smaller than the parameter count implies, which is why it fits on a laptop at all. Simon Willison ran it locally and found it drew a better pelican-on-bicycle SVG than Claude Opus 4.7 — a narrow and somewhat absurd benchmark, but illustrative of what “3B active” can actually do.
The architecture is where things get technically interesting. Instead of uniform full attention across all layers, Qwen3.6 alternates between two attention types: Gated DeltaNet blocks and standard Gated Attention blocks. There are 40 layers total, with this alternation built into the design.
Gated DeltaNet is a form of linear attention. Standard softmax attention has quadratic complexity in sequence length — — because every token attends to every other token. Linear attention approximates this with a recurrent state that processes tokens sequentially, keeping complexity at . The problem with naive linear attention has always been expressiveness: the fixed-size state can’t capture everything full attention can. DeltaNet addresses this through a delta rule update mechanism that makes the recurrent state more selective about what it retains, improving associative recall and reducing the practical gap with full attention.
The practical consequence is a model that handles 262,144 tokens natively (extensible to roughly 1 million via YaRN scaling) without the quadratic memory costs that would typically require specialized infrastructure. At 128 dimensions per head with 32 linear attention heads in the DeltaNet layers, and 16 standard attention heads in the Gated Attention layers, it’s a carefully engineered hybrid rather than a full linear-attention replacement.
The other architectural detail worth noting: the MoE routing uses 8 active experts out of 256, with heads split as 16 queries and 2 key-value pairs in the standard attention layers (grouped-query attention). This keeps KV cache size manageable during long-context inference, which matters when you’re targeting agent workloads where sessions accumulate context across many tool calls.
On benchmarks, 73.4 on SWE-bench Verified is the headline coding number. AIME 2026 at 92.7 and GPQA Diamond at 86.0 round out the reasoning picture. These are numbers that, a year ago, would have required a frontier proprietary model; now they come from something you can run with ollama pull qwen3.6:35b-a3b.
The “thinking preservation” feature is worth calling out for agentic use. When enabled, the model retains its chain-of-thought from previous turns rather than discarding it, so a multi-turn coding session doesn’t start from scratch on each exchange. The flag is "preserve_thinking": True in the API. For iterative debugging loops — where context from prior reasoning steps is genuinely useful — this is a meaningful quality-of-life feature rather than a gimmick.
Apache 2.0 means it can be used commercially without licensing friction, which is the other half of why this matters. A model that benchmarks well and requires no proprietary API agreement is increasingly the default starting point for teams building internal tooling.
The trend line here is clear enough: the gap between what runs locally and what requires a frontier API is narrowing in ways that make practical sense. The interesting question isn’t whether open models are “catching up” in the abstract — it’s which specific workloads now genuinely belong on open models, and Qwen3.6’s coding performance suggests the answer includes a lot of software engineering tasks.