AMD's acquisition of Toronto startup Taalas bets on a structural alternative to GPU-based inference: model-specific integrated circuits that etch weights into mask-ROM on the die itself, eliminating the memory-bandwidth wall that constrains all-general-purpose accelerators. Taalas's HC1 claimed 17,000 tok/s for Llama 3.1 8B at one-tenth the power of an H200. The tradeoff is inflexibility — a finished chip runs exactly one model — but AMD sees a disaggregated future where Taalas handles token generation and Instinct GPUs handle prefill.
Wafer.ai's benchmarks of Kimi K3 on AMD MI355X tell a story that goes beyond the numbers: the hardware was capable all along, but two ROCm bugs were blocking it. Fixing them yields 48 tok/s per GPU-dollar, against 7 for B200 — a gap that challenges the assumption that NVIDIA owns frontier model inference.
A detailed engineering account of bringing DeepSeek-V4-Flash up on AMD MI300X reveals the real cost of AMD's software ecosystem gaps: FP8 format fragmentation, missing kernels, and HIP graph constraints that each required dedicated engineering effort before getting to 2,700 tokens/s.
