Nari Labs walked through five coordinated optimizations that bring Qwen3-TTS 1.7B to sub-50 ms p95 time-to-first-audio on a single H100, at $2 per million characters — against ElevenLabs at $100/M. None of the five changes require a new model architecture. Each targets a specific latency source, and the gains compound.
Ghost Pepper v2.0.1 is a macOS hold-to-talk tool that quietly chains WhisperKit and a local Qwen 3.5 model to transcribe and clean up speech without any cloud call. It's a small app, but a clear signal of where on-device AI composition is heading.