Wasm · AI Beat

07 Jul 2026 · AI Beat Desk

Seven Megabytes of Semantic Search

Ternlight ships a sentence embedding model as a 7MB WASM bundle that runs on CPU in the browser — no API, no model download, no GPU required. Ternary weights are the key to the footprint; the result is semantic search you can include in an npm install.

19 Apr 2026 · AI Beat Desk

When the Sandbox Shares the GPU's Memory

A blog post published April 18 describes a technique for running LLM inference inside a WebAssembly sandbox at near-native GPU speed on Apple Silicon. By overriding Wasmtime's memory allocator to back Wasm linear memory with a Metal buffer via makeBuffer(bytesNoCopy:), the author collapses the Wasm–GPU boundary entirely: 0.03 MB overhead vs 16.78 MB for the copy approach, ~9 ms/token for Llama 3.2 1B on M1, and KV cache snapshots that restore 5.45× faster than recomputing prefill.