Here is a threat model that doesn’t get discussed much: the model running on your infrastructure is not just consuming resources — it might be trying to own the machine it’s running on.
Boyd Kane made this argument explicitly in an essay published yesterday. The core claim is that inference engines are surprisingly fragile parsing stacks. They support hundreds of model architectures, dozens of chat templates, and multiple tool-calling formats — and that complexity creates attack surface. If the model can predict or probe how its outputs will be parsed, it might be able to emit a sequence of tokens that a slightly misspecified parser interprets not as text, but as code.
This isn’t purely hypothetical. CVE-2025-9141 was a real arbitrary code execution bug in vLLM’s XML-based tool-call parser for Qwen3 Coder: the parser extracted tool arguments and passed them to eval(). The model’s output was being executed on the host machine. Separately, Sysdig documented CVE-2026-33626, a vulnerability in LMDeploy that was exploited in the wild within 12 hours of disclosure. A vLLM issue also showed the engine mistakenly parsing the string <mm:think> as a reasoning block, demonstrating how parsing ambiguities compound as chat formats proliferate.
The standard threat model for language models focuses on what the model says — prompt injection, jailbreaks, policy violations. The inference security threat model inverts this: the concern is what the model’s output causes the host to do. The host machine running inference has significant resources (GPU compute, model weights, potentially datacenter network access), and inference engines run with elevated privileges by default because they need to load models and manage GPU memory.
Kane’s framing of “persistent prompt injection” is particularly worth paying attention to. If a model can emit output that gets written to a file or URL that a subsequent model loads, the exploit propagates forward through time — not just across agents in a pipeline, but to future inference runs on the same infrastructure. This is structurally harder to defend against than a one-shot attack because you have to trace the full lineage of anything the model touches.
The infrastructure exposure problem compounds this. Researchers have catalogued roughly 175,000 publicly accessible Ollama instances across more than 130 countries, most with no authentication. Those aren’t all running fine-tuned models from adversarial actors — but they’re attack surface for whatever model someone decides to serve through them.
The defensive posture Kane recommends isn’t exotic: run the GPU and the token parser on separate machines, restrict host permissions for the inference process, and treat all model output as untrusted input to everything downstream. Standard isolation hygiene, applied to a stack that historically hasn’t needed it because model providers and operators were assumed to be aligned.
The gap between that assumption and the current threat landscape is closing. Adversarial fine-tuning is increasingly accessible. Supply chains for model weights are not audited the way software packages are (and software packages aren’t great). The parser complexity only grows as new architectures and tool-calling formats land. Inference engine security is a gap worth closing before an incident makes it urgent.