> despite bulletproof input sanitization not having been invented yet!
I don’t think it can be.¹
¹ https://matthodges.com/posts/2025-08-26-music-to-break-model...
I don’t think it can be.¹
¹ https://matthodges.com/posts/2025-08-26-music-to-break-model...
Work is still being done on how to bulletproof input “sanitization”. Research like [1] is what I love to discover, because it’s genuinely promising. If you can formally separate out the “decider” from the “parser” unit (in this case, by running two models), together with a small allowlisted set of tool calls, it might just be possible to get around the injection risks.
[1] Google DeepMind: Defeating Prompt Injections by Design. https://arxiv.org/abs/2503.18813
At a fundamental level, having two contexts as suggested by some of the research in this area isn’t enough; errors or bad LLM judgement can still leak things back and forth between them. We need something like an SQL driver’s injection prevention: when you use it correctly, code/data confusion cannot occur since the two types of information are processed separately at the protocol level.
Simon Willison has a good explainer on CaMeL: https://simonwillison.net/2025/Apr/11/camel/
I have significant doubt that a P-LLM (as in the camel paper) operating a programming-language-like instruction set with “really good checks” is sufficient to avoid this issue. If it were, the P-LLM could be replaced with a deterministic tool call.