Can't the attacker then jailbreak the first LLM to generate jailbreak with actions for the second one?
See also CaMeL https://simonwillison.net/2025/Apr/11/camel/ which incorporates a type system to track tainted data from the Quarantined LLM, ensuring that the Privileged LLM can't even see tainted _data_ until it's been reviewed by a human user. (But this can induce user fatigue as the user is forced to manually approve all the data that the Privileged LLM can access.)
Non-deterministic security feels like a relatively new area.
https://gandalf.lakera.ai/baseline
This thing models exactly these scenarios and asks you to break it, its still pretty easy. LLMs are not safe.