> connecting their output to anything important is a very bad idea for now.

This may be true for images, but at least for text, treating an LLM as an untrusted person in your threat models will at least allow you to apply defense in depth to the downstream systems consuming from the LLMs

Tldr: engineer other systems to treat LLMs as a potentially bad actor by default.