The first LLM doesn’t have to be thought of unconstrained and freeform like ChatGPT is. There’s obviously a risk involved, and there are going to be false positives that may have to be propagated to the end user, but a lot can be done with a filter, especially when the LLM integration is modular and well-defined.
Take the second example here. [0] This is non-trivial in an information extraction task, and yet it works in a general way just as well as it works on anything else that’s public right now.
There’s a lot that can be done that I don’t see being discussed, even beyond detection. Coercing generation to a format, and then processing that format with a static state machine, employing allow lists for connections, actions, and what not. Autonomy cannot be let loose without trust and trust is built and maintained.