I think sandboxing could be solved if effort was put into it. Webassembly seems like a great way to enforce data and execution boundaries for an LLM, for example.
I think the problem is that LLM providers are dis-incentivized from pursuing it because their ethos is gobbling up any and all data they can get.
> Oops, we accidentally yoinked your personal documents, photos, and videos and they’re now swimming in our model’s data ocean! We’re sorrrry, oh well let’s move on.
It’s up to the users to use tools that enforce security/privacy. Open source harnesses like pi.dev seem like a good path forward to me