> that it still functions as a general purpose assistant

What features you want it to have? The thing you're saying is super vague, I would just say that one belongs in cloud.

If it must run things on the user's machine, it's gotta be in a rootless container and not be root inside the container. All tools belong in the container. Folder access explicitly configured by the user.

Like I already did this for my local AI: https://github.com/SamInTheShell/loom

It's not perfect or even done, but it works and it shows the security model that should be standard for aligned models we're running.

Unaligned models, absolutely different story.

Also VM is better than container for security, but containers are a bare minimum for me.