You turn all permissions on if you are actually using it as if it is Node for writing servers. But in the case when you are using a (potentially untrusted) code snippet to perform certain operations (like encode/decode base64 values), you do might want a sandbox.
(For the case of Node-like usage, we actually also have a basic implementation of whitelisted directories/files for certain permissions, though not perfect atm)
We have also discussed about possibilities of implementing an in-memory file system. (Maybe we will also bring sessionStorage to Deno)
Flags per module is slightly trickier, while whitelisting of certain feature is trivial to implement.
Package signing is indeed a topic that Deno contributors need to discuss further and possibly finding a solution.