Unsurprisingly, Meta's new Muse AI agent blatantly ignores users permissions
appleinsider.com
appleinsider.com
Permissionless action is about to skyrocket as an issue, but this particular scenario strikes me as incredibly unlikely. Would be interested to know if Muse can provide more meaningful data provenance/logs.
Scanning iMessage dbs as a passive part of full disk access (and not a messages grant), if true, is a little sketchy, regardless.
Even as a technical person, it's not trivial to sandbox agents correctly. The fact that an mis-clicked permission popup could give an agent unrestricted access to a user's disk is a massive risk vector in the hands of lay people who barely understand how any of this works.
So much of current security depends on the model of tying access control to a user account. A lot has to be re-thought in terms of how to grant access to an agent working on the user's behalf, in a way that doesn't make it completely useless, and also doesn't require every user to become a sysadmin managing fine-grained agent permissions manually.
I think the problem is that LLM providers are dis-incentivized from pursuing it because their ethos is gobbling up any and all data they can get.
> Oops, we accidentally yoinked your personal documents, photos, and videos and they’re now swimming in our model’s data ocean! We’re sorrrry, oh well let’s move on.
It’s up to the users to use tools that enforce security/privacy. Open source harnesses like pi.dev seem like a good path forward to me
I.e. if I have an agent running in a WASM sandbox with no access to the host system, I can't ask it to clean up my files. Same thing with things like giving an agent access to your email inbox: doing so allows the agent to provide utility, but it comes with risks, as the agent can delete important emails, or leak sensitive data.
I think a big part of the problem is, a lot of the systems we use and would like agents to help us with don't have a concept of separated roles with different levels of access which can be applied. A lot of times it's all or nothing.
And even when we do have fine-grained access control available, it's a pain in the ass to manage it. Like you can create a GitHub token with fine-grained access control to your repositories and make sure the agent only uses that one to connect, but it's a whole lot easier to use a broad-access token, or just let the agent use your own token, so lots of people will just end up doing that.
And I also like pi, but it's probably one of the worst in terms of sandboxing as it's yolo by default.
I'm a HUGE proponent of putting them behind task gating trees, and sheathing them with QA/QC checks on their processes, especially at this stage. Unfortunately its so easy to create, and all of that takes time and design that many just throw away for getting to results.
Even fine grained ACLs, which are great, dont have the structured approach such autonomous agents need to shore them in (imo).
> And I also like pi, but it's probably one of the worst in terms of sandboxing as it's yolo by default.
Agreed, but the nice thing about pi is the plugin system and how configurable it is. I can easily hack on the pi harness, whereas a more opinionated one like opencode is more difficult
And I love pi - it's my daily driver - but the extension system itself is an attack vector. If any process manages to write an extension to your .pi directory, it could rewrite your prompt to have the agent exfiltrate your secrets, or take whatever action on the host system if you don't sandbox it.
The problem is that, for most potential users to whom personal (not software dev) agents are being marketed, security & privacy for AI agents is a higher load domain to manage than the things they would want to delegate to agents to relieve load, so taking up the required security & privacy management to avoid problems using the agents creates disutility that exceeds the utility of the agent.
Which is why anyone selling them is going to distract from the issue rather than try to inform their customers.
EDIT:
Another problem is that in many cases, BECAUSE of the assumptions about how people use computers, web services, etc., the required tools DO NOT EXIST. Most account based web services DO NOT have ways to create, say, access tokens with a subset of full account permissions that you could give to an agent rather than full control of the account, because that kind of delegation to limited authority actors was never part of the usage model the vendor designed for.
This is all amateur hour shenanigans.
For something like muse that's supposed to be a general-purpose assistant, how do you give it enough access to be useful, without giving it too much access, and creating unacceptable risks? And how do you do that in a way that's comprehensible the average Facebook user who's the target market of this product?
When you're a company running around with more than a few billion in the vault, you have no excuse for the level security negligence going on at every phase of rollout.
I gawked at Cursor executing a python script one time and decided enough was enough, I don't raw dog these tools anymore because their developers are the dumbest people to task with security work. They just don't care.
What features you want it to have? The thing you're saying is super vague, I would just say that one belongs in cloud.
If it must run things on the user's machine, it's gotta be in a rootless container and not be root inside the container. All tools belong in the container. Folder access explicitly configured by the user.
Like I already did this for my local AI: https://github.com/SamInTheShell/loom
It's not perfect or even done, but it works and it shows the security model that should be standard for aligned models we're running.
Unaligned models, absolutely different story.
Also VM is better than container for security, but containers are a bare minimum for me.
Nether service has a way to configure fine-grained access for a secondary user.
How do I go about giving the agent the ability to perform these tasks without exposing myself to the risk of unexpected destructive behavior from the agent?
If you want to design a system specifically to make sure you're not going to get stonewalled for sending me an emdash, you build and maintain a set of permissive rules for sending/reading to avoid my blacklist of people I'll never work with.
Once you got API stuff sorted, just throw a cronjob at it or build a service for that stuff.
^ There be prompts all over the place here, obviously. Realistically your MCP server will end up hacking on POP3.
Meta rushed this out, to grab relevancy, especially after losing out to Sam with OpenClaw.
I just dont fundamentally trust such a blackbox with my data. They've literally turned your data and your life into their biggest asset. For it to now have a level of control in your life just seems... irresponsible.
I say that from coding experience. You don’t want to approve 100 windows to get a task done. In the end the only “sane” solution for my setup was a dedicated machine just for the agent with Bitwarden for secrets and full access/yolo mode.
If you are concerned about the agent deleting everything make sure you have a process backing up the git repositories at least to a separate service/hosting and that’s it…for now.
So the solution is to have backups and a way to restore data…
When I was kicking the tires on pi, one of the first things the agent did was push an update to one of my published Rust crates (not the project it was working on).
That in itself wasn't harmful, but it did convince me it was worth the effort to figure out sandboxing after that.
I’d argue this is a five alarm fire for macOS and Meta simply exploited it.
Open your terminal app and run /Applications/Firefox.app/Contents/MacOS/firefox
This opens a normal-looking Firefox window, but it has whatever permissions you gave to the terminal, which likely has Full Disk Access.
It’s insane.
This used to work when you could trust the software you ran on your system to have access to everything you have access to on your computer. I'd argue that time has largely passed, for most third-party commercial developers and even for some OS vendors.
Best solution is to simply not run software made by blatantly untrustworthy developers. Second best solution would be to run such software as a severely sandboxed user who basically doesn't have access to anything important on your system.
> I'd argue that time has largely passed, for most third-party commercial developers and even for some OS vendors.
Agreed, but what can you do about your OS vendor?
Muse is not available in the macOS app store. Almost certainly because of the sandboxing requirements.
This is not at all how it works on macOS, which is what is being discussed in the original post. There are a million different things that require per-app explicit opt-in permissions. This is a case of user error.
Is this a setting configured in Muse itself?
> It took Meta a single day to begin "helpfully" pitching article ideas based on texts he'd sent to a podcast co-host. When he asked Muse how it got the information, it said that it read banners from incoming texts. But that's not true, either.bAfter doing a little digging, Aten says Muse synced 187,000 lines from his Messages database, despite Full Disk Access being off.
Is full disk access enforced on the OS side, or the app side? Like is this claiming MacOS security was breached by Muse somehow acting in spite of deliberately disabled access somehow?
Has this been reproduced / recorded?