Hacker News Reader

Comment by maggit | Hacker News Reader

"System Integrity Protection (SIP)" seems to be the correct name here, for anyone as confused as me :)

(More details here: https://support.apple.com/en-us/102149 )

Original author here. Thanks for spotting the typo, fixed.

WRT security implications of disabling SIP - I don't think OS becomes any less vulnerable than usual Linux/Windows installation.

And that’s a good thing? I’m not sure the benefits of containers would be worth disabling SIP.

Maybe for an automated test CI/CD running system (which is probably the main use-case), but not on anything that users would interact with.

> And that’s a good thing?

That's a technical limitation.

Technical limitations aren’t excuses for a bad design. If it’s not a good design due to a technical limit, the answer isn’t to sacrifice security for functionality.

If it really isn’t technically possible (which I think you might be able to do in a Darwin VM), then maybe this approach isn’t a good idea.

> Technical limitations aren’t excuses for a bad design.

Tomorrow Apple might decide it is safe to chroot with SIP enabled (I actually do not understand why they restrict it, chroot is a tool to increase security). Does that suddenly convert bad design into a good design? But this is exactly the same design.

Design operates within the contexts of a system. A perfectly viable design can be made bad by a change in the system for which it was designed.

Does this current design require disabling SIP? Then I don’t think it is worth my effort to use (for my use case). If Apple changes the system in the future, my opinion might change.

But a design cannot be judged as good or bad outside of the context for which it was designed.

If you feel this is a better way to tackle the problem, then talk to Apple about it.

Docker Desktop doesn’t require me to disable SIP. Why would I use this if it requires that?

To containerize darwin binaries?

This is not an alternative to remotely connecting to a VM to control Linux containers (which DfM is)

And what is the benefit of that? Who would use that and for what? Containers solved deployment, and software distribution problems related to diversity of runtime environments on linux. It also has some security benefits, but their adoption was successful because it solved real world everyday problems. It eventually allowed better utilization of servers.

They did not have such success on windows, despite Windows also having a container subsystem, as windows servers already did this with IIS web app containerization.

On MacOS desktop software distribution is largely a solved problem since ages. On MacOS/Darwin servers... are there such in industrial use apart from some research installations?

> Who would use that and for what?

I believe this project can be useful for CI and testing scenarios.

> On MacOS desktop software distribution is largely a solved problem since ages

Are you talking about App Store? Or Homebrew? Or MacPorts? Or... Wait, isn't this too many tools for a problem that was solved?

So why don’t they run a Darwin VM that doesn’t need SIP and control it in the same manner?

Docker For Max marshals filesystem events over a VM host/guest boundary which can grind the most powerful computers to a halt if you’re sharing directories between the host and guest. For example, at my last company we developed Python apps and ran them in Docker for Mac containers by mounting the source code directory into the containers (so we wouldn’t need a build step) but as our project grew the filesystem event marshaling became exponentially slower until we eschewed Docker from our dev iteration loop entirely (the fidelity benefits weren’t worth the performance hit). Note: there are lots of projects and hacks that claim to solve this problem but none made an ounce of difference.

The alternative is to use a performant file sharing system through the VM isolation. Some people use NFS, I personally use VMware Fusion + vmhgfs.

Sure it does not attempt to map fsevents<->inotify 1:1 but honestly I can live with that limitation given that it's a 10x performance increase compared to the DfM kitchensink.

I do go native darwin when I can / it makes sense.

Is that something you can configure in DfM or are you saying you sort of roll your own DfM alternative? Also, what does "VM isolation" mean here? How is NFS or VMware Fusion more "isolated" than DfM?

> Is that something you can configure in DfM

> or are you saying you sort of roll your own DfM alternative?

Yes, I set up a NixOS VM and use DOCKER_HOST=ssh://docker@<ip>

If you want to have it easy you can roll with lima/colima (but I found the fs sharing slower than vmhgfs)

> Also, what does "VM isolation" mean here?

The host/guest boundary. The guest is just like any another, remote machine. DsM adds smoke and mirrors to make it look like guest and host are one.

> How is NFS or VMware Fusion more "isolated" than DfM?

It's not, both are VMs and need a way to expose the host fs to the guest, NFS or vmhgfs are a means to that.

Understood. Thanks for the explanation/clarification!

> until we eschewed Docker from our dev iteration loop entirely

Can you say anything more about what you did instead, and how it ended up working out?

We used native MacOS processes and it worked out fine. In practice I don't recall any bugs due to different behavior between our dev environment (MacOS) and our production environment (Linux) and we saved a ton of time during development. Docker just wasn't worth it. This worked for us because our app didn't depend on any platform specific behavior.

Of course, if you are using Go or some other language with a fast, static cross-compilation step, you don't need to mount a source code volume into your container, you can just rebuild the whole container image or rebuild on the host and `docker cp` the new binary onto the target container.

Docker Desktop for macOS does full-blown hardware virtualization, which is just silly.

Docker Desktop runs an entire second kernel in a VM.

This should have faster file access than Docker Desktop. For large projects, disabling SIP may be worth the increase in performance.

If you really want good adoption, you’ll have to figure out a way for devs to try it out without first having to disable SIP.

Is this related to the code you tried to have merged here: https://github.com/containerd/containerd/pull/8789 ?

Nope, that PR was an attempt to upstream my changes: https://github.com/macOScontainers/containerd/commits/macos

Vanilla containerd cannot mount anything on macos.

> If you really want good adoption, you’ll have to figure out a way for devs to try it out without first having to disable SIP.

I can't stress enough how I also would like it to work with SIP enabled!

Sounds like if Apple wants developers who wants to use containers natively, they need to address the problem of not being able to offer this feature without disabling SIP.

No matter what you or I think about what's needed for adoption, technical problems get in the way of the tool working with SIP, so seems it's in Apple's ball court really.

In corporate managed laptops it may not be an option to disable SIP.

Understatement of the year. I am sure there are some places where being caught doing something like that (without authorization) could result in one of those “my hands are tied, I have to fire you” situations.

Think places where security is a big deal, like finance, military, aerospace, critical infrastructure etc.

> I don't think OS becomes any less vulnerable than usual Linux/Windows installation.

is not a good enough argument.

For the story, SIP is Apple's "rootless". Effectively the OS runs with less privileges than root. Disabling SIP significantly increases the attack surface.

That being said, I'm grateful that someone decided to do something more native for containers in macOS.

I think it's an OK argument given that most people run (and have been running with no alternative until very recently) docker in such a way that there's a trivial privesc to root. In general it seems like docker users are, overall, willing to take that tradeoff.

How so? I use docker pretty frequently, but I’m sure that my user is part of the docker group before I do, so I don’t sudo anything.

Is there anything else I should be doing security wise?

I’ve been hearing podman is more secure, but I think it’s still containerd under the hood, so idk how true that is.

In general if you can `docker run` without sudo then that means you have a trivial privesc path since you can do `docker run` with the various flags that run it without any sandboxing, get a shell, and just ask to be let out of the namespace.

The way that podman and newer versions of docker get around this is using unprivileged user namespaces. Unprivileged user namespaces are not a free lunch - in fact, they're a bit of a security disaster in their own right.

In a typical installation, being in a docker group gives you access to a socket that controls docker daemon and that daemon runs as root. `sudo` is not important in this context.

Thankfully there is rootless mode for some time now: https://docs.docker.com/engine/security/rootless/.

Podman, too, can run in rootful and rootless mode. Rootless in podman still feels to me to be more like first class citizen, as opposed to docker case.

In both cases it's important to keep in mind in which mode you operate. Both from the perspective of security and day to day operations, as some aspects of behavior will differ between those modes.

This is not necessarily the case.

On Linux, more or less the entire permissions system makes no assumption about SIP existing (as it doesn't there), so other protections are relied upon to secure the system (such as SELinux, granular directory permissions, etc.).

On both Linux and Windows, TPM and secure boot provide similar protections to SIP on macOS, but are optional (it's encouraged more forcefully on Windows 11).

Removing SIP from a system that relies on it as a basis for platform security is different than using a system that wasn't relying on it in the first place.

macOS had granular directory permissions way back when it was NeXTSTEP, long before SIP was introduced. Where are you suggesting they disappeared to?

It still has them, of course, but the concern is that after ~8 years of SIP basically ~everywhere, platform security decisions have been made assuming it is present.

This concern is definitely not totally unfounded, back in 2019 Chrome shipped an update that rendered systems with SIP disabled unbootable: https://support.google.com/chrome/thread/15235262?hl=en

The "varsectomy" bug in Chrome isn't the example you think it is, because disabling SIP was not sufficient conditions for it. There were 3 other conditions that had to be met, the most notable of which is that "/" had to be writeable by tho logged-in user, which is not the default.

This is an example of defense-in-depth being present, and defense-in-depth still failing for some users who gave escalated permissions to some installers, allowing them to run roughshod over their filesystem permissions, leaving them vulnerable to a subsequent varsectomy. If one did the same thing to their Linux system, the same thing could happen.

https://arstechnica.com/information-technology/2019/09/no-it...

The two other conditions listed in the page you link basically amount to "the buggy code has to run". Related, I think SSV [1], introduced last year, would also have blocked this bug even with SIP disabled. But none of that invalidates concern that areas of the OS we don't know about might not have the level of defence in depth that we would like - it's not like Apple has never cut corners or shipped bugs to hit a date.

1: https://support.apple.com/en-gb/guide/security/secd698747c9/...

One can have vague, general concerns about any operating system if one lets their imagination run wild, though, and one doesn't care about the presence or absence of specific examples supporting the suggestion that the "permissions system...makes assumption[s]" about SIP existing.

I think we have a rather specific example of a Chrome bug hosing OS installs. Had SIP not existed, there's virtually zero chance that bug wouldn't have been caught before shipping.

Here's a better one: https://gist.github.com/ChiChou/e3a50f00853b2fbfb1debad46e50...

What exactly attack vectors you think are possible against macOS without SIP but not possible against Linux?

I think the argument is that file permissions may not be applied as rigorously, with the assumption that SIP is in effect.

SIP won't save you from wrong file permissions.

And SIP doesn't defend you from editing files in /bin. They are guarded by the fact that root filesystem is mounted read-only.

Entitlement stealing, for example.

With SIP enabled it’s not possible to load arbitrary kernel extensions, for one - they must be signed.

I'd like to digress to your "Encouraged more forcefully" phrasing which is quite interesting if you think about it. In my view, it would mean something like pervasive dialog box ala EULA, some UI hoops you need to resolve, alike going with local account on installation.

In reality they done basically everything to force users to use secure boot. If they disabled normal boot altogether, OS adoption would suffer heavily. They could've obscured that option, but it would be found out, and enterprise users would be pissed at them because they didn't gave them a provisionable way while the way exists. So it came down to normal variables in installer registry.

However modifying, e.g making users "hack" the ISO is really as forceful as it gets without market loss.

Note: There may be more normal way today than modifying the registry of ISO, I installed 11 once when it came out.

> I don't think OS becomes any less vulnerable than usual Linux/Windows installation.

A modern Linux with SELinux enabled (the default in e.g. Fedora) running apps inside rootless containers (Podman doesn’t even need a daemon) is likely much more secure than your default MacOS or Windows.

I think you're probably @totallywrong

Well Linux powers just about all the most important systems in the world and SELinux was originally developed by the NSA, but I'd love to get your insights.

TBH, Fedora Silverblue with the parent commenter's approach it's far more secure.

If you're talking about desktop use, not really. All the software in a default Fedora desktop install, runs as unconfined by SELinux policies.

Where SELinux really shines is in server installs.

No, that's not true. Turning off SIP is usually about equivalent to allowing instant escalation to root privileges; sometimes even worse.

Can you please expand on this? It wasn't my understanding at all.

A decade ago SIP didn't exist at all. I thought disabling SIP just put us back to how things were in Yosemite. What changed?

Apple relied on it being turned on and started introducing permission checks where you'd have things like "do thing as root" and check for some flag being set that is protected by SIP. If you have it turned on, there are no issues, because the check blocks both root and non-root users! But if you turn it off, now the "do thing as root" is available to any account because the flag is editable.

Reply on news.ycombinator.com