Hacker News Reader

Comment by mehrdadn | Hacker News Reader

Could someone comment on how secure such a container is, at least nominally? Should I be able to theoretically run untrusted code on such a container if the system is bug-free and I add proper error-checking to the code? Or are there things that you'd need to worry about the code being able to access? Any considerations regarding sudo permissions?

View on HN

Definitely not secure. The author did a great job explaining container runtimes in basic terms, but there's a lot of security features missing. Mainly: * Reducing the container's capabilities * Restricting access to resources through cgroups * Applying seccomp filters to prevent certain syscalls.

As another comment suggested, user namespaces are another hardening feature, but not all container runtimes enable it by default. Podman does, Docker doesn't. In fact user namespaces are so powerful that I believe they pretty much cover most of the hardening provided by the three features I listed above. If you're wondering why they're not enabled by default in Docker, take a look at this [1].

Exploiting the missing isolation mechanisms, the following bash commands will allow you to escape from the author's containers:

$ ls -al /sys/dev/block # find the root fs device (e.g. /dev/sda1) major and minor device numbers (e.g. {maj=8, min=1}, {maj=259, min=1})

$ mknod --mode 0600 /dev/host_fs_dev b $major $minor

$ mkdir /host_fs && mount /dev/host_fs_dev /host_fs

(warning: shameless plug to my posts follows:)

If you want more details, I wrote a post on this exact same problem in the context of three vulnerabilities I found in rkt (another container runtime) [2].

Beside the issues above, the author's runtime also exposes host file descriptors like /proc/self/exe that can be used to escape the container. This is a post I wrote on runC CVE-2019-5736 that explains this kind of issues.

[1] https://docs.docker.com/engine/security/userns-remap/#user-n... [2] https://unit42.paloaltonetworks.com/breaking-out-of-coresos-... [3] https://unit42.paloaltonetworks.com/breaking-docker-via-runc...

Wow thank you!!

Thank you for detailed answer and interesting links!

Could you please explain/point me to some information/source, why docker can't use -net=host namespace if userns is enabled, while on the other hand rootlesskit[1] which uses userns by default, dont have problem with using host netns (--net=host) ?

[1] https://github.com/rootless-containers/rootlesskit

The code from the article misses an important thing w.r.t. security - user namespaces, which can be used to map container UIDs to a subset of host UIDs and container root to non-root user. But it is a namespace that is more complicated to configure than others.

Ah thank you! That's what I was looking for.

Docker has a concept of layered images where only the top-layer is writable. The layer above the "scratch"[1] image usually contain all the files of the base images OS and that's what you set your root directory to. The writable layer disappears when the container is stopped. If you mount your DIY container into /tmp for example, the process running inside your container won't be able to access any OS functionality. You couldn't run a web server in such a container for instance. On the other hand, whatever your containerized process writes into the mounted part of your hard disk won't disappear when the container stops. Because of that, I wouldn't run untrusted code in it.

[1] https://hub.docker.com/_/scratch

this is the file system not docker itself. you can get the overlay behavior without any docker

Can you point me to some online resources? I'd like to learn more about this.

I'm not sure if this is what the parent is referring to but there are overlayfs and unionfs in Ubuntu for example.

start with: https://windsock.io/the-overlay-filesystem/

after that read more about overlay, overlay fs, for historical reasons aufs.

https://lwn.net/Articles/324291/

https://lwn.net/Articles/325369/

https://lwn.net/Articles/327738/

nice. thank you

I see, thanks. It's just a matter of preparing the directory before and cleaning it up afterward though, right? Not a security hole exactly?

Correct.

Big “if”. There has never, in thirty years, been a Linux that lacked a user-to-root privilege escalation path. Running untrusted code in containers is the same as it’s ever been: totally unsafe. VMs are safer, or, minimally, ptrace sandboxes intercepting all syscalls.

Are you saying because of bugs or are you saying it's by design? I explicitly said ignore OS bugs.

Yes you did, but that’s as useless as a discussion based on ignoring the laws of thermodynamics.

If you find it useless that doesn't mean I do. There's no obligation to contribute if you don't have anything to.

That may be hard. Some bugs are elevated to features and then become part of the design.

This is too vague to be useful.

One can write the same sentence about VMs. The most recent Xen escalation bug I can find in 5 minutes of Googling has a publication date of Jan 2020.

Emulating an entire machine, all of its diverse hardware, their bespoke protocols, and all the weirdness of the x86/amd64 ISA is fraught with peril. It is a large attack space. So too is the Linux kernel.

And frankly, inside the VM, half of us are running Linux anyways. I feel like for a lot of use cases, compromising the VM's guest OS (Linux) is enough to have a really bad day. Compromising the hypervisor? Bad, yes, but now it's AWS's problem.

There's more to containers, too, than just the security thing, and I think there are enough other advantages (I can more easily bin-pack services together; I can separate the FS and thus dependencies of unrelated components; I can more easily test them; etc.) that containers are worth it. Often and even on top of a VM. (My current work is with containers, and we run them on VMs.)

here is a secret: there is no such thing as a container. it’s an abstraction we made up and containers rely on kernel features. if you use those features correctly it’s as secure as it gets - chances are that if you’re going to roll your own you’ll miss some things.

That doesn't answer my question at all. I'm well aware containers aren't a "real thing". My entire question was about the "if" part that you didn't address. Is anything missed here? is my question.

in theory, it’s a secure as the kernel if your code does the right thing.

that being said, the attack surface is wider than say if you would run it in a VM or its own physical machine

Well obviously "if your code does the right thing" then it's going to be secure rather than insecure... by definition. That's again a pretty unhelpful tautology.

I'm asking about the code in the webpage, not code I'm writing personally. I'm saying let's assume it has error-handling added to it. That's it. I am not writing any code otherwise. Is that code doing "the right thing"? Or are there more things it needs to be doing?

The container process has still full access to /dev, /sys, all capabilities of root, and the ability to insmod.

I didn't notice that, thanks! Although I imagine they can't do much with /dev etc. unless they get sudo.

Unless you're using user namespaces (which this doesn't) then root inside a container is equal to root outside the container. You don't even need access to /dev, because the container process could just mknod(2) any device and access it with full permissions.

This is only possible in this example because the container has the full capability set (including CAP_MKNOD) and the devices cgroup hasn't been configured to restrict device access. Real container runtimes always restrict device creation by default, and usually don't allow CAP_MKNOD by default.

Thanks!

Reply on news.ycombinator.com