Hacker News Reader

Comment by amelius | Hacker News Reader

Why does the Linux kernel/Posix make sandboxing so difficult?

It doesn't, really. Since there are umpteen ways to do things in GNU/Linux, developers often end up with less-than-ideal implementations for what they are trying to do using a single user account.

The single-user approach creates clashes between the network devs who want to build empires of containers they "own", and the stack-level bare metal purists who want the system to be as clean and secure as possible by isolating things where they should be isolated (to a single user instance for that purpose alone). This is not a new problem, nor a very well thought-out solution.

Containers are always a less-than-ideal implementation for people running Linux natively. The ideal way to sandbox in Linux is create a user account, download and test whatever code, see what breaks or infringes with its unique notion of "privileges", and delete the user when done.

But because you can't switch users on the same kernel when you're not running Linux natively, we have containers and all the messes they create. https://developers.slashdot.org/story/12/12/29/018234/linus-...

I have to admit that I'm quite confused about this comment. Are you saying simply running a command under its own uid is enough to provide the same isolation that containers do, and that the latter were only created because people are not running Linux natively?

I think he is saying there are a lot of 80% solutions like user based isolation that could have been made more secure, but instead people invented a new solution that has its own problems, and that the fractured landscape of solutions we see now is due to the freedom of open source.

Run application:

- under its own userid

- in its own namespace for mount, network, process-id, user-id, ipc, uts and control groups

and presto, you are running in a container.

The problem with using separated user accounts for isolation is that various applications assume that they either run as root (like various package managers) or need to start child processes with different user id. Sometimes this can be worked around, but overall the amount of efforts is very non-trivial.

So people thought that instead of fixing the apps it was easier to fix the kernel. But this resulted in a big complexity with namespaces, capabilities, cgroups etc.

POSIX doesn't speak to the problem at all (except for chroot). It's just not part of the standard.

With linux, it's because the tools are exposed piecewise. Creating a process namespace isn't the same thing as creating a virtual network device. You can use cgroups to stop a container all at once, sure, but you can also use them for other useful purposes and the code is the same. Some of these APIs are simple, some less so. But they're tools.

I know, I know, now you're asking "well, why didn't someone just put all the tools together in one box that would do it with one sane UI?!"

They did. It's called Docker.

Actually lxc is fairly mature and a more "Linux native" sandboxing feature. But docker certainly was/is more popular.

I know. And the linked project is an effort in that direction too. The point was that a knee jerk "sandboxing in linux is hard" argument is just silly on its face. Because Docker. Just go read any of the bajillion tutorials and testimonials about how to get your process to run in a sandbox.

Are any of the linux sandboxes considered secure yet?

Last I checked, they weren’t suitable for multi-tenant machines or for running untrusted code.

Fundamentally they share a kernel with the host. This has its own implications.

Beyond that it is very difficult to get things right, and there's definitely some information leakage (e.g. see the output of `mount` in a docker container... not a docker specific problem).

The main issue is, having a VM layer is always going to be "more secure" (because it's another security boundary) than not having a VM layer. Of course VM's have their own issues.

So if you have a need to run multi-tenant/untrusted workloads it's kind of a CYA situation where if you don't use this extra security boundary and there is a problem sometime down the line then you'll have to answer questions like "why didn't you do ..."

The reality is you can do a lot to lock things down with seccomp+selinux/apparmor.

Secure for what? I mean, sandboxing is more or less inherently a security technique. An app running in a Docker (or whatever) container is less exposed than one running on a big shared host. Is it as firm an isolation as you get from a proper virtual machine layer? No. But then, as we've discovered over the past year the hardware itself can have bugs that poke holes in that protection.

No free lunch. But yeah, Linux's suite of "container" tools are basically as secure as any other OS's "containers", no matter what spin you're reading elsewhere.

> I know, I know, now you're asking "well, why didn't someone just put all the tools together in one box that would do it with one sane UI?!"

> They did. It's called Docker.

Yes I see that. But I was hoping more for a toolbox like GNU fileutils, not a toolbox with a billion dollar valuation (which is what Docker is)

Take a look at rkt. It seems like it would be much closer as it does script based container execution.

https://coreos.com/rkt/

Rkt is pretty much dead as a standalone technology. It was influential in a number of ways though (for example, it forced the CRI to come into existence in Kubernetes).

If you're already using systemd, there's systemd-nspawn.

> I was hoping more for a toolbox like GNU fileutils, not a toolbox with a billion dollar valuation

What do you see as the primary differences between those two projects, or types of projects?

Well, there is unshare. The problem really is network devices, which require tinkering with C.

So because a tool is valuable you don't want to use it?

podman: https://github.com/containers/libpod

Interestingly, there was a post in HN recently about the next build of Windows introducing sandboxing. It really feels like there has been a steady stream of new security features coming out of Microsoft lately, which is great to see.

The upcoming Microsoft capability appears to be inspired by Bromium, https://www.pcworld.com/article/3169668/browsers/hp-hardens-..., which has long provided "seamless ephermal virtualization" on Win 7/10.

Of course, this “sandboxing” is the equivalent of running apps in a CoW virtual machine.

The sandbox can share memory with the host but says it is running a separate kernel https://techcommunity.microsoft.com/t5/Windows-Kernel-Intern...

I think just saying it's a VM is an over simplification

That’s what I said, though. It will copy-on-write memory pages and files.

Sandboxing in general is thought of as a policy thing that should be implemented in userspace. Kernel provides basic tools (namespaces, seccomp, whatever...), and userspace decides how to combine them to make a sandbox, whatever the developer wants that to be.

compared to?

I'm assuming they're referring to BSD here.

Admittedly jails and pledge(2) seem rather nice to work with compared to Linux's equivalents.

Linux was never designed for it, that's why. It's not a first class kernel feature like in other operating systems. Talk to Linus (who doesn't care about security, so don't lead with that as the killer feature)

Linus cares about security, after all he includes things like seccomp and selinux which are features only intended for security, and he supports the hardening project etc.

He has spoken out against the modern security industry and security theatre, both of which are distinct from security.

Reply on news.ycombinator.com