Bubblewrap: Unprivileged sandboxing tool for Linux | Hacker News Reader

Bubblewrap: Unprivileged sandboxing tool for Linux

github.com

189 pointsvarbhat49 comments

Note that a sandbox escape is often possible via TIOCSTI (CVE-2017-5226) [0] unless a special flag (--new-session) is used.

Bubblewrap is aware of this, yet their documentation gives no indication that this flag is necessary to produce a secure sandbox. In --help, the documentation of --new-session is simply "Create a new terminal session," which severely understates its importance.

It's frustrating to have such a useful tool be knowingly easy to misuse.

[0]: https://github.com/containers/bubblewrap/issues/142

FWIW, it's disabled in nsjail by default https://github.com/google/nsjail/blob/6483728e2490c1fc497a81... with relevant comment.

By the way, it looks like OpenBSD disabled TIOCSTI [1] [2].

[1]: https://undeadly.org/cgi?action=article;sid=20170701132619

[2]: https://marc.info/?l=openbsd-cvs&m=149870941319610

Still requires SUID. It makes no sense to have to increase privileges in order to be able to decrease privileges. Any process should be able to spawn a child process with strictly lesser privileges, including granular cpu, memory, network, and filesystem (including path and size restrictions) access. It's sad there isn't a simple, standard cross-platform API for doing this.

I would be fine having to rewrite my apps to take advantage of some fancy capabilities-based security paradigm, but give me realistic APIs to do so.

I'm currently experimenting with shipping apps as QEMU VMs, packaging QEMU and a minimal kernel with the app. It works surprisingly well, even on Windows with full x86 emulation. And with their newish WHPX API (basically kvm for Windows) the future might be really exciting.

EDIT: Someone on the Firejail thread says bubblewrap can be compiled non-SUID and that's the common usage these days. I need to look into this more.

I've also been looking into shipping apps as VM images with a minimal kernel. Do you know if WHPX requires the user to have admin rights? On the host side, Windows and Mac ports of crosvm [1] could be useful. crosvm seems to have all the necessary virtio device types, but a greater focus on security than QEMU.

[1]: https://google.github.io/crosvm/

WHPX does not require admin in my testing, but it does require Windows Pro and several virtualization features to be enabled. It's not too bad but still not ideal.

Currently I'm not banking on it though and focusing my testing assuming full emulation on older hardware.

If you're interested in collaborating on this feel free to email me or open a thread over on

https://forum.indiebits.io/

EDIT: Also, Firecracker seems to have more steam behind it than crosvm. I really wish it ran on Windows. That said, QEMU does have a microvm mode[0] that looks very interesting.

[0]: https://mergeboard.com/blog/2-qemu-microvm-docker/

To clarify, are you interested in shipping server or desktop applications as VMs?

Initially server apps. My overall goal is to lower the barrier of entry for self-hosting. The idea is you download an app like Nextcloud which would run inside an unprivileged QEMU VM, go through a quick OAuth flow to connect it to a tunneling service, and you have a secure, private, end-to-end encrypted private cloud without ever having to understand TLS, HTTPS, NAT, IP addresses, ports, CLI, etc.

hmmmmm > without ever having to understand TLS, HTTPS, NAT, IP addresses, ports, CLI, etc. HMMMMMMMM!

Honestly not sure what you mean.

Bubblewrap is indeed most often used without SUID nowadays; in particular, SUID is generally only used if unprivileged user namespaces are unavailable.

But unprivileged user namespaces are a risk on their own: Code that is supposed to be unprivileged suddenly runs as uid 0. Well, uid 0 "only" in a non-root namespace, but occasionally there have been vulnerabilities that missed the "only".

Indeed there have been CVEs this year where code in a user namespace was able to trigger them where a standard unprivileged process could not (e.g. CVE-2022-0185)

In response to the edit, I also read that and immediately came back here to check this thread.

If you want to properly isolate the network (veth device, separate IP(4/6) address) you need at least a suid helper to attach that to a bridge, it can't be done with userNS alone. Userspace proxying are a kludge and don't provide all the tools that the kernel network stack does.

That said, if you're using userNS then certainly the main executable should not be privileged, only some specialized helpers. And of course it should use caps instead of suid.

Bubblewrap is a surprisingly useful tool for general system administration tasks.

Even though the documentation claims that "[y]ou are unlikely to use it directly from the commandline, although that is possible," I use it as a helper tool in this mode very frequently.

This can be very useful for debugging since, for example, you can `bwrap --ro-bind / / --tmpfs ~ $SHELL` to get a "clean" shell in which you can isolate yourself from the effect of configuration dotfiles and can even `--ro-bind my-hosts /etc/hosts` to simulate certain system-level state (without requiring a full VM, heavier container, or root access.)

Of course, I've also written some simple shell scripts around `bwrap` to make this all a bit simpler (since this quickly reaches `qemu`-levels of argv proliferation.)

Nice! The other day I wanted to do something like this - ended up with docker to have a clean HOME, but bubblewrap might be easier and faster.

"How does it compare to firejail?" you were going to ask: https://github.com/containers/bubblewrap#related-project-com....

You're welcome.

Bubblewrap: Unprivileged sandboxing tool - https://news.ycombinator.com/item?id=12241971 - Aug 2016 (8 comments)

This looks like a very useful tool especially for isolating old unsupported daemons and not having to write really complex selinux rules especially in places where others are tempted to just set selinux permissive. And not just because I like bubblewrap IRL.

I see there are a few usage examples [1] out there. Are there any bigger collections of examples that people have run across?

[1] - https://wiki.archlinux.org/title/Bubblewrap

some time ago i made a comparison between different jailing tools: https://ctrlc.hu/~stef/jails.txt

I know HN people are all about simplicity, but just wanted to let you know that your txt file is unreadable on mobile.

Readable, accessible, responsive version: https://gist.github.com/remram44/fe30af580bfe1494931a392e957...

This seems to be the best of breed and it basically the same thing used by flatpak.

My understanding is that this is exactly what flatpak uses.

My reading of https://github.com/containers/bubblewrap#related-project-com... is that it is literally what flatpak uses:

> Firejail is similar to Flatpak before bubblewrap was split out

Here is something I'd like to have in fedora (or desktop linux in general): A sort of “workspace manager” using bubblewrap for isolating user-defined groups of applications and file hierarchy.

I imagine it as some kind of GUI with “boxes”, each defining a “workspace” with specified permissions (access to that and that folder, network, etc), such that the user could easily drag and drop apps into new workspaces.

This way, I could have a “banking” workspace just containing, say, firefox, and a “work” workspace with, say, firefox and thunderbird and whatever, etc. The “workspaces” would by default be as unprivileged as possible, with the possibility to give access to folders, or even maybe to use unionfs or similar to combine the views of different workspaces.

For ease of use, I could assign a color or icon to each workspace, and the manager would automatically generate desktop files so that I would easily be able to spot “banking firefox” from “work firefox”.

I feel like with bubblewrap, this is only a short python GUI away, and would be super helpful to strengthen security on linux, even more so for non-technically inclined, or lazy people.

[Edit:] If someone wants to discuss things further and even maybe try and write a “MVP”, I'd be happy to try.

This is almost exactly the concept behind qubes-os except it uses the xen hypervisor instead of bubblewrap and vms for boxes.

https://www.qubes-os.org/

Right. I'm being a bit less ambitious here: I've recommended fedora to plenty of (non-tech) people around me, and fedora presumably already has all the necessary tools to add such a simple “workspace manager” without much disruption. It would provide an easy intuitive step-up in terms of security, starting from something people already know.

First problem is that proper (incl. X11...) isolation costs you 3D (so not great for Firefox). Otherwise there is "toolbox" by Fedora, which you could surely combine with Xephyr for a wine-like, more isolated experience. I do that with a shell-only workflow (though on my HiDPI-machines I still have to patch Xephyr to scale the output (which is pretty easy using its GL-backend tbh, but I need to refine my amateur C dabblings to release that).

EDIT: I'm happy to discuss how to use Wayland with proper isolation and 3D! (and some frame-decoration)

I didn't know about "toolbox": it seems close to what I want. About 3D, does calling a program through `bwrap` disable opengl or something?

well, depending on what you do, you usually need access to the dri-device, the X11-driver and X11 (socket). This is relatively insecure I suppose because it exposes a giant attack-surface. Everything trying to mitigate this (short of running VNC) unfortunately still has some, but I would expect noone targets these broad-spectrum. An interesting thing to do would be to run with VGL and a quadro/...-GPU which allows partitioning into different rendering devices.

Haven't tried reading about this with wayland.

> Xephyr

It's easier to use Wayland. Additionally, Toolbox does not sandbox the application.

Then how will wayland tell me, which container a window comes from? (besides that, most software I use in containers is very much preeee-wayland). Thanks for the correction on toolbox, I know remember again, why I have my own bwrap-wrapper.

>Also, @cgwalters thinks trying to whitelist file paths is a bad idea given the myriad ways users have to manipulate paths, and the myriad ways in which system administrators may configure a system.

So you either get the entire filesystem or no file access? Isn't this a huge dealbreaker for almost everything?

I think this is just a comment on Firejail's implementation, as I think Firejail runs as root and thus must enforce its own permission checks.

It seems like bubblewrap uses a mount namespace created by the current user which would allow controlling access without any special checks.

I'm running firejail now and It doesn't look like it runs as root.

Correct, it is an SUID executable[0], as is bubblewrap.

[0]: https://en.m.wikipedia.org/wiki/Setuid

This is poorly documented, but if the kernel supports unprivileged user namespaces, bubblewrap works without suid.

Good to know, thanks.

Does/can firejail, or is that a difference?

bubblewrap is only setuid if user namespaces are unavailable.

Better to use selinux than path whitelisting.

Does anyone know if it's possible to configure bubblewrap to attach to an existing namespace (network namespace in my case)? I'm selfhosting stuff and wanted a diy container runtime but i can't seem to isolate stuff from the system and still retain some linking between some parts (ie doing stuff like what a docker-compose file specifies).

I enter bwrap namespaces with classic nsenter (bwrap has no option), doesn't this work for you?

Thanks, that should do it indeed, i'll try that in my scripts! Tbh i didn't search long, i was just a bit disappointed that bwrap didn't have this capability. (because now i have the choice of doing nsenter outside or inside of bwrap.. on the top of my head that shouldn't make a difference if the kinds of namespaces touched is disjoint, but i'm sure there are some subtle bugs waiting to be introduced)

Who is https://github.com/containers? Well, I see the member list and recognize some names as podman developers. But is there any presentation, policy, statutes...?

AFAIK it's Red Hat.

I really wish bubblewrap had a way to mount filesystems as copy-on-write instead of read-only.

Having ways to impose limits on memory would be nice too.

Reply on news.ycombinator.com