Hacker News Reader

Sounds pretty cool! TBH, I don't understand how the "modern" app security models are not deemed to be fundamentally insecure in 2021 where any random app on any random system can simply leak your whole photo library to their servers, or delete all your files. Why do banking apps need full access to my filesystem? Why don't the reviewers of the app stores prohibit such practices of excessive permissions?

Yes, there's always a question of usability, but if you're an advanced user, it's still scary that you're not afforded any control to prevent these incidents unless you go ahead and redesign the whole way all these apps are working all by yourself.

It seems terribly inefficient if every engineer has to do it on their own, and in their own incompatible way. Obviously most people simply give up after a while, since maintaining such a setup might itself be a whole full-time job.

"... I even had a separate Firefox jail that was only allowed to get out via a Tor socks proxy to avoid leaks ..."

I have looked into doing this many times and it's neither simple nor straightforward.

Specifically: jailing a GUI app that you can interact with on your desktop.

I can't remember what the most promising recipe I saw for this was but it wasn't quite promising enough to compel me to built it up ... and this discussion is always (rightly) hijacked with "just use Qubes" ...

Firejail with spawning nested Xorg works fine for me, including text-only copypaste between "host" and "guest" and automatic file synchronization through bind-like mounts. For some firejails I also use Linux network namespaces to control traffic going through taps. My introduction to this approach was the alternative Gentoo handbook by Sakaki[1], but the principles would apply on any distro.

There's also a very interesting read on Qubes-like experience on NixOs with Wayland and XWayland[2,3].

[1]: https://wiki.gentoo.org/wiki/User:Sakaki/Sakaki%27s_EFI_Inst...

[2]: https://roscidus.com/blog/blog/2021/03/07/qubes-lite-with-kv...

[3]: https://roscidus.com/blog/blog/2021/10/30/xwayland/

What's nested Xorg? Do you mean you run each app in a separate VNC or RDP server?

An example would be Xephyr[1]. Archwiki[2] has a decent summary.

In my case, I have the standard xorg session started by my login manager. Then I start Xephyr with a separate DISPLAY, that shows up as just a window in the parent environment. It does look kinda like RDP or VNC.

[1]: https://linux.die.net/man/1/xephyr [2]: https://wiki.archlinux.org/title/Xephyr

That's very cool, thanks!

Bastille template to bootstrap Firefox:

https://github.com/ddowse/jailfox

Is there a reason why FreeBSD doesn't default to running all applications in jails? Seems like this would be a pretty huge advantage compared to the typical Unix system's almost complete lack of sandboxing.

what should run in a jail? you need the base system to run jails. and that is pretty much the only thing that is installed.

Because if I don't run in a jail, every software I run has full read/write access to all my files. Almost none of the software I run needs even read access to anything except its own files.

Jail is not a Sandbox but OS-Virtualization, those are two different things, Capsicum is the sandbox for FreeBSD...like secomp is for Linux.

I beg to differ, given that HP-UX Virtual Vaults were quite alright and precede Jails.

(noticed your reply to my other comment too, please consider this a reply to both)

I wasn't aware of HP-UX Virtual Vaults, thanks. However, I'd say FreeBSD Jails still had an advantage, due to being free and running on various hardware platforms, and more importantly on commodity hardware.

Eventually they were replaced by HP-UX Containers (SRP) a couple of years later.

Unfortunately it is hard to still find documentation, given the troubles HP-UX has gone through at HP (which kind of plays into your remark regarding FreeBSD).

Still there you go, https://support.hpe.com/hpesc/public/docDisplay?docLocale=en...

In the same way LPARs preceded Virtual Vaults on System z? :)

I guess so, but I was constraining myself to UNIXes. :)

Yes, but in a sense that's the essence of why the technology got left behind. Jails were a mechanism for expert admins to play with container ideas.

What the market actually wanted was Docker. And what Docker needed was Linux containers (complicated, flexible, piecewise technology) and not jails, which were higher level abstractions (but yet not high enough) with jargon and framework assumptions that didn't match Docker's needs 1:1.

And slowly the only thing left from Docker will be the image format.

And the Registry API too, I expect, but yeah, the untimely demise of Docker as a company feels inevitable at this point.

The "market" did not want Docker. Docker as a product failed.

There are many reasons why FreeBSD jails count not get out for FreeBSD land, one, very important thing is the Linux community's NIH attitude.

I don't think you can explain that as NIH. From what I have seen, Linux namespaces are a much more powerful primitive than BSD jails.

Well, looking at the following things sort of gives it away:

- brtfs vs zfs

- cgroups vs jails

- SystemTap vs dtrace

- Systemd vs smf

I get it, many of these were due to licensing issues. So they said[1]. Anyways, there are still some things to implement for linux. pf is my favourite (software) firewall. It would be great to see it ported to Linux.

1. https://opensource.stackexchange.com/questions/2094/are-cddl...

In my experience, both Linux developers and BSD developers don't seem to care too much about porting things to the other's operating system. If you want to do things the Linux way you can use Linux, and if you want to do things the BSD way you can use BSD. That's seen as easier than trying to glue two incompatible things together.

BSD developers can't port things from Linux in a straightforward manner due to license issues. But that doesn't apply the other way around.

Won't, not can't. *BSDs shipped GPL components for decades before they decided to go for purity. It's a policy decision, not a incapability or mandate.

I don't see why. BSD and GPL are equally compatible, it doesn't matter which way you go. I can see why they wouldn't want a GPL component to be mandatory but it can be made an optional component for Linux compatibility which seems to be the way BSD would want it anyway.

namespaces in what sense? network namespaces? because freebsd has VNET for a while now. which seems to do the same thing.

And all the other ones: https://man7.org/linux/man-pages/man7/namespaces.7.html

BSD jails are similar but not quite the same thing.

I don't get it. How are people using this flexibility to get things done in practice, and what uses aren't allowed by the jail model?

You can just compare the APIs, namespaces are like the individual components of a jail. You can use them to build something like a jail, or something different that has a different security model. This was discussed a lot in an old HN thread: https://news.ycombinator.com/item?id=13982620

This doesn't really answer the question. Yes, the Linux API seems more flexible, but when you think about it, it really isn't, because all the models that actually make any sense can be implemented using simpler interface, which is what jails provide.

One real difference is that you need to be root to create a jail. It'll get fixed eventually - FreeBSD already has unprivileged chroot, jail isn't that much different.

>Yes, the Linux API seems more flexible, but when you think about it, it really isn't, because all the models that actually make any sense can be implemented using simpler interface, which is what jails provide.

Not really, the example of Docker would probably be the most straightforward there. I don't think it's possible to fully port Docker to jails or at least I've never seen a successful port, some of the network topology features seem to just not be possible or straightforward. But I could be wrong, I have not looked into the technical details of this in years, somebody told me it might have been working a while ago but I never heard anything else about it since.

Needing to be root is a major deficiency though and I can't take jails seriously with that, one of the main focuses on Linux containers in the past several years has been to make unprivileged namespaces a good option.

Docker is literally just a jail. You can do whatever network topology you want using vnet.

And yes, having to use root is a major issue. Looks fixable though.

Sorry, then I must be remembering some other issue. The effort to port docker to BSD seems to have disappeared.

>And yes, having to use root is a major issue. Looks fixable though.

AFAIK it took a long time to get this to work on Linux, there are a lot of security issues that it can cause.

Docker is practically dead, no wonder if disappeared :-)

Many things are hellishly complicated in Linux, due to politics and technical difficulties. Case in point: when I’ve started to work on NFSv4 ACLs, support in Linux was “worked on”, there was a prototype. It was 12 years ago. In FreeBSD, full supper for NFSv4 ACLs, from file systems to userspace tools, shipped decade ago. In Linux it’s still not there.

Docker wasn't dead 5 or 6 years ago when I heard about this. In my experience some things are easier to implement in Linux and some are easier to implement in BSD. I don't particularly care for BSD's internal politics either, such as the licensing issues mentioned elsewhere here.

It might be because it would require reimplementing all the system-specific Docker parts from scratch. Not sure though.

BSD doesn't really have any licensing issues, thanks to BSD license, but politics is directly related to project size. In FreeBSD it's pretty much unnoticeable, but in Linux it can be a huge deal.

That has not been my experience. The issues are with licenses other than BSD but that's the same in Linux; Linux can also use BSD code, a lot of Linux code is actually dual licensed as BSD already.

It’s not - Linux does have problems with licenses which are incompatible with GPL, such as MPL/CDDL. BSD doesn’t, because you can’t have license incompatibility without throwing GPL in the mix - it’s the only Open Source license that can be incompatible with others.

FreeBSD does avoid pulling restrictively licensed (closed source or GPL) code into the base system itself, but a Docker port would be third party (ports/packages), not the base system.

Not really, with Linux it's the same as it would be in BSD if they wanted to avoid conflicts with GPL. You put that code in an optional module and have the user compile it. I am unsure as to why BSD people seem to think that using BSD means you can avoid problems with the GPL, if you use any GPL code for any reason (and there's a lot of it) then you have to pay attention to these things. If you insist on only running BSD and CDDL code then you can avoid it, but that's going back to putting politics over software again, the kind of thing that you were saying you were trying to avoid.

Nope - FreeBSD is almost free of GPL, and will probably become entirely GPL-free in 2022: https://wiki.freebsd.org/GPLinBase.

Also, it’s not politics - it’s mostly just that the old GNU cruft is being replaced, and newer, better solutions prefer more liberal licensing, see GCC vs LLVM.

> It was 12 years ago. In FreeBSD, full supper for NFSv4 ACLs

You are right, but ironically we have to thank Microsoft for that ;)

Why, apart from inventing them (in Windows NT) in the first place?

Well the posix conglomerate had such big problems implementing a ACL (mostly because of blabla)... Then Microsoft did one for NT which was then took over to NFS, later the posix peoples decided on a ACL but no one wanted it anymore. However FreeBSD supports both, but the NFS/Microsoft one is the standard.

> Needing to be root is a major deficiency though

Note: Linux also needs root for its namespaces. Or at least CAP_SYS_SYSADMIN, which grants enough that it's pretty much as good as root. See setns(2) and clone(2) for details. This is one of the complaints the plan 9 people have always had with Linux namespaces.

Not anymore, unprivileged user namespaces make it so you don't have to do that. That's how podman's "rootless containers" are able to work.

Yes, I am aware that it's got more moving parts. What are you using this flexibility for?

I'm using them for several things but the most straightforward one is probably that namespacing can be gradually added to services, you most likely see benefits from this already if you use systemd. That's one way that namespaces can be used in a different way from the docker model.

What are you adding gradually, specifically? Like, a concrete example that names a namespace you may want to use. I'm trying to figure out what problems a half sandbox solves, and a vague "I just want to enable some capabilities" doesn't help here.

A lot of the various security options in systemd: https://www.freedesktop.org/software/systemd/man/systemd.exe...

The sandboxing and mount-related ones are implemented with namespaces, and the idea with them is to not make any of them mandatory so they can be slowly added to system services. That way you can get some of the benefits without needing to build a full rootfs/container for the service. I am not sure how any of those would be done with jails because jails require you to create a chroot and network interface, whereas in Linux the mount and network namespaces are just optional namespaces and you can still use the other namespaces without using them.

> jails require you to create a chroot and network interface,

They don't: you may chroot to /, share the host's network interface, or disable networking.

It's literally a single command:

  trasz@v3:~ % doas jail / foo 127.0.0.1 /bin/sh
  # ps aux
  USER   PID %CPU %MEM   VSZ  RSS TT  STAT STARTED    TIME COMMAND
  root 37975  0,0  0,0 13500 3056  3  SJ   09:11   0:00,01 /bin/sh
  root 37976  0,0  0,0 13624 2776  3  R+J  09:11   0:00,00 ps aux

also, with epairs you can do some really flexible networking stuff on freebsd between jails/jails and the host system and even jails and ipsec tunnels.

Side note: I suppose you could chroot to /.

> How are people using this flexibility to get things done in practice

Um... to loop back to the upthread point: Docker. People are using Docker, and docker is using this stuff.

And how is it mixing and matching these APIs? Given that there's an OCI-compatible runner for jails (runj, compatible with runc -- which is what docker uses to start containers), it seems to me that Docker isn't in actually using the flexibility afforded by the APIs here, but is just using a relatively fixed set of options.

If I'm wrong: what is it using, and what problems is this flexibility solving?

I haven't tested runj but just from looking at it, it seems it is not fully compatible with everything that runc does because the OCI itself specifies a lot of Linux-specific functionality.

Can you provide some examples?

runC is literally the abstraction layer docker wrote internally on top of linux containers! It exists as a separate layer now because they spun it out precisely to freeze the API and enable other efforts like runj.

And runj, IIRC (though I'm not an expert in the space) wasn't a trivial 1:1 thing and required changes to the underlying jails layer to enable it.

> The "market" did not want Docker. Docker as a product failed.

Docker as a paid product failed. Docker as a company is failing. "Docker" in the sense I meant (of the software people use to launch containers), is pervasive and dominant. It won. And jails, in comparison, "lost", because jails didn't really do what Docker wanted. And what the market wanted was Docker.