FreeBSD – a lesson in poor defaults | Hacker News Reader

FreeBSD – a lesson in poor defaults

vez.mrsk.me

195 pointsbeefhash69 comments

> I think swap should always be encrypted. FreeBSD developers disagree. It's surprising just how much private data in memory gets written to disk. Someone I know has run hexdump on his swap partition and found PGP private keys in plain view. Your SSH key might be password-protected on disk, but it could end up in swap sooner or later... with no password needed.

GnuPG and a few other security-conscious programs try to avoid this by using mlock() [1]. Unfortunately, by default the ZFS ARC can grow beyond the limit for wired pages, which will make those mlock() calls start failing once the system has been up long enough [2].

[1] http://pubs.opengroup.org/onlinepubs/7908799/xsh/mlock.html

[2] https://ximalas.info/2012/04/10/not-enough-room-for-wired-pa...

Moreover, you have stuff like password managers going though the clipboard.

Relying on all tools correctly identifying their sensitive datasets and protecting them though mlock() is doomed to fail. If anything, a proper security concept should whitelist insensitive datasets, instead of blacklisting sensitive datasets.

I don't get why any modern OS would use unencrypted SWAP. The default should be either encrypted SWAP, or no SWAP at all. The downsides are negligible compared to the upsides.

> encrypted SWAP

When I was setting up dm-crypt for my Linux dev box at work I opted for using an encrypted swap as it really made sense: why keep on-disk stuff encrypted if swapped memory is in plain sight.

It turned out to be a somewhat bad decision, because when the system did run out of memory (thank you, Firefox and Skype) and began swapping, the whole box just freezes as the kernel was too busy encrypting more and more stuff, not having time to even run the OOM killer.

Maybe the systems use unencrypted swap to avoid triggering such bugs in default setups.

I don't doubt your story, but modern CPUs can perform AES-GCM at gigabytes per second per core (5GB/sec on my laptop). Disk encryption is a different block cipher mode, but it shouldn't be much slower. So if implemented correctly, encrypted swap should not slow you down vs unencrypted swap. If in your setup encrypted swap is much slower than unencrypted swap, it's a bug in the kernel crypto and it should be fixed.

Actually, Samsung 960 Pros are fast enough that encryption becomes a bottleneck, at least on my Xeon E5-2620v4.

You can test it yourself with `cryptsetup benchmark`:

    # cryptsetup benchmark --cipher aes-xts
    # Tests are approximate using memory only (no storage IO).
    #     Algorithm | Key |  Encryption |  Decryption
            aes-xts   256b  1874.2 MiB/s  2042.6 MiB/s

I was surprised and it made me doubt whether it's using AES-NI at all, but then again it's not an issue in practice.

    cryptsetup benchmark --cipher aes-xts
    # Tests are approximate using memory only (no storage IO).
    #  Algorithm | Key |  Encryption |  Decryption
         aes-xts   256b  2266.2 MiB/s  2273.8 MiB/s

perf says

    21.86%  cryptsetup  [aesni_intel]           [k] _aesni_dec4                       
    19.83%  cryptsetup  [aesni_intel]           [k] _aesni_enc4                       
    17.80%  cryptsetup  [aesni_intel]           [k] aesni_xts_crypt8                  
    13.73%  cryptsetup  [kernel]                [k] copy_user_generic_unrolled        
     2.20%  cryptsetup  [kernel]                [k] get_page_from_freelist            
     1.86%  cryptsetup  [glue_helper]           [k] glue_xts_crypt_128bit             
     1.53%  cryptsetup  [kernel]                [k] put_page                          
     1.36%  cryptsetup  [kernel]                [k] blkcipher_walk_done

so it's using the code from:

https://github.com/torvalds/linux/blob/master/arch/x86/crypt...

I don't know if using AVX would speed it up. openssl is faster (openssl speed -evp aes-256-gcm and aes-256-xts).

Even if a modern CPU can to do, the question is whether on a stock distro kernel they will do that. I could see the default kernels not being compiled with the extensions that may allow that speed for compatibility purposes. Then again, I remember when distros often shipped different i386, i586 and i686 kernels, and when Red Hat switched away from i386 as the default kernel (which might be a lot more recent than you think, if you weren't thinking of all the cheap i386 devices it was still working on in the early 2000's). Then, even if the stock kernel does support it through a kernel module, the question is whether that module was loaded. Finally, from looking at an Intel/AMD AES intructions compatibility list[1], not all processors as of even a couple years ago supported AES instructions (notably, it appears Intel used it to differentiate the i3 line from the more powerful ones)..

1: https://en.wikipedia.org/wiki/AES_instruction_set#Intel_and_...

I'm uncertain if it's safe to use AESENC and friends in kernel mode in Freebsd. A number of kernels can't safely use e.g. FPU registers in-kernel and aesni operates exclusively on SSE state.

It's safe. The FreeBSD kernel is capable of preserving user FPU state when kernel threads (e.g., aesni(4)) need to use FPU registers.

Most new drives can do encryption in hardware with zero performance penalty using either the ATA password or Opal for NVMe. For those doing software/CPU encryption on the swap, or any other partition for the matter is simply wasteful and often slower.

Isn't on-drive encryption considered untrustworthy?

now i’m curious if anyone has tried to write their own encryption routines to any common hard drive firmware stacks

Depends on your threat model. If you want to protect your data when your laptop is stolen, on-drive encryption is probably more than sufficient.

In my experience, linux locking up for minutes at a time while in a swapping situation happens if the disk is encrypted or not, and is a function of a crappy swapping algorythm.

In my experience, swap usually does not actually work out helpfully. The system slows down so much it's unusable, and you wish the OOM killer killed something so you can recover some processes/state, but it takes way too long to do so, many minutes while the system is effectively frozen and you are helpless. (Some people think the OOM killer kicks in too soon. They're crazy ;)

I also ran into this when setting memory limits on docker containers. It basically made them freeze for entire minutes when they hit the limit instead of killing them. So instead I wrote my own script to check and signal them to gracefully exit before memory runs out (and after a few more seconds SIGKILL them).

Swap wins when you have different large processes open that you aren't actually using simultaneously. On my laptop with 2GB of RAM, it's why I don't have to close my browser before opening something else.

Sure, they'll have an encrypted SWAP, and an unencrypted swap, and developers will choose which to use. Leverage Unix's case-sensitivity.

To tilt defaults to a safer side is usually (though, not always) good, if it's meant to be out-of-the-box system. Definitely, not the case here as bare-bones FreeBSD is essentially a platform for building a user's own solution for own needs. Not making too much assumptions about what they are is actually very rational for FBSD devs. I know my goal better, I know which choices/compromises I want to make.

FreeBSD is a tune-it-yourself OS. You think your swap should be encrypted? Fine, append ".eli" to the name of your swap GEOM (partition, slice, whatever) in fstab, and you're done. Some may not require this at all, because FreeBSD may be used in a lot of different situations. Your needs - your choice.

The submission in question is about defaults, not about choice.

https://news.ycombinator.com/item?id=16012670

Been here more than once, for past comments see

469 days ago https://news.ycombinator.com/item?id=12484248

647 days ago https://news.ycombinator.com/item?id=11318508

For those who already have read it, document has been addended multiple times (see bottom of doc)

> For those who already have read it, document has been addended multiple times (see bottom of doc)

Thank you, I was just wondering how up to date this is.

This was "Last updated 12/08/2017." That update does not include removing portions that are no longer true — instead they have published an update section at the very bottom.

Re: sendmail, check out the age of most of those CVEs (2006 or earlier). But also, see https://lists.freebsd.org/pipermail/freebsd-arch/2017-Decemb... .

A lot of these has been already addressed by a security menu in the FreeBSD installer, since settings and/or mitigations for mentioned issues have been possible in FreeBSD but have not been enabled by default due to POLA (project's rule of Priciple Of Least Astonishment) in order to prevent users expecting some behaviour to be surprised by changes. Those who knew about these could always enable those, and those who didn't now have option to do it via the default installer.

The analysis is not in any way serious, just some feelings the author has about certain things that may not even matter to anyone nor improve security in any meaningful way, if not worsen it by providing a false sense of security.

I wonder how this compares to other unix systems, such as OpenBSD or various Linux Distros.

HardenedBSD (https://hardenedbsd.org) also. Which is a recent fork of FreeBSD that has a focus on security.

Unfortunately, HardenedBSD is nothing more but a PR campaign from a one person who feels rejected due to poor code quality and bad reviews from FreeBSD developers who expect a bit more than just throwing random patches around and saying 'here, I fixed all these security problems with three lines of code'. I would recommend taking HardenedBSD and its announcements with a huge portion of salt.

The funny part is that FreeBSD has had years to even implement anything as simple as ASLR but it didn't and instead starting to reject other peoples code. What HardenedBSD did is basically following the PaX model which has been documented for a long time[1]. It seems like FreeBSD is just lacking any developers willing to deal with these security features or let alone upstream it because its usually a pain in the ass to upstream stuff like this just look how Linux is having a hard time implementing even basic stuff from grsecurity. Exploit mitigations are not for fixing those security problems it is to make it harder to exploit certain vulnerabilities.

1: https://pax.grsecurity.net/docs/index.html

The funny part is that even amongst top security researchers there doesn't seem to be an agreement over the fact if ASLR is worth anything, as it was proved to be breakable in no time. And it seems like grsecurity code suffers the same problems as the code from HardenedBSD, that is the quality matching the respective project's standards.

Who are these top security researchers? ASLR on its own is useless but security is about layers there isn't one mitigation to rule em all. But ASLR combined with other features work very well, the funny part about FreeBSD that you can just apply 90s blog post about smashing the stack and you can exploit with it.

grsecurity code is clean and uses some neat tricks with the C language, most features have been there for at least a decade and recently things like RAP have come along but its all pretty clean. The problem with upstreaming is that you will have people that think the code is shit or doesn't work properly or all kinds of other stuff and that takes a lot of time that could be spent somewhere else. About the quality matching the respective project's standards is bullshit because many developers aren't security engineers or have never dealt with exploit mitigation's and instantly think/say the code is shit, biggest reason why Linux will never get actual important features from grsecurity because it takes lots of time and developers that actually understand what they are doing. FreeBSD just doesn't have those developers and neither has Linux, now you know why out of tree patches just work for this kind of stuff. Microsoft seems to take the upper hand in exploit mitigation's at the moment.

Long-time OpenBSD user here. OpenBSD has sane, secure defaults with nothing enabled in a fresh install. This is a desired default, as everyone uses OSes for different reasons. OpenBSD makes a phenomenal firewall, a pretty good Web server, and a super nice, secure file server.

OpenBSD shines because of the ongoing code audit, the minimalism, and the ability to easily make it into whatever you want. I don't care for any other OS for serious business. I will almost always use OpenBSD for servers, with RedHat or CentOS being the only other choice because of SELinux.

I would also venture to say that OpenBSD has the absolute best documentation of any OS out there, even better than FreeBSD.

Can you elaborate more on the CentOS/RedHat choice? I'm curious about how other Linux distros are handling in this space.

Sure. Some clients prefer us to use Linux, since their teams large or small are comfortable with it. OpenBSD is still a rather esoteric OS for most. I've been using both since 1998, so I can go either way. RedHat/CentOS offer support for 10 years is high up on the totem pole for me and plenty of others. OpenBSD releases every 6 months like most Linux distros, so some mission-critical users are uncomfortable with this pace, and I cannot say I blame them.

SELinux is great for areas that are into DAC/MAC/RBAC, plus the aforementioned lengthy support. I see more and more customers wanting Red Hat/CentOS than any other Linux distro and for good reason. It just works. Things like CPanel only run on these distros. Red Hat/CentOS are also very predictable and they offer superb documentation. More and more people are uncomfortable with things like Debian because there is no throat to choke if they need support or if something goes pear shaped.

I have worked on Linux for 20 years now and I have never, ever seen Debian or Slackware or even Ubuntu in a TRULY mission-critical role. People may disagree, but there are reasons why Red Hat/CentOS are chosen. They are typically very stable, use tested software, and are well documented for the use cases they excel in. Red Hat runs some truly mission critical stuff like power grids, and more and more SCADA systems, now that they are being upgraded due to threats.

>OpenBSD makes a phenomenal firewall

How about a router? I've been considering getting some single-board computer (probably an APU2[1]) and turning it into a router, possibly using OpenBSD + pf. Or maybe just install Pfsense. Or maybe use Linux and...iptables? (Not sure how that stack would look exactly)

1. https://www.pcengines.ch/apu2.htm

Reminds me of the old "Fixing Solaris" file from the '90s (and the Fixing SunOS that was its forebear.)

Is it archived anywhere?

> Being alerted to vulnerabilities in your installed packages is nice, but there's simply no need to be doing this operation as root. The dangerous combination of laziness and poor software design is quite prevalent here.

Just a thought - if an unprivileged user could download the vulnerability list, couldn't a malicious but unprivileged local user just "download" an empty vulnerability list? At least it would take a user account created specifically for this purpose. (Which I admit would be a good idea!)

A compromised unprivileged user could download an empty vulnerability list. A compromised root user could download an empty vulnerability list or do anything else root can do.

Previous discussion: https://news.ycombinator.com/item?id=11318508

How does pfsense differ from FreeBSD in light of these comments?

As someone looking to move a cluster from ancient Debian, I was looking at either CentOS or FreeBSD but this looks not so nice..

Honestly, most of the concerns are rather superficial.

You might want to try HardenedBSD instead of regular FreeBSD though, to get all the exploit mitigation stuff.

Don't do that. HardenedBSD is rather a one man project, who's patches have been reviewed and rejected by FreeBSD developers due to bad quality, poor design and lack of cooperation in bringing them to FreeBSD expected standards. It seems like a PR campaign from people who can write some C, but don't have much credibility in writing secure operating systems or production ready code. Beside the quality of the proposed solutions, there was nothing in the FreeBSD project stopping the code from being accepted. Wether the bad ASLR implementation or no implementation is worse, it's up to you to decide, but I'll vote for quality first.

I could be wrong but I was under the impression certain sec folks objected to a lot of his patches because they just are not sold on if ASLR really adds any benefit. Or any benefit vs the complexity it adds. Correct me if I’m wrong. I know Theo has bought into it. I just recall in FreeBSD it wasn’t accepted if it really fixes anything.

FYI since a couple weeks ago, all of your comments except this one appear dead. You must have made someone mad.

Yet FreeBSD hasn't done a single thing to improve the current state of exploit mitigations. It doesn't matter how many people are writing code, grsecurity is mainly Spender & pipacs aka PaXTeam yet 14 Linux developers couldn't spot the vulnerabilities they've created by copy pasting and editing code from grsecrutiy. You can't really blame others for trying while FreeBSD hasn't tried a single thing yet.

No one, at least not me, is blaming him for trying. But he is to be blamed for sending a poorly written and badly designed patch, unwilling to work on it until it meets the quality of the FreeBSD project and then 'forking' FreeBSD, applying these widely criticised patches and making a social campaign of "they don't want to fix security" instead of "they don't want to introduce bad code" what would be much closer to the truth.

Debians do upgrade well from one version to another. I've done a fair amount of upgrades of old servers to current stable.

As an infosec guy having read some of the past discussions here — mostly about how infosec people are overzealous and don’t understand how disruptive these types of changes can be to existing installed bases — we do get it. The problem ends up being that improvements already take ages to work their way downstream, and not making them for compatibility reasons is a self-defeating prophecy. Those compatibility concerns will still exist next week, next month, and next year without actually working to address and reduce the problem. Often times they’re even worse as the installed base continues to grow and/or calcify around the existing setup.

The net result is that the compatibility argument, usually unintentionally, results in the status quo being kept for years. And when it already takes so long for these changes to make it out to users due to slow upgrade cycles, slow repackaging by downstream maintainers, slow uptake of new defaults, etc., with this same argument happening at each and every level… it often results in little to no progress over the span of years to common threats and pitfalls that are making people vulnerable today. That’s also why some of us try so hard to make sure developers get things right early on in their designs, since fixing it after the fact can be such an ordeal.

Of course, we understand the plight of maintainers not wanting to disrupt their users. It’s a difficult situation from both sides of the debate for sure. And also of course developing new features is more likely to make both users and developers happier than fixing security bugs is, at least in the short term. Just wanted to put the opposite perspective here as well since it felt underrepresented in previous discussions.

One thing to note, this isn’t necessarily indicative of my agreement with all of his analysis or belief that many of these are severe and need urgent fixing. This is more targeted at the meta-discussion around overzealous security people.

While you're here, have you donated[0][1] yet? :) You may or may not be aware, but FreeBSD runs your movies on Netflix, your games on PlayStation and Nitendo Switch, your files on FreeNAS and ZFS, your friends on WhatsApp and OpenBSD runs everything else on OpenSSH. ;)

So, you may or may not know that, but you need FreeBSD and OpenBSD and they also need you! Every cent counts and so does every contributor, that helps the foundations keep their non-profit status. Also, you CAN be the change, if you specify what you'd like your donation to be used for (like more secure defaults for the OS).

[0] https://www.freebsdfoundation.org/donate/

[1] https://www.openbsd.org/donations.html

Seems to me that Netflix, Sony, et al should be paying FreeBSD, not the users. Why would I subsidize FreeBSD so that mega corps can leech off the hard work of volunteers, and by extension leech off my donation?

They do pay, both in code and donations.

Obviously these hugely profitable companies don't pay enough. Otherwise I should not need to donate.

Obviously Windows should be free, with all the huge companies that use that. The logic makes no sense.

It makes perfect sense if you don't personally own or use Windows. I don't pay Microsoft again to play TitanFall just because their servers are hosted on Azure.

I understand we all probably use tools everyday that can be attributed to these projects and I don't mean to imply that you shouldn't give them money. Just that the Netflix/WhatsApp comments seem a little disingenuous. That shouldn't be in the "reasons you need BSD", those are just reasons Neflix and WhatsApp need BSD.

You might one day want to run a startup or a project and then you could just do it on FreeBSD in any way you like, given the permissive license and this is exactly the reason why you need BSD, not only because Netflix or WhatsApp uses them to serve you the service.

I should donate to every project on github in case I need to use one somewhere in the future!

I can guarantee you'll need to use an operating system, they're rather essential. Ensuring the basic building blocks are available for future use is beneficial, and showing what has previously been built with them shows why.

No need to be smug, FreeBSD is hardly the only operating system available to us.

I'm not saying that it is, just that you can't compare it to "every project on github."

I'll try to explain my reasoning, then.

If Netflix and other very profitable companies are, as I'm told, contributing in both code and money to FreeBSD, and if I'm also being asked to donate, then the contributions from said corporations are obviously not enough to cover the costs of FreeBSD development.

Since I pay for Netflix, why should I double-pay by donating to FreeBSD? Seems to me that Netflix should be digging a little deeper into their pockets.

I don't understand your point re: Windows. If companies are paying for Windows, then Microsoft is receiving a payment which they have presumably set at a level they deem sufficient to support continuing development of Windows and other projects. I see no parallel to my point about FreeBSD other than the fact that both are operating systems.

You don't pay Netflix to use FreeBSD, you pay them for a streaming video service that would be indistinguishable to you if they used Linux.

I'd expect Netflix to pay enough for FreeBSD to fund things they're interested in for running their own infrastructure. This is how free software development works, if you have an itch you either scratch it yourself or pay someone to scratch it.

You wouldn't want a FreeBSD that would be entirely funded by Netflix either, because nobody would be working on anything Netflix itself didn't need, such a system would quickly end up being useless to you.

You're not supposed to do that so that the mega corps could do that. You're supposed to do that so that _anyone_ could do that and benefit from the open source code that's best suited for running a business, even if your idea of running a business and using open source is to not share anything back.

Any citations about Switch using FreeBSD by chance?

http://www.osnews.com/story/29698/Nintendo_Switch_runs_FreeB...

That line of citations just ends in a tweet quoting a copyright notice from the FreeBSD kernel.

That doesn't mean the Switch runs FreeBSD, just that Nintendo used some code from the kernel, which could be as small as adapting a single driver to their in-house OS.

It's not FreeBSD[1]. This is doubled up by the fact that the networking system module is called "bsd"[2].

[1] https://reswitched.tech/software/oss

[2] http://switchbrew.org/index.php?title=Sockets_services

Reply on news.ycombinator.com