DragonFly BSD 5.2 | Hacker News Reader

DragonFly BSD 5.2

dragonflybsd.org

159 pointsjoeschmoe350 comments

Just the other day I voiced [1] a concern on a ZFS thread about not enough modern filesystems [2] shipping as defaults (or being recommended as such), despite some of these filesystems having been around for many years.

This is a promising move to the contrary.

[1] https://news.ycombinator.com/item?id=16798244

[2] in some loose, but recognizable definition of modern, featuring checksumming, CoW, deduplication, snapshots...

I'm pretty sure the default is still UFS, but FreeBSD allows you to use ZFS as the root filesystem. I've been using this for a while and it's been incredibly solid. It's a first class feature that's supported by the bootloader and the GENERIC kernel. My experience with it has been uneventful, which is exactly how a filesystem should be.

Doesn't TrueOS default to ZFS?

It did when it was still called PC-BSD. I don't think it has changed.

It still is. From the homepage of the site:

"TrueOS has been using OpenZFS as its exclusive file system for several years, ensuring advanced OpenZFS functionality is heavily tested and 100% production-ready."

It did even before FreeBSD did.

In some ways I think we actually need simpler file systems more than fancier ones. Here's an exercise: create a USB flash drive that can be both read and written on any system it is plugged into, by any user, without using FAT.

Can't be done. It's 2018 and FAT is still apparently the best we can do for universally usable filesystem.

And don't go trying to pin this on Windows not supporting ext or something stupid like that, because an ext filesystem cannot by default be both read and written universally even on systems that have full ext support, because they bake in permissions and the default umask botches this simple use case.

I realize this is sort of tangential to what you're saying, but I think a big problem we have in tech communities is that we focus too much on fancy interesting things and not enough on stuff that gets things done simply.

What’s wrong with FAT?

It doesn't scale well and it is legally encumbered.

I really would have thought that the open source community could have tackled this problem, but no one seems interested.

Have you tried UDF?

http://duncanlock.net/blog/2013/05/13/using-udf-as-an-improv...

UDF has a lot of issues in actual implementation. For instance MacOS will only support UDF on an unpartitioned disk, while Windows will only support it on a partitioned one (IIRC was true for 7, may not be anymore). Additionally, since it has posix permissions, I expect it will have the same multi-user issue on posix systems that ext does, though I haven't tested this.

With DragonFly's architecture being designed with multiprocessing and clustering in mind, how does it fare (relative to Linux and other BSDs) in modern environments with many-core CPUs, horizontally-scalable architectures, reliance on VM-based workloads, and containerization?

I would think it would really shine, but haven't seen comparisons - or much market uptake.

Wouldn't significant advantages result in lower costs (e.g., fewer CPUs needed because of higher performance per core due to lower OS overhead) and therefore economics would drive greater use? If it hasn't, why not?

I did a little testing about ~4 years ago and found most of the speed increases were in the netstack, but I didn't do any computational testing either.

As for your last question, I think largely due to the fact that it's just not mainstream yet.

I'm not an expert on DragonFly, but from what I know it's design differences aren't really that radical. It differs substantially from FreeBSD, but it's not like a fundamental re-imagining optimized for SMP. Performance is a bit better in some places, and about the same in others.

I do think DragonFly is neat and quite interesting, but I suspect it didn't gain as much momentum because the community behind BSD is already small and reluctant to fracture without stronger reasons. It's worth noting that this is not like spinning off a new Linux distro where you're ultimately just repackaging different variations of the same basic OS - DragonFly is effectively an entirely different OS since the kernel is significantly different.

Slightly out of date, but still relevant benchmarks: https://www.phoronix.com/scan.php?page=article&item=freebsd1...

Thanks, I took a look through that and most (all?) of the benchmarks seemed to be fairly traditional tasks, some not even highly threaded.

I guess I'm wondering if complex, multi-box, cloud-scale workloads have any advantage on Dragonfly - or if the optimizations for clustering are in such a narrow band of use cases that modern architectures have passed it by just through brute force.

There's an irony here, in that Matt Dillon was an ex-Amiga hacker (IIRC, some of Dragonfly's architectural choices were inspired by the Amiga). Anyway, the Amiga's linear memory, co-processor-driven architecture was ridiculously more elegant than the PC's segmented memory, CPU-bound approach. Didn't matter though, a huge investment in working around the PC's limitations led to its market dominance.

So, if massive technical accommodations for less efficient OS architectures end up marginalizing Dragonfly, it would be a bit of history repeating itself.

And I'm sure the irony wouldn't be lost on Mr. Dillon.

Phoronix has done some benchmarks vs Linux https://www.phoronix.com/scan.php?page=article&item=dragonfl...

I’d bet it’s nearly nowhere. It doesn’t have the driver support of Linux.

I never heard of HAMMER2( or HAMMER) until just now, but a quick search seems awesome. I might have to give bsd another try, its been over a decade.

Anyone know what the state of GPU support actually is? The Supported Hardware page [1] only says "Intel GPUs are supported up to the Skylake generation (2016 machines)", but this seems to be outdated. At the very least, Matt Dillon himself reported good OpenGL performance with a Kaby Lake machine [2].

[1] https://www.dragonflybsd.org/docs/supportedhardware/

[2] http://lists.dragonflybsd.org/pipermail/users/2017-September...

DragonFly web documentation is wiki-based and tends to not always be up-to-date (contrary to man pages).

All recent Intel GPUs are supported, including Kabylake and Coffeelake.

A few years ago I wanted to try this as a local Minecraft server but it was 64 bit only and the spare machine I had was 32 bit. Glad this post came up because now I've got a 64 bit machine sitting in my basement I had forgotten about until now.

Can you not just spin up a cloud instance and boot it off of a cd image?

Are there any plans to bring HAMMER2 in to Linux?

Afaik the whole purpose of DragonflyBSD is HAMMER so no.

Can you go into more detail please? That sounds interesting. I thought it had to do with concurrency.

You're not wrong. Interview from 2004 [0]: "DragonFly split off from FreeBSD-5 over major architectural differences, not anything else. We really do feel that FreeBSD-5 is taking the wrong approach to SMP and building something that is so complex that it will ultimately not be maintainable. We think we have a better way."

But their original architectural decisions were done with this goal in mind [1]: "The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. This type of functionality requires a sophisticated cache management framework for filesystem namespaces, file spaces and VM spaces." It's not difficult to see how HAMMER came to be three to four years later, given their stated design goals.

[0] http://www.onlamp.com/pub/a/bsd/2004/07/08/dragonfly_bsd_int... [1] https://www.dragonflybsd.org/history/

Dragonfly also has lightweight kernel threads and some other different architectural decisions. Not just HAMMER. But you are right, it will never reach Linux.

Not at all, originally the whole purpose of DragonFly was a test ground for its scheduler. It moved on to being much more though, refactoring the entire source tree and rewriting parts of the system. The whole purpose of DragonFly is DragonFly.

From the discussion I've heard, it would be challenging even to port HAMMER2 to other BSDs, so I wouldn't expect to see it on Linux any time very soon.

Are there many users of dragon fly bsd? Are people using it in production, and why/why not?

Still haven’t quite managed to get Dragonfly going in Azure, but not given up on it either

MD5 release checksums? c'mon folks, get with the times...

I generated them; what would you prefer, and why?

I know MD5 hash collisions are vaguely possible, but I would think they would be extremely unlikely here.

FWIW Gentoo uses multiple algorithms plus file size in package checks, see https://wiki.gentoo.org/wiki/Repository_format/package/Manif...

I recently made an argument to add a new one, Streebog, and you can read the discussion at https://bugs.gentoo.org/show_bug.cgi?id=597736

There's no vague about it. MD5 collisions have been possible to generate for over a decade now.

http://www.win.tue.nl/hashclash/SoftIntCodeSign/

A sha256 or sha512 would be more appropriate.

If you don't trust the dragonfly team to not produce collisions, you probably shouldn't use the software anyway.

After this thread, I don't trust the dragonfly team to tie their shoes. Not a chance I'll be using their software.

The malicious case would be a bad actor distributing phony files that appear to check out because the flaws have been found that would allow someone to do so.

In practice today this requires that the bad actor collaborates with the real distributor.

MD5 fails to _collision_. A collision is when you can find two different things with the same hash. But being able to collide the hash is NOT the same as being able to find a second pre-image, which is what you'd need in order to get "phony files that appear to check out" if the person who originally issued the MD5 checksums didn't collaborate with you.

There have been practical demonstrations of using padding in several formats to generate valid files that collision with an original one, but with entirely different contents.

This means that it is possible that someone could download the real image, introduce some rootkit, and then tinker that (for instance, by adding a hidden file with carefuly crafted content) until the resulting md5 is the same as that of the original image. Then hack the server and upload the modified image in place of the original one, and everyone who installs Dragonfly is now their minion.

If you use a stronger hash (which is not harder for anyone than using md5), then this attack vector becomes impossible. So... even if it is a remote possibility, just use the stronger hash because it is just a dominant strategy (it has upsides, yet 0 downsides).

There have been no practical demonstrations of colliding the MD5 of an arbitrary file (ie. pre-image attack), only situations where two files are created specifically with the intended purpose of creating a collision. This is precisely what the post you replied to said but you seem to have not understood that there's a distinction.

Yes, it is possible that the DragonFly developers could collude to create two ISOs with the same MD5, one good and one malicious. No, it is not possible that random, evil ne'erdowells could replace the ISO with one with the same MD5, unless the DragonFly developers have conspired with them to make that possible.

If you don't trust the DragonFly developers not to collide the MD5s, you probably shouldn't trust them with the code running in your kernel anyway.

Those flaws have in fact not been found in MD5.

The issue is that if these files are distributed elsewhere by 3. parties, it is trivial for those 3. parties change and compromise the files, but still make the files produce the same MD5 sum.

Wouldn't it be easier for said third parties to just distribute changed checksum?

Then it would also be trivial to discover if you go and look up the sums at the official site.

If you think a preimage attack against MD5 is "trivial" you should demonstrate it. People would be very interested in this because no one has managed to do it yet.

Creating two files with the same MD5 is a very different beast from creating a file with the same MD5 as an arbitrary pre-existing file. These third parties would need to have colluded with the DragonFly developers to make what you're proposing possible.

Should really be using SHA256 or BLAKE2 at this point

While the likelihood of a collision (whether "accidental" or malicious) is probably still relatively low even with MD5, what reasonable, compelling reason is there to still be using MD5 -- especially when much stronger hash algorithms such as SHA256 and SHA512 are available?

Generate both SHA256 and SHA512 hashes or maybe SHA512 and some other "unrelated" algorithm (if you want to really play it safe), dump them into a checksums.txt (or whatever) file, then PGP/GPG sign that file (with a widely distributed and certified/signed public key, of course) and you can effectively eliminate any chances of a collision whatsoever.

It seems that the benefits of switching would greatly outweigh the costs associated with doing so (unless this would require some major code changes to your processes/pipelines/etc.).

Rather than "vaguely possible" MD5 collisions are now so easy you can make new ones at home on an ordinary PC in a few seconds, and you can do chosen prefix collision (thus enabling you to pick a "format" and collide within that) in a few hours. Cryptography never gets _less_ broken, only more broken with time.

Now, in this particular scenario (a publisher tells us the MD5 of an image, and we can check it to see we got the image) collision isn't so important. We have to trust you anyway, so we may as well trust you to not collide the hash too. But MD5 is no longer even proof against second pre-image attacks, albeit the best known are as yet impractical. This is really bad news.

MD5 has been known to be irrevocably broken since 2004, and had been expected to fall since the mid to late 1990s). DragonFly BSD was only started in 2003. Why use MD5? Imagine if in 2003 you'd decided to build a 16-bit OS, or one that doesn't do TCP/IP, because after all, in the mid-to-late 1990s that might have seemed fine too...

Don't think the MD5 is there for security reasons, just to detect broken downloads.

Reply on news.ycombinator.com