Hacker News Reader

Comment by ahl | Hacker News Reader

ZFS was developed in Solaris, and at the time we were mostly selling SPARC systems. That changed rapidly and the biggest commercial push was in the form of the ZFS Storage Appliance that our team (known as Fishworks) built at Sun. Those systems were based on AMD servers that Sun was making at the time such as Thumper [1]. Also in 2016, Ubuntu leaned in to use of ZFS for containers [2]. There was nothing that specific about Solaris that made sense for ZFS, and even less of a connection to the SPARC architecture.

[1]: https://www.theregister.com/2005/11/16/sun_thumper/

[2]: https://ubuntu.com/blog/zfs-is-the-fs-for-containers-in-ubun...

View on HN

> There was nothing that specific about Solaris that made sense for ZFS, and even less of a connection to the SPARC architecture.

Although it does not change the answer to the original question, I have long been under the impression that part of the design of ZFS had been influenced by the Niagara processor. The heavily threaded ZIO pipeline had been so forward thinking that it is difficult to imagine anyone devising it unless they were thinking of the future that the Niagara processor represented.

Am I correct to think that or did knowledge of the upcoming Niagara processor not shape design decisions at all?

By the way, why did Thumper use an AMD Opteron over the UltraSPARC T1 (Niagara)? That decision seems contrary to idea of putting all of the wood behind one arrow.

I don’t recall that being the case. Bonwick had been thinking about ZFS for at least a couple of years. Matt Ahrens joined Sun (with me) in 2001. The Afara acquisition didn’t close until 2002. Niagara certainly was tantalizing but it wasn’t a primary design consideration. As I recall, AMD was head and shoulders above everything else in terms of IO capacity. Sun was never very good (during my tenure there) at coordination or holistic strategy.

Niagara did not shape design decisions at all -- remember that Niagara was really only doing on a single socket what we had already done on large SMP machines (e.g., Starfire/Starcat). What did shape design decisions -- or at least informed thinking -- was a belief that all main memory would be non-volatile within the lifespan of ZFS. (Still possible, of course!) I don't know that there are any true artifacts of that within ZFS, but I would say that it affected thinking much more than Niagara.

As for Thumper using Opteron over Niagara: that was due to many reasons, both technological (Niagara was interesting but not world-beating) and organizational (Thumper was a result of the acquisition of Kealia, which was independently developing on AMD).

Thanks. I had been unaware of the Starfire/Starcat machines.

Yeah I think if it hadn’t been for the combination of Oracle and CDDL, Red Hat would have been more interested in for Linux. As it was they basically went with XFS and volume management. Fedora did eventually go with btrfs but dints know if there are are any plans for copy-on-write FS for RHEL at any point.

Fedora Server uses XFS on LVM by default & you can do CoW with any modern filesystem on top of an LVM thin pool.

And there is also the Stratis project Red Hat is involved in: https://stratis-storage.github.io/

It looks like btrfs is/was the default for just Fedora Workstation. I’m less connected to Red Hat filesystem details than I used to be.

TIL Stratis is still alive. I thought it basically went on life support after the lead dev left Red Hat.

Still no checksumming though...

RedHat’s policy is no out of tree kernel modules, so it would not have made a difference.

It’s not like Red Hat had/has no influence over what makes it into mainline. But the options for copy on write were either relatively immature or had license issues in their view.

Their view is that if it is out of tree, they will not support it. This supersedes any discussion of license. Even out of tree GPL drivers are not supported by RedHat.

We had those things at work as fileservers, so no containers or anything fancy.

Sun salespeople tried to sell us the idea of "zfs filesystems are very cheap, you can create many of them, you don't need quota" (which ZFS didn't have at the time), which we tried out. It was abysmally slow. It was even slow with just one filesystem on it. We scrapped the whole idea, just put Linux on them and suddenly fileserver performance doubled. Which is something we weren't used to with older Solaris/Sparc/UFS or /VXFS systems.

We never tried another generation of those, and soon after Sun was bought by Oracle anyways.

I had a combination uh-oh/wow! moment back in those days when the hacked up NFS server I built on a Dell with Linux and XFS absolutely torched the Solaris and UFS system we'd been using for development. Yeah, it wasnt apples to apples. Yes, maybe ZFS would have helped. But XFS was proven at SGI and it was obvious that the business would save thousands overnight by moving to Linux on Dell instead of sticking with Sun E450s. That was the death knell for my time as a Solaris sysadmin, to be honest.

ZFS probably wouldn't have helped. One of my points is, ZFS was slower than UFS in our setup. And both where slower than Linux on the same hardware.

Thanks. Also, the Thumper looks awesome like a max-level MMORPG character that would kill the level-1 consumer Synology NAS character in one hit.

Reply on news.ycombinator.com