Hacker News Reader

Comment by mhw | Hacker News Reader

> Examples of things you don't want to do if you want to keep your data intact include using non-ECC RAM

I'm planning to build a (Linux) development workstation soon and was intending to run ZFS on at least a proportion of the disks in the system, if not the whole thing. But building a workstation that supports ECC appears to be ridiculously hard and expensive - most of the Haswell CPUs don't support ECC. Can anyone point me at a sensible CPU/motherboard combo that's comparable to something like an i7-4770k/Gigabyte GA-Z87MX-D3H?

(As an indication of the state of the component market, pcpartpicker.com doesn't support searching CPUs or motherboards by ECC support, but it does allow you to search by what colour the motherboard is!)

View on HN

YMMV, but I've used ZFS on all my workstations and servers for over 2 years now. Many of them are ZFS on top of LUKS/cryptsetup for FDE. None of the machines use ECC RAM, and due lots of things (ie. me doing dumb shit, bad cutting edge kernel builds, fubar GFX drivers causing X to boot to black screen etc) I frequently either hard reset them, or Alt + Prtscreen + REISUB them, I've never had a single issue with ZFS.

By comparison, I've had issues with virtually every other filesystem that exists for Linux: ext(2,3,tho not 4), reiserfs, and especially the much hated xfs, death to xfs etc :)

I run his hardware, picked because... well cheap:

   GIGABYTE GA-990FXA-UD5 ( AMD 990FX/SB950 - Socket AM3+ - FSB5200 ) 
   4 x DDR3 4GB DDR1866 (PC3-15000) - KINGSTON HyperX [KHX1866C9D3/4G] 
   or 
   4 x 16gb of the same type as above
   AMD FX 8-Core FX-8320 [Socket AM3+ - 1000Kb - 3.5 GHz - 32nm ]

The ZFS all include root + swap running on ZFS with FDE, this sort of thing for workstations:

   NAME                    USED  AVAIL  REFER  MOUNTPOINT
   rpool                  13.2G  25.9G    30K  none
   rpool/root             11.1G  25.9G  5.10G  /
   rpool/root/home-users  6.02G  25.9G  6.02G  /home
   rpool/swap             2.06G  27.4G   646M  -

servers:

   NAME                         USED  AVAIL  REFER  MOUNTPOINT
   rpool0/ROOT/ROOT               6G  1.42T   198K  /
   rpool0/ROOT/swap              32G  1.42T   700M  -
   rpool0/ROOT/BACKUP           676G  1.42T   676G  /nfs/backup
   rpool0/ROOT/HOME            14.1G  1.42T  14.1G  /home
   rpool0/ROOT/MUSIC            358G  1.42T   358G  /nfs/music
   rpool0/ROOT/portage         14.9G  1.42T  14.9G  /nfs/portage
   rpool0/ROOT/vmware          45.9G  1.42T  45.9G  /vmware

The server hosts several virtual machines, many of which have their own root over NFS, the point being that the ZFS gets plenty of use.

You ever try JFS? I standardised on it for all my machines a few years ago and it's served me very well - fast fsck and no corruption so far. Bit long in the tooth now but that's no bad thing for a filesystem.

I have not, but the only reason for not trying it is because of my increasing use of RAID in hardware and VMs, I wanted something that was completely agnostic to disks, which is something ZFS delivers but JFS didn't seem to when I checked.

Basically I wanted something that doesn't intertwine the concept of file-system with the concept of disks or partitions, but as space. So changes to the underlying pool of disks (or partitions, or loopback files, etc) don't mean so much painful rebuilding for me, something I spent many hundreds of hours on in the past, often at stupid o'clock in the morning. Did I mention how much I hate XFS? That's why :)

I feel your pain -- I've been trying to figure out the state of (cheap-ish) ecc support for a while now (the freenas forums can be a big help here).

Basically you can roll the dice with AMD, or just go with Intel (what I'll be doing). If you want reasonable performance, get a socket 1150 xeon v3 (basically a re-branded i7 with ecc support). The low-mid xeon v3 appear to score pretty well on a price/performance ratio. Can't be overclocked, and you'd probably get more single thread performance/price with an i5 -- but apart from that they seem pretty solid. As others have mentioned, (some) i3s are also an option -- but as you'll likely have to pay a little more for a main board with ecc support, that doesn't make much sense as I see it.

The final option is to get one of the newish atom based Avoton boards, like ASROCK C2750D4 INTEL C2750 AVOTON OCTACORE MITX -- not for a workstation, though. But should be nice as a NAS.

If anyone knows of a useful overview of AMD cpus and mainboard combinations that support ECC -- please let us know. I've yet to find anything beyond the anecdotal "Here's my build, and it's got ECC sticks in it, and maybe the ECC is actually enabled, but I haven't really checked."

While the recommendation is in general sound, I don't think the article applies to using ZFS on a workstation. It's about using ZFS as a storage subsystem (of multiple TB size) where data must be kept safe and without errors.

You can still get most of the benefits on a workstation with non-ECC RAM, consider that any other file system would have the same challenge to deal with on non-ECC RAM.

So, as in all things, it's a money/benefits tradeoff: Does the data you're working with warrant spending a ridiculous amount of money?

You need a Xeon v3 for the ECC support, basically. Also obviously getting ECC ram for stability and getting a K series CPU for overclocking is just an incompatible mindset.

You can use ECC on some i3 CPUs [1] as long as the motherboard supports it. Supermicro has a X10 micro-atx board [2] which would work.

[1] http://ark.intel.com/search/advanced?s=t&FamilyText=4th%20Ge...

[2] http://www.supermicro.com/products/motherboard/Xeon/C220/X10...

I wonder if an i3 has enough grunt for ZFS. I definitely notice the difference between a middling i3 and my i7 2600k.

Fiddling with the filters on that page, it seems some Atom processors support ECC, too - http://ark.intel.com/products/81270/Intel-Atom-Processor-C25...

Supermicro has a ton of mainboards with ECC support. Many of them have integrated "RAID" or JBOD. 8 SAS + 6 SATA is typical. I haven't checked the prices for a while, but last time I bought, it was about $250 for a mainboard with integrated 8x SAS JBOD and 6x SATA. Very cheap, when you consider no separate JBOD card is required.

Those boards and their SAS chipsets (LSI SAS2008, etc.) tend to be well supported on platforms capable of running ZFS, like Solaris, FreeBSD etc.

Other typical goodies include IPMI (integrated IP-KVM), good quality dual NICs and generally good quality components.

Most AMD chips support ECC on the cpu, many compatible motherboards include it but don't bother to specify on the spec sheet. I was running an Asus M5A97 with ECC with ZFS for a few before it died a tragic early death.

I don't know what your budget is, but I found that the ASRock E3C226D2I Mini ITX "Server Motherboard" fit my needs well. Being an LGA 1150 board, it seems to support the i7-4770k CPU you mentioned and is about $100 more than your comparable setup.

It has 2 USB 3.0 ports, supports up to 16GiB of RAM, and has 6 SATA 6Gb/s ports, so I think it would also work well for a workstation board, if you added a video/audio card -- HDMI out is the only glaring omission for a workstation.

I'm using it in my FreeNAS fileserver and have been happy with it.

Reply on news.ycombinator.com