7 watts idle – building a low powered server/NAS on Intel 12th/13th gen

I personally don't need more than 10TB space, so 2 10TB Seagate EXOS in ZFS mirror would just work fine with 120W as long as you don't run Prime95 the whole time. You might go up to Pico PSU 150 then.

Staggered spin-up it was called.

Turned on one disk after another.

So the PSU didn't bother, or stalled.

Done.

It surprises me that people are happy with 64GB+ builds of non-ECC, especially for NAS (ie. very long term storage, where corruptions probably wouldn't be noticed for years).

Periodically I look at replacing my small fleet of HP micro Gen8's, which use Xeons, have 4 x 3.5" bays (with proper h/w RAID1), but max out at a frustratingly low 16GB. They're quite robust, but because of their age - and HP - a horse veterinarian approach to component failures is usually indicated.

A Ryzen + ECC whitebox build is massively appealing, but almost everyone's build-out includes caveats like 'check the datasheet of the mobo and CPU' (because series aren't consistent), and about half the time a terrifying disclaimer to the effect of 'ECC is present / enabled in the kernel, but I can't tell if it's actually functioning properly'.

If so, I wonder if there is a hit in efficiency because of the required step-up / step-down converters in that adapter.

However, on my board there seems to be an unsoldered 24pin connector, that could be just used as is with a little soldering, but since it is on-hold my replacement system if my ...-B variant dies, I'm not willing to risk too many experiments :-)

I think we'd still arrive at a weird place where we see people buying Asustor or other NAS hw just to run DSM even if it's not supported and but not a complete hack.

People buy convenience and DSM offers everything NAS related it really well.

I suppose they have internal plans to make DSM go the other way and lock out attempts like Xpenology. They aren't late either as other rival is still miles behind.

My home net is 1G with two MikroTik hAP ax3 which are connected with the single 2.5G poe port they have (and one powers the other).

For less power consumption Ryzen 8000 is coming up (wait for Jan 8th, CES) and the APU tend to be monolithic and draw a lot less power than the chiplets.

Either way, it's awful there is not more 10 GbE connectivity available default. There's no reason it shouldn't be the next level up, we have been at 1 / 2.5 for far too long.

My build is storage, gaming and a bunch of VMs.

Used Epyc 7000 was the other option for a ton more PCIe. I have no need for more network speed.

And yes they start at around 500.

Same with TrueNas mini

Love my X9 and X11 boards.

What hardware would that be, specifically? The low end embedded platforms that don't even support ECC?

> how many recoveries and transitions have you been through

3 or 4, at this point, using the same 2 disk zfs mirror upgraded from 1TB to 3TB to 10TB.

The tl;dr of my rant was around shortcomings in NFS permission configuration, and a failure of the iSCSI feature (the appliance crashed when you sent it data).

Further, these appliances invariably use vanilla RAM sticks, so you're exposed to gentle memory-based file corruption you probably won't notice for years.

So I'd argue the hardware is 'better equipped', and I'd also argue the software as shipped matches the marketing promises accompanying same.

Things have doubtless changed - I'm sure those bugs are long gone now - but unless you're looking at an ECC appliance, I'd say you're better off building your own white box.

[0] https://jeddi.org/b/brief-rant-on-trying-to-use-iscsi-on-a-q...

That's my setup for nearly 1/3 PB.

ZFS is designed for HDD-based systems after all. actually it works notably kinda poorly for SSDs in general - a lot of the design+tuning decisions were made under the assumptions of HDD-level disk latency and aren't necessarily optimal when you can just go look at the SSD!

however, tons and tons of drive spin-up cycles are not good for HDDs. Aggressive idle timeout for power management was famously the problem with the WD Green series (wdidle3.exe lol). Best practice is leave the drives spinning all the time, it's better for the drives and doesn't consume all that much power overall. Or I would certainly think about, say, a 1-hour timeout at least.

https://www.truenas.com/community/threads/hacking-wd-greens-...

However, block-level striping like ZFS/BTRFS/Storage Spaces is not very good for spinning down anyway. Essentially all files will have to hit all disks, so you have to spin up the whole array. L2ARC with a SSD behind it might be able to serve a lot of these requests, but as soon as any block isn't in cache you will probably be spinning up all the disks very shortly (unless it's literally 1 block).

Unraid is better at this since it's a file-level striping - newer releases can even use ZFS as a backend but a file always lives on a single unraid volume, so with 1-disk ZFS pools underneath you will only be spinning up one disk. This can also be used with ZFS ARC/L2ARC or Unraid might have its own setup for tiering hot data on cache drives or hot-data drives.

(1-disk ZFS pools as Unraid volumes fits the consumer use-case very nicely imo, and that's going to be my advice for friends and family setting up NASs going forward. If ZFS loses any vdev from the pool the whole pool dies, so you want to add at least 2-disk mirrors if not 4-disk RAIDZ vdevs, but since Unraid works at a file-stripe level (with file mirroring) you just add extra disks and let it manage the file layout (and mirrors/balancing). Also, if you lose a disk, you only lose those files (or mirrors of files) but all the other files remain intact, you don't lose 1/8th of every file or whatever, and that's a failure mode that aligns a lot better with consumer expectations/needs and consumer-level janitoring. And you still retain all the benefits of ZFS in terms of ARC caching, file integrity, etc. It's not without flaws, in the naive case the performance will degrade to 1- or 2-disk read speeds (since 1 file is on 1 disk, with eg 1 mirror copy) and writes will probably be 1-disk speed, and a file or volume/image cannot exceed the size of a single disk and must have sufficient contiguous free space, and snapshots/versioning will consume more data than block-level versioning, etc. All the usual consequences of having 1 file backed by 1 disk will apply. But for "average" use-cases it seems pretty ideal and ZFS is an absolutely rock-stable backend for unraid to throw files into.)

anyway it's a little surprising that having a bunch of individual disks gave you problems with ZFS. I run 8x8TB shucked drives (looking to upgrade soon) in RAIDZ2 and I get basically 8x single-disk speed over 10gbe, ZFS amortizes out the performance very nicely. But there are definitely risks/downsides, and power costs, to having a ton of small drives, agreed. Definitely use raidz or mirrors for sure.

5400 drives? How many and how bad the performance was?

In practice, I found I needed a bit more power but you can get some of the Fujitsu boards with a CPU for $30-40, which is hard to beat.

my university issued T4220 convertible laptops, with wacom digitizers in the screens. I rarely used it but the pivot in the screen made it indestructible, it survived numerous falls hitting the corner of the screen/etc because the screen simply flops out of the way and pivots to absorb the energy. I later got a ST6012 slate PC that my uni bookstore was clearing out (also with a wacom digitizer, and a Core2Solo ULV!). Both of them are extremely well-thought-out and competently designed/built hardware. Doesn't "feel" thinkpad grade, but it absolutely is underneath, and featured PCMCIA and bay batteries and other power-user features.

https://www.notebookcheck.net/Fujitsu-Siemens-Lifebook-T4220...

https://www.ruggedpcreview.com/3_slates_fujitsu_st6012.html

They also did a ton of HPC stuff for Riken and the other japanese research labs, they did a whole family of SPARC processors for mainframes and HPC stuff, and pivoted into ARM after that wound down. Very cool stuff that receives almost no attention from mainstream tech media, less than POWER even.

https://www.youtube.com/watch?v=m0GqCxMmyF4

Anyway back on topic but my personal cheat-code for power is Intel NUCs. Intel, too, paid far more attention to idle power and power-states than the average system-integrator. The NUCs are really really good at idle even considering they're using standalone bricks (my experience is laptop bricks are much less efficient and rarely meet 80+ cert etc). A ton of people use them as building blocks in other cases (like HDPlex H1 or Akasa cases), they don't have a ton of IO normally but they have a SATA and a M.2 and you can use a riser cable on the M.2 slot to attach any pcie card you want. People would do this with skull canyon f.ex (and HDPlex H1 explicitly supports this with the square ones). The "enthusiast" style NUCs often have multiple M.2s or even actual pcie slots and are nice for this.

https://www.amazon.com/ADT-Link-Extender-Graphics-Adapter-PC...

And don't forget that once you have engineered your way to pcie card formfactor, you can throw a Highpoint Rocket R1104 or a SAS controller card in there and run multiple SSDs (up to 8x NVMe) on a single pcie slot, without bifurcation. Or there are numerous other "cheat code" m.2 devices for breaking the intended limits of your system - GPUs (Innodisk EPV-1101/Asrock M2_GPU), SATA controllers, etc.

https://www.youtube.com/watch?v=M9TcL9aY004 (actually this makes the good point that CFExpress is a thing and is very optimized for power. No idea how durable they are in practice, and they are definitely very expensive, but they also might help in some extreme low power situations.)

Personally I never found AMD is that efficient at idle. Even with a monolithic apu you will want to dig up an X300TM-ITX from aliexpress, since this allows you to forgo the chipset. Sadly AMD does not allow X300 to be marketed directly as a standalone product, only as an integrated system like a nuc or laptop or industrial pc, despite the onboard/SOC IO being already quite adequate for beige box usage. Gotta sell those chipsets (hey, how about two chipsets per board!?). But OP article is completely right that AMD’s chipsets just are not very efficient.

Thinking about chucking in 25gbit cards.