At this point, picking AMD for your CPU becomes such a no-brainer. Compounded by Intel’s security issues and all.
If you want verified ECC support, you need to buy the workstation chips and motherboards: Threadripper Pro or EPYCs.
I went through the effort of using qvl memory, but actually testing ECC is a bit more difficult. While ECC is supported & active, memory errors are sadly not reported to the OS. I remember seeing a forum post somewhere of somebody overclocking/undervolting the ram to force errors, but I can't seem to find it right now. There's a fine line between stable, stable with recovered errors, and unstable.
So on Linux, ECC is fully supported, even with Ryzen.
https://www.servethehome.com/asrock-rack-x470d4u-review-amd-...
With Ryzen 3xxx, I have used ASUS Pro WS X570-ACE ($315), which is sold as a workstation board, so you definitely should expect ECC to work without problems, and also the Mini-ITX MB ASRock X570 Phantom Gaming-ITX/TB3 ($230), which also worked OK.
I expect that the other ASRock MB's (most of them or maybe all of them specify the support of ECC) also work OK with ECC modules.
The ASrock Rack server board should also not have any problems with ECC. IIRC that server board supports only up to DDR4-2933, but until now faster unbuffered ECC memory modules were not sold anywhere, so that is not a disadvantage.
Edit: I have looked again at the ASRock site and they have updated the memory support specification. If you use only 2 UDIMM modules (i.e. up to 64 GB total), then you can use up to DDR4-3200 (which I have never seen offered anywhere until now; 2666 is easy to find, 2933 is also supported by Intel since March, so it should become available soon).
https://forums.freebsd.org/threads/how-to-find-out-if-ecc-is...
Example: https://www.asus.com/no/Motherboards/Pro-WS-X570-ACE/HelpDes...
Extra funny when you notice that certain Xeon lines are actually i7 with different branding and ECC left enabled.
The problems with ECC on AMD comes from consumer vendors not putting the time into testing, and possibly not even connecting the ECC lines (remember, ECC requires putting additional traces between memory controller and memory slots). Then you have to deal with whatever customisation the vendor of the motherboard did to firmware - their changes might have resulted in effective disabling of ECC.
With Intel, you either have the same game as above (with the non-Xeon ECC-capable parts), or pay through the nose for comparable performance "workstation/enterprise" gear, as ECC support being used for market segmentation by intel is pretty much an open secret.
The only advantage of Intel Xeon at this point is RDIMM/LR-DIMM support on their Xeon-W platform. AMD supports that only on Epyc and Threadripper Pro.
And that gen of CPUS had low power T-series chips rated at 65w where today it's 35w. The power drop is part of why I'm happy with my SFF setup as the thermals are quite good on 14nm locked CPUs like my 7700.
It's a shame they made the TR platform much more expensive in the last generation.
For example I am using an ASUS Pro WS X570-ACE, which is a reasonably priced workstation board ($300) with a Ryzen 7 3700X and ECC memory.
ECC worked OK, without any problems. I have also used a couple of ASRock MB's and ECC also worked OK on them.
I would much prefer more guarantees from AMD, but rather than buying a slow Intel CPU I prefer a little risk with AMD.
In that price range, AMD markets Threadripper and Epyc, both with proper ECC support.
ECC support in Ryzen systems is up to the motherboard manufacturer, and some manufacturers advertise support very clearly. E.g., at least a couple years ago, ASRock explicitly supported ECC in all their Ryzen motherboards.
Most people wanting ECC just for the warm fuzzy enthusiast feeling would prefer Ryzen with good MB that supports ECC. Intel makes more sense at the high-end many-core Xeons, due to supporting huge memory with much higher throughput (LGA3647). But that segment is bonkers-expensive.
That's not a real need, of course, it's just some kind of whim, but if I can pay for it, why not. Just like those crazy overclocker dudes pay for their whims.
tl;dr: one bit error in 4GB every 72 hours
https://nakedsecurity.sophos.com/2011/08/10/bh-2011-bit-squa...
It would be interesting to repeat this experiment today.
1. https://web.archive.org/web/20010612184424/http://www.boeing...
FIT (failures in time, or failures per billion hours) are about 20-30 per DRAM (18 per DIMM in the study) at a fault level, which may not rise to an error condition. ECC corrects many faults, if present, which don't become errors.
So expect a (4GB in this study) DIMM to have a fault once per 200 years on average. Of course, this is server-class hardware running ideal conditions.
This is, notably, in stark contrast to Google's results (https://research.google.com/pubs/archive/35162.pdf&) of about 20,000-70,000 FIT per Mbit, but closer reading suggests that there were a fairly large number of high-fault DIMMs affecting the mean and Google historically bought the cheapest hardware available. Even at this fault rate an average DIMM wouldn't fail in any given month, for example.
Personal gaming, browsing and light document work PCs do not need ECC. Video/photo editing/development PCs might be on the threshold for not wanting to produce or store corrupted work.
Multiply the risk out for any organization with many people doing the same kind of work where corruption on a single workstation could cause annoying trouble for everyone else.
You want ECC in your desktop if you cannot afford your data to become (silently) corrupted. A single bit-flip in a JPEG image can totally ruin it. And you won't notice without opening it, because the thumbnail is unaffected. When you finally notice, the corruption has possibly spread to backups already.
For servers that is especially important, because data will reside cached in memory for days or even weeks. Also ECC is often the last line of defense against Rowhammer.
Here is an example: ASUS Pro WS X570-ACE
AMD is pretty comparable in that regard: https://en.wikipedia.org/wiki/Video_Core_Next but I don’t have computers with AMD APUs or GPUs, and don’t have a hands-on experience with these features.
I'm still probably 3-6mo away from a new server build so I'll just re-evaluate then I guess and honestly I might just go with another storage server and leave my intel/QS server as-is and just go with a AMD that plays nice with UnRaid.
It has a maximum of 64gb ram and is tiny. With an nvme drive (Samsung Evo) it is really really fast.
The next best option as far as I was concerned was the NUC8, and while I’d love to have the PSU onboard and no brick, a Mac Mini is a lot of money.
The Nuc8 is a better option than the 10 for anything that’s needing actual graphics. The 10 has a very anaemic GPU compared to the 8, but has 2 more CPU cores when comparing i7s.
To everyone else: If you have an Intel CPU that supports QS and you are running Plex in a docker container then I HIGHLY recommend you look into enabling HW acceleration (both in plex settings and by mounting a device, /dev/dri, into your container). Here [0] are the directions I followed. Make sure you read the instructions, the first time I tried to follow this I misread the instructions and thought my CPU didn't support it or there was some other issue but it couldn't have been easier when I actually stopped and read instead of skimming.
[0] https://forums.unraid.net/topic/77943-guide-plex-hardware-ac...
https://www.google.com/amp/s/amp.reddit.com/r/PleX/comments/...
> I hadn't considered a NUC to run the plex server
Mine is more of a Docker host. Home Assistant absolutely flies on that hardware (it was previously on a Synology 918). It has a few other well used containers too - Wireguard, several Minecraft and some administrative tools. There are several testing machines for pfsense and some Linux machines I wanted to play with too.
In my experience, GPU vendors, all 3 of them, are including reasonably well-made media foundation hardware transforms as a part of their GPU drivers. MF API is vendor agnostic. Apart from a few bugs I found in Intel’s drivers (it was about h265 hardware encoder, Intel neglected to react), the same API works with all capable hardware.
I agree, it's convenient to have a GPU on most CPUs, but AMD puts less priority on that market, so we just have to wait.
There are also things like Intel MKL. A lot of software can use it when compiled on a user machine.
AVX1 and AVX2 performance is on par.
For instance, vmulpd AVX1 instruction is faster on AMD, 3 versus 4 cycles. vpaddd AVX2 instruction is same at 1 cycle latency. vfmadd132pd FMA instruction is slightly faster on Intel, 4 versus 5 cycles. Throughput is the same across these two. I was looking at AMD Zen2 versus Intel Ice Lake.
Some Intel chips have AVX512. Still, many practical applications don’t need that amount of SIMD wideness, and these who do are often a good fit for GPGPUs.
> There are also things like Intel MKL
There’re vendor-agnostic equivalents like Eigen.
Also https://www.intel.com/content/www/us/en/artificial-intellige...
Also Tensorflow is awesome fit for GPGPUs and is usually way faster on them.
AFAIK this is a break from previous Intel nomenclature, where any processor supporting e.g. "SSE4.2" instructions was guaranteed to support all SSE4.2 instructions.
I'm concerned that sometimes this causes confusion when talking about processor -- software compatibility.
That's looking at the wrong layer of the hierarchy, I think. There are many open-source linear algebra libraries, but iirc they all link against something that has a BLAS/LAPACK API. That might be something like MKL, OpenBLAS, ATLAS, etc.
When I last checked, MKL was much faster than its competitors, and is only available (at full speed) on Intel CPUs. Has that changed?
Eigen can consume these I think, but they are optional. It has it’s own implementation of these, written in manually vectorized C++, with intrinsics, up to and including AVX512 (controlled with macros). For parallelization it uses OpenMP provided by the compiler (also controlled with a macro).
> Has that changed?
It’s hard to directly compare Eigen to the rest of them. They don’t do the same thing.
One feature of Eigen is lazy evaluation. Expressions like a+b or a×b don’t return another matrix or vector; they return a placeholder object that only computes something on assignment. For complicated expressions this can be a huge win, e.g. r=a+b+c+d will read from a,b,c,d, compute sum of the 4 on the fly, and write into r without temporary copies in memory.
However, also makes Eigen’s source code outright scary, and hard to debug or optimize.
Anyway, based on the old pics there http://eigen.tuxfamily.org/index.php?title=Benchmark they are more or less comparable. Things like alpha·X+beta·Y were much faster in Eigen (probably due to that lazy evaluation thing), Hessenberg was much faster in MKL, in general they are close.
That has never been generally true in my experience measuring over the years. It has been true at times for specific cases, e.g. OpenBLAS until it got avx512 support on a par with MKL (at least for serial DGEMM -- I've forgotten quite how the rest of level 3 goes).
The cheaper models with only one AVX-512 FMA unit have a lower throughput, which will be exceeded by Zen 3, even at the same clock frequency.
For multi-threaded tasks, Zen 3 CPUs will have a higher clock-frequency than any Intel CPU, so it is expected that any older Intel CPU will be beaten easily.
It remains to be seen which will be the performances of the Ice Lake Server CPUs, to be launched before the end of the year. However, miracles are not expected, because these are using the older Intel 10 nm technology, not the improved one used by Tiger Lake.
I think there are relatively few with only one FMA (and there's no way of interrogating them at runtime, sigh) but, yes, if you know you have one, you use AVX2 for GEMM kernels, as a specific example.
For general computational workloads, you're likely better off with more AVX2 cores and high memory bandwidth, even without whatever improvements there are in Zen 3.
Well, directly, no.
Indirectly ... find the model and use a lookup table to find how many FMA units it has :P
However most server models with AVX-512 have only one 512-bit FMA unit. That includes all Xeon Bronze, all Xeon Silver and a few Xeon Gold.
All Xeon Platinum and most Xeon Gold have dual units, but those are extremely expensive.
I'd think AVX512 would still be advantageous for gemm kernels. If you use a 16 x 14 microkernel instead of an 8 x 6 microkernel, you'll reduce memory bandwidth by halving the passes over the cache-sized blocks in memory. Well optimized implementations don't really have a problem with this (getting close to the CPU's peak flops on AVX2), but it still seems better than not doing it, and is an advantage for code with register tiling more generally.
I suspect that it'd help people using Tullio.jl in Julia, for example.
The mask registers can eliminate special prologue or epilogue code for the loops and having more and larger registers make it much easier to overlap enough computations so that the latencies of the operations are hidden.
I would never buy again an Intel CPU without AVX-512, because that has already for some time been their main advantage and now it remains their only advantage over AMD.
Unfortunately for Intel, even if AVX-512 is a great improvement, it cannot compensate for a number of cores half of the competition. Intel needs to launch the 8-core Tiger Lake H no later than March 2021, but it would be better for them if they could do it earlier.
That's not what I understood from people who've gone through the exercise for GEMM. One probably relevant measurement: on BLIS' generic C GEMM micro-kernel, GCC doesn't get nearly as close to the hand-coded avx512 version as for avx2 with appropriate bock sizes.
There are also things like OpenBLAS, and BLIS (which AMD support).