ECC Memory and AMD's Ryzen – A Deep Dive | Hacker News Reader

ECC Memory and AMD's Ryzen – A Deep Dive

hardwarecanucks.com

130 pointsnacc20 comments

The conclusion of the article is mostly false at least for Linux: UE errors have no reasons to panic the machine in all cases, and by default under Linux the affected processes are simply killed. Of course, if this is kernel memory, then you will panic, but the probability of it being kernel memory is low (amount of kernel memory / total memory...). This has been signaled in the comments (not by me) but unfortunately the article has not been updated to reflect that fact. Also, there is no reason that the policy is not managed by the software, so as long as it is detected, the kernel is free to do what it wants with UE, and everything is fine.

Also I have no proof that any crazy thing can not happen, but there is no reason for single bit errors not to be corrected regardless of the OS. The worse that should happen for them is to not be reported.

IMO if you have the opportunity (the category of HW you want supports it) you would be crazy not to use ECC RAM. Non-ECC RAM is basically the only component in a PC that is not protected. Obvious weak point. I've been beaten at least twice (two defective components, way more than 2 errors before I figured out what was happening) only on computers I was directly owning or using at work (among a total of a dozen of computers). Now I don't want to loose my time anymore, I always use ECC memory when possible (I'm not going to pay a computer twice the price just for that, so it is a "little" difficult with laptops which also have a plethora of other choice criteria, but it is very easy to get affordable workstation desktop computers with ECC)

No modern digital communication bus will be designed without any form of protection, so this make not much sense to have computers without ECC RAM. I would even like to have it on smartphones, but unfortunately I doubt this will happen soon.

The fun thing is that things like rowhammer can cause errors in unrelated memory.

At least ECC turns a RCE into a DoS, even if it's not the best situation.

Nah you just flip three bits at a time instead of two.

But can you do that controlled and fast enough that the hardware doesn't notice? As far as I know, no-one has managed to successfully perform a rowhammer-based attack against a system with ECC ram.

What are the benefits for the average workstation user? What kinds of problems would I have on a non ECC development box?

I thought perhaps video editing and rendering would be a more likely area where ECC could be valuable to the non-scientific user. I have gone googling for examples of artifacting in videos caused by non-ECC ram errors, but I found nothing.

The tangible benefits, in 2017, for the "average workstation user" (if such a thing even exists) are vanishingly slim.

The sole benefit is correcting random bitflips in memory. These bitflips don't occur often enough per byte of RAM in workstation environments to matter: It's far more likely you'll have an issue with corruption due to poorly developed software than due to EMI.

However if you're building a home NAS on commodity hardware, and you're concerned about things like data integrity, ECC is definitely preferred.

Let's say the workstation packs 64GB of RAM, and it's used for number-crunching tasks which include jobs that take as much RAM as they can get.

How often do you believe these random bitflips could happen in a year of continuous use?

It depends, with perfect components maybe as rare as one bitflip a year, or maybe even less. However you can have all the spectrum with slightly defective components. There are some rates that are really unfortunate because you don't even detect them with a few hours of memtest, yet when you use your WS for lets say a few weeks every so often you have a random extra non-deterministic fun crash, like an ICE that disappear when you relaunch your build.

Unless you have some funny notion of how you want to spend your time, you don't want to risk to fall in that situation...

That's bullshit. You have no reason to care about data integrity in your NAS but not in your workstation. Plus "random" bitflips can have various causes, and except if you spend a weekend to a month memtesting your non-ECC ram you have no way to not know if you don't have a slightly defective chip. Also, if you wish to risk to trash your most important design files after a good afternoon of work, or miscompile a binary for extra fun, or maybe even an FPGA bitstream, well that's your problem and to each one their way to have fun, however on my side I rather not piment my life with that kind of crap.

The last thing you want if you are doing any kind of creative work is more or less random bitflips that happen very rarely but still more that lets say once in a year (YMMV). If your bitflips are too frequent you will soon notice and "fix" the problem (if you don't switch to ECC you actually fix nothing, but if you have a defective chip and switch to a good one at least you mitigate your issue)

For information the defect rate in consumer electronic is in the order of 1% (not specifically for RAM, I don't have any exact figure, but as a general order of magnitude figure it should be something like 1%). I know there are too much software bug here and there, but I don't want to add extra ultra-non-deterministic data destructing bugs just for fun.

I would actually prefer if the uncorrected memory exception were handled by the operating system.

I would far more prefer that the affected program(s) have a chance to react, or be killed as a subset of the system. If the error occurred in a filesystem context there may be other ways of correcting the issue (particularly if it's merely in read cache instead of write cache).

Obviously unhandeled exceptions should cascade until they are either contained or until the entire system halts.

Uncorrected memory exception are handled by the OS (at least on sane OSes?). I don't know why the author think otherwise.

a problem arises when the memory error occurs in the subsystem designed to handle memory errors....

I think that just hangs the whole machine, which is no worse than what happens when the OS does not handle uncorrectable errors.

I think we would actually be in undefined behavior's territory. If that demon is kind the machine hangs, if not it could start sending and endless stream of gibberish down the SATA bus.

Other kinds of faults are handled just fine. There's a fault handler, a double fault handler for when that has a problem, and triple fault is a reset.

There's no reason a fault while in the ECC error handler shouldn't have the same progression.

> Since we don't have our own particle accelerator to bombard the memory modules with in order to cause radiation-based errors

I really want to see someone get some radioisotopes and place them next to both ECC and non-ECC RAM (while forcing reads and writes to the affected memory) to see what sort of soft errors / SEUs happen.

The folks at Los Alamos have done that one better and actually measured GPUs placed in neutron beamlines:

https://www.cs.utexas.edu/users/skeckler/pubs/SELSE_2014_Rel...

http://users.nccs.gov/~vazhkuda/hpca.pdf (section 6)

Lovely!

It's a bit terrible that the author implies ZFS is more susceptible to bit errors because it scrubs data, and any errors will make it go haywire. As opposed to other systems like NTFS/ext4 which presumably cope fine with undetected but errors...

Reply on news.ycombinator.com