Hacker News Reader

Comment by gruez | Hacker News Reader

> That we cannot get laptops or even desk machines with ECC, and so have them crash frequently, is seen as a trivial side effect of the strategy

I’m not sure what you mean by “frequently”, but my non-ECC machines definitely do not crash “frequently”.

> before MS, a program crashing was grounds for a refund

Source?

View on HN

The problem with untrusworthy memory (or any other component) is not that your system crashes, it's that it doesn't.

I don't doubt that non-ECC hardware experiences some non-zero number of bitflips per year. I'm just doubting the parent commenter's claim that non-ECC ram is causing computers to crash "frequently".

And the parent is pointing out that not crashing on bit flips is exactly the problem.

Actually crashing on bit flips is a second problem, and an indicator of the first problem.

Crashing is "not the problem" because crashing prevents the far worse problem of incorrect data or operation that you don't know is incorrect.

The difference in significance is so great that by comparison a mere crash is no problem at all.

In fact you design systems with hair triggers to 'crash' on purpose as readily as possible. Trying desperately to crash at all times every millisecond all day every day.

IE, halting all operation of some subsystem, or the whole thing, the second a single bit wrong is detected. Better to kill the hd or the entire machine than let it keep running one second after getting any hint it might not be 100% trustworthy.

'crash' in quotes because really all you want to do is halt, and you're doing it on purpose, but that is still a crash, in the sense that your application does not want to halt and it isn't necessarily halted gracefully with any chance to conclude anything or save anything. Those are just more operations you can no longer trust to be correct, and so should not be allowed to do.

Frequently, for me, means at all. Computers are designed as deterministic state machines. Any unintended indeterminism is a failure.

Microsoft, uniquely exercising monopoly power, was able to institute a "no refunds, nohow" policy and make it stick: Windows95 machines commonly crashed several times a day, enough so that "crash" came to mean, instead, you have to reinstall the OS, which happened at least one per several months.

And here we are.

I guess you can use the term however you want, but FWIW if you say "frequently" and mean "at all", it makes it a lot harder to understand the point you're trying to make.

If it happens to me, it is happening to untold others, too. Thus, frequently.

"Infrequently" would mean I had not heard of it, or that I heard only of isolated instances, well publicized.

I feel like I just fell into a bucket of logical fallacies.

We all have no choice but to sample imperfect information. You can no more than pretend better.

Horrifying, can you imagine an alternate reality where we have to actually write working software? Microsoft, for once, did a Good Thing in a way.

I see a ton of replies here. I am a tech user with thousands of terabytes stored locally. Most of those terabytes are mirrored in the cloud. In all of those situations I've never actually had any corruption related to memory or storage. I have 932gb in PICTURES ffs.

I get the claimed need for EEC, but yet I don't. I own a number of web properties and regularly update logos, software packages, html files, PHP files, ruby files, etc. No corruption.

How do you confirm that there's no corruption? My understanding was a silent bit flip would be difficult to detect. I ended up using ZFS and ECC memory to avoid silent corruption, but I'm largely relying on the motherboard to do the right thing when an ECC error is detected. It'd be nice to verify after the fact.

bit flip hard to detect - true for RAW, false for .jpg or other compressed files.

git fsck

On 932 GB of binary data? I'm probably operating on old impressions of git, but that sounds like it'd be a nightmare to work with. Ideally, I'd like something that runs in a reasonable amount of time so it can be run on a regular schedule. I currently run a ZFS scrub that takes about 90 hours to finish on 32 TB of data, so it's only run monthly.

If your pictures are not RAW but compressed (say jpgs) then the corrupted photos will really jump out at you - half the picture is there, but then noise. I used to make sure I was storing jpgs not RAW, but then learned that wasn't always a good thing, several times over. Dunno whether damage happened in RAM or on HD, but I strongly suspect RAM.

The bad habit that nailed me was probably this one: I would ball up a whole bunch of photos in a big .zip; (to enhance copying speed and maybe privacy) then next year unzip that, add that year's selected photos and zip it again. Year after year. That added up to a lot of trips through RAM in which one bit flip could take out an image. Bugs in compression software aren't impossible as the villan, but that's not my bet.

Solution: I now zip up a few years together and copy each chunk I don't ever rezip old large chunks. No new errors noticed.

You can be pretty sure that you have some corrupt bits

No, we really can't be sure. I'm annoyed when people are sure about it without posting their ECC correction rate stats from live systems.

I'm continuously writing metrics on a non-ECC system. There are 3 places that the bitflip can affect the data: pre-writing (checksum will be correct), when flushing to the drive (checksum mismatch), calculating the checksum (mismatch again). There may be some silent corruption (in 1 out of 3 cases), but I've been scrubbing the drive every day for ages and there's not a single error.

Non-ECC seems to be doing well. Other claims could really use actual numbers.

"frequently" is very subjective or relative in this context. 25 years ago I had a crash per hour on almost any regular computer, but zero crashes per month on servers with ECC. In the past couple of years I think I had a few cases of frozen apps, but I don't remember of any OS level problem. At the same time, on servers I see from time to time ECC fixing a bit, but on the desktop or laptop I have no idea how many times corrupted bits went undetected and what is the consequence.

>25 years ago I had a crash per hour on almost any regular computer, but zero crashes per month on servers with ECC

If it's crashing once per hour, it's probably unstable drivers/software or flaky hardware that needs to be RMAed, not random bitflips.

I think sometime else is wrong if you've had a "crash" per hour.

Reply on news.ycombinator.com