Hacker News Reader

Comment by ansible | Hacker News Reader

There's no argument that the Xeon server is going to be less power-efficient than the RP2 cluster.

Before you go out an order a bunch, do keep in mind that the RP2 cluster doesn't support ECC memory and other features often considered necessary in a server environment, like remote management. I'm pleased to see the A1100 does support ECC.

Still, it is an interesting test. I look forward to reading more about ARM servers. And I hope the Mill Computing guys will ship something eventually too.

View on HN

> ECC memory ... remote management ... often considered necessary in a server environment

Do note that an idiomatic Erlang cluster (where the distribution of the system is used to achieve fault-tolerance, rather than for higher throughput) doesn't strictly need ECC memory (though, obviously, it wouldn't hurt.)

Also, such a system should be plenty manageable on its own—insofar as even an Erlang unikernel makes for a relatively ops-friendly system. There's little "remote management" can do that can't be done from an Erlang remsh. (Erlang even offers easy ways to expose e.g. SNMP MIBs.)

If you're talking about being able to re-image the system into being a completely non-Erlang system remotely, that might be hard—but a cluster-of-inexpensive-low-power-nodes setup like this only really has this one niche use-case to begin with, so I'm not sure what else you'd want to change it into.

ECC can prevent program crashes, but it can also prevent computation of incorrect results. Even if your memory is basically reliable, cosmic rays still can flip bits. As RAM sizes go up and up, your chances of seeing a memory error increase.

My point, mainly, if we're talking about costs and power consumption, it isn't quite fair to compare server-grade hardware (which is more expensive than consumer-grade) with a RP2.

An idiomatic Erlang cluster will have great benefit from ECC memory. The problem is that bit-flips can yield the wrong results for a while before the system crashes. And you may not be able to tolerate these wrong results.

Stupid question -- why wouldn't you need ECC? What does Erlang do special?

I've found C programs occasionally returning incorrect results with non-ECC memory, as a memory location got corrupted. Does Erlang have some magic to avoid that?

> ECC memory...often considered necessary in a server environment

This is very much workload-specific. If you cycle your processes out of memory even once a day, the chances of seeing a random gamma ray burst is vanishingly small. Of course, I wouldn't host ZFS or a data store.

In my experience, ECC shows benefits more often when a DIMM starts to fail, not in protecting us from cosmic rays (though I don't think I was able to make a note of the last cosmic ray that flew by me).

I've had systems limp along reliably while throwing EDAC messages every few hours. With non-ECC memory, they would have crashed and burned.

Of course, if you're seeing EDAC messages, you'd be better advised to just shut down the metal and shift your workload to a different machine until you can replace the DIMMs in question.

Yeah, I don't know that people are doing that on a regular basis. I also think you are underestimating the amount of cosmic ray events that occur on a regular basis.

https://en.wikipedia.org/wiki/Cosmic_ray#Effect_on_electroni...

And the PI interconnects are much slower so if you cluster is doing any East <> West traffic its going to kill performance.

Reply on news.ycombinator.com