Building a 32-Thread Xeon Monster PC for Cheaper Than a Haswell-E Core I7 | Hacker News Reader

Building a 32-Thread Xeon Monster PC for Cheaper Than a Haswell-E Core I7

techspot.com

168 pointselbigbad40 comments

This completely overlooks the wonderful D-1541 (only 8 cores/16 threads) through D-1587 (16 cores/32 threads) series of embedded xeons that run at ~45W to ~65W compared to 115W of the E5-2670 (which this article says to run 2x of!).

http://ark.intel.com/products/93365/Intel-Xeon-Processor-D-1...

If you need just a ton of cores there are more efficient things out there! Not to mention that the D-1541 is available in Mini-ITX with 6 SATA 6.0GBps.

While I am not selling these things I have been drooling over them for a while, there are some great guides on tinkertry.com where they demonstrate this server: https://tinkertry.com/superserver-first-impressions.

A D-1587 setup like this won't cost you ~$1k in hardware.

It might save you $200 a year in electricity costs, but if it costs me +$2k, well, yeah. I'm not going to run this setup 10 years.

I am definitely more an outlier here. I am very happy to run machines 6 or 7 years as they move from primary system to network appliance. The other main feature I really like about the D-1541 and D-1587 is that they are mini ITX form factor (which fits in most ATX derivative cases) and it is small enough to easily transport from place to place, where the i7 is much too cumbersome to do. My goal is to have a system I am completely comfortable throwing in a backpack or the back of my car and move my entire VM infrastructure to the job at hand- A cloud in the pocket if you will.

Curiously the D-1587 doesn't show up if you look at

http://ark.intel.com/products/family/87041/Intel-Xeon-Proces...

but I see a very appealing core count/core frequency compromise:

http://ark.intel.com/products/93356/Intel-Xeon-Processor-D-1...

I built a workstation using 2 E5-2670s about a month ago. I'm pretty happy with it so far. I did need to repin to get the second EPS connector on my power supply, but my power supply wattage was overkill otherwise.

One nice thing is that DDR3 ECC RAM is extremely cheap. I bought 32GB of RAM for $60, and have since seen 64GB for $85.

Also, it will boost to 3GHz on all cores at once, as long as it stays cool enough.

If you truly run multi-threaded applications all the time, it's an amazing value right now - ~$70 for an 8-core CPU with a huge chunk of cache.

I did a similar build. I had no clue other people were thinking the same. The entire build I was super nervous. I kept worry I was buying components for a system that wouldn't be worth it. "How can this be so cheap?" I kept wondering. Admittedly, I went for used E5-2687Ws from ebay. Which increased the risk IMO.

My experience mirrors the article and your experience. Fantastic value for a really great workstation! I happily run several VMs for dev work, the usual IDEs and switch between games with zero issues. I picked Noctua coolers, same as the article, and the noise level is also excellent.

My motherboard, RAM, and processors were all used. I wasn't concerned too much about the RAM or processors, but I've seen a lot of posts about people getting bent CPU pins on used motherboards. Luckily, everything was working for me when I got it.

I was initially considering a new Xeon E3 system, because I wanted to have ECC memory, but the price on these used components had a significantly better price/performance ratio. Whenever I decide to update to a new system, I'll be looking at something like this again before I consider something new.

Which motherboard did you go with?

Ended up getting a used Supermicro X9DAi from eBay.

Edit: Should mention that Supermicro has proprietary ATX/E-ATX motherboards. I went for them because I thought it would just work with an E-ATX case, but I ended up drilling 5 or 6 holes in my case for the standoffs.

I went down a similar route last year. Bought a used 2009 Mac Pro, firmware upgrade to make it just like the 2012. Took out the two quad core, (and thanks to the firmware update) put in two x5675. Two GTX 970s (flashed to thermal limit of 150W) and 4x HGST 6TB. Fast PCIe SSD. Ubuntu 14.04 and ZFS.

It's a really interesting box to program concurrent and distributed algorithms on. Latency between 2 processors vs 1, HT, etc.

But the power consumption is bad. I want to switch to a new Xeon D if/when this system dies. Or maybe 4x ODROID 8 cores so I can play with networking.

I also did a slightly less ambitious build myself, my craigslisted Mac Pro only had the single CPU tray, so I have to be content with a single hex-core upgrade (and only a single GTX 750 Ti SC). 32GB of ECC is nice, but the lights flicker when it powers up, which makes the lower-power Xeons look appealing.

For those who want pre-built system, the first gen HP z620s use the E5-26 series of procs.

ebay is your friend here.

Lots of USB, really solid case and PSU.

Funny how prices are region-dependent. I just checked and E5-2670 retails for… wait for it — between $2000 and $2150 here (EU, or more precisely Poland).

No cheap workstation for me, I'm afraid.

That's new price. Here in .nl the first hit I get from a second hand site is e125 if you buy two. Double what they cost in the US it seems, but still.

It's the same with used Thinkpads, they are dirt cheap in the US too. I suppose European companies don't do four year leases on their hardware...

the only funny thing is your lack of googlefu

$130 http://allegro.pl/intel-xeon-e5-2670-8x2-6-ghz-lga-2011-sr0h...

$80 http://www.ebay.com/itm/100-OK-SR0H8-Intel-XEON-E5-2670-2-6-...

the keys here are these xeon cpus are "used" and their power consumption is also monstrous, compared to new-gen cpus (along with mb or other components)

also i'm always suspicious about the life span of these used cpus as cpu gets busted much more often than any other components (next to hd, my experience).

at the end i guess the total cost for such xeon monster pc might be much higher (including replacement, power consumption....) compared to new systems.

What are you doing to your CPUs? In my experience, CPUs are reliable enough to be nearly beyond reproach when troubleshooting a broken machine.

There are things such as electromigration that mean that solid-state stuff can wear out.

(Electrons are being pushed around by electric fields higher than 1 MV/m, and do smash into things ...)

I'm not complaining about my seven year old machines, but I'm not too sanguine about the current crop lasting as long ...

Indeed - and smaller feature sizes increase that risk.

Electromigration is a thing, but if you're applying your design rules correctly, it's one you worry about over decades, not months or years. I've had 30 year old ROMs/PROMs fail due to electromigration (presumedly, I didn't decap them and look), but never a CPU.

I used to believe that CPU should be the last culprit. One time for my faulty q6600 system i replaced everything except the CPU and finally realized the CPU was the problem. And it's not even overclocked.

Out of my experience of ~8 PC assembled (super small sample size), 2 CPUs kissed bye right after the 3-yrs warranty period, 1 CPU DOA. the good thing is that good guy Intel replaced DOA CPU very fast.

I guess there are lemons with everything. I'm sitting here still using my Q6600 workstation with no issues. I keep upgrading around it as it's still fast enough.

I thought so too, but recently upgraded to a broadwell i7 (4 cores, 8 threads) and compile times have fallen enormously.

I am an animator so my requirements are different. A video card makes all the difference for me for viewport rendering speeds. I don't render out that much, but I did notice that my wife's iMac (which is a few years newer than my q6600) renders out at about 2x the speed.

I assemble PCs for my friends, family and I since 20 years, we have something like 10 servers and 30 desktops (some are more than 8 years old) at work and I never had a CPU die on me.

You are really unlucky imho.

I have hardly, if ever, seen a CPU die, but I agree on the point of power. I have a ~2007 MacPro (2 Quad-core Xeon CPUs, 16 GB RAM) I was lucky enough to buy off a friend for cheap.

Performance-wise, if the workload can be spread across all the cores, it is ... not too slow. ;-) But I can tell if that thing is on or not by the room temperature, because that thing gets hot when it is busy. It gets hot, in fact, when it is idle, too, so I mostly keep it asleep and only wake it up for short sprints of activity, like encoding a video or something.

Well, actually, the MacPro doesn't get hot itself, because it has lots and lots of fans, but the rest of the room does.

So it is nice to own such a powerful machine, very, very nice, but, yes, more recent machines tend to get more efficient, evergy-wise.

Never seen a cpu die? It looks awesome https://en.wikipedia.org/wiki/Die_(integrated_circuit)#/medi...

I a know what a die is and what it looks like. ;-) Athlon XP CPUs used to have a naked die, and one could kill them when mounting a heatspreader - a friend of mine did, once.

Just in case I was being unclear: I meant the verb "die", as in "to give up the ghost". More likely, though, you couldn't resist the pun... ;-)

Niiice!

I wonder how it behaves in games. Did anyone try it yet?

Also, the Xeon E5-2670 is still priced at 1900+ euros in Europe at the moment...

Games benchmark is on page 7, does pretty okay.

Then you should probably import one -- start with the link in the article. No way the duties will be anywhere near that much.

IIRC Linus Tech Tips does take a look at a similar system for video rendering. It's probably not going to do super well in games because the bottleneck is usually at the GPU anyway.

There are benchmarks on page 7. To summarize, it's slower than a 4690K in all cases, and running it in 2-CPU mode is slower than 1-CPU mode in all cases. Still beats AMD's high-end FX-8350 processor in all cases, amusingly enough.

That's pretty much what you'd expect. Gaming performance is strongly correlated with single-threaded performance. Many games are essentially single-threaded and even when they're multi-threaded they are still subject to Amdahl's Law. Certain parts of games cannot be multithreaded (eg all draw-calls must be made from a single thread in all APIs prior to DX12 and Vulkan) and they tend to limit overall performance.

So with narrow exceptions, single-thread performance is king. The best example of this is the G3258 - it's not really that powerful a processor, but it can be overclocked to really extreme levels and keep up in many games just through sheer single-threaded performance.

Going across multiple processors is also particularly brutal because of greatly increased thread and cache synchronization costs. So again, not really a surprise that it's actually slower.

If you need more than a 4790K or 6700K can deliver, your next stop is the 5820K or other -E series processors. But not many games really need or can make good advantage of more than 4 threads, and even fewer need more than 8.

> Still beats AMD's high-end FX-8350 processor in all cases, amusingly enough.

A brand-new AMD FX-8350 costs around $160, while a Intel Xeon e5 2670 is sold for $1560 (newegg prices)

The Xeon isn't 10x the processor that the AMD FX is. Not by a long shot.

Yes, while workstation chips are insanely expensive (from a consumer perspective) they do often underperform enthusiast chips like the Core i5 series.

The point here is that the FX-8350 does not. An older workstation-class chip outperforms it pretty handily. And once we move into consumer-class chips, its Ivy Bridge class equivalents totally outclass it, as do even its Sandy Bridge class predecessors. The Bulldozer architecture is absurdly bad for gaming usage, and should never be recommended, full stop. Single-threaded performance pushes frames, not multi-core performance.

The AMD equivalent is likely several times as much as the FX processor as well. Not quite as much as the Intel, certainly, but not $160.

Or you could buy an entire system for 700 euros?

http://www.ebay.com/itm/HP-Z420-Workstation-E5-2670-Xeon-RAM...

cpu is $60, but mobo will kill you :(

But can it run Crysis?

Reply on news.ycombinator.com