Hacker News Reader

Comment by queuebert | Hacker News Reader

I wonder who buys these. Crypto miners? My institution would make it nearly impossible to buy a secondhand supercomputer.

View on HN

There's a whole sub-industry of people bidding on government auctions in order to part out the stuff. I'd be pretty surprised if the whole cluster got reassembled. But people on a budget will buy those compute nodes, someone trying to keep their legacy IB network will snap up those switches, the racks, etc.

r/homelab will have a field-day getting those nodes up, some people will want just one for practicality, some people will want at least a couple and a IB switch just for the novelty of it.

I can't imagine anyone from r/homelab has an SGI 8600 E-cell laying around that they could slap these blades into.

I can imagine it, some people on there are ridiculous, but yeah in my experience these supercomputer nodes are a lot more integrated/proprietary than most standard server hardware. It's not straightforward to just boot one up without all the support infrastructure. I'd assume they'd mostly be torn down and parted out.

You might be surprised - because they're pretty custom they are often "more open" than you might expect; as long as you have the connectors you can often get things running something. Sometimes they have bog-standard features present on the boards, just not enabled, etc.

It's the commoditized blade servers, etc that are stripped down to what they need to run and nothing more.

Oh I'm speaking from experience with the SGI supercomputer blades. They're pretty wacky, 4x independent, dual cpu boards per blade and all sorts of weird connectors and cooling and management interfaces. Custom, centralized liquid cooling that requires a separate dedicated cooling rack unit and heat exchanger, funky power delivery with 3 phase, odd networking topologies, highly integrated cluster management software to run them etc. I'm not sure if they have any sort of software locks on top of that, but I would bet they do and presumably NCAR wipes all of them so you likely won't have the software/licenses.

I dug up a link to some of the technical documentation https://irix7.com/techpubs/007-6399-001.pdf . Probably someone can get it working, but I imagine whoever is going to go through the hassle of buying this whole many-ton supercomputer is planning to just strip it down and sell the parts.

Yeah the licensing is often the stumbling block, unless you can just run some bog-standard linux on it. It sounds like this might be custom enough that it would be difficult (but I daresay we'll see a post in 5 years from someone getting part of it running after finding it on the side of the road).

Ultimately SGI was running Linux and AFAIK the actual hardware isn't using any secret sauce driver code, so yeah if you can get it powered on without it bursting in flames and get past the management locks you can probably get it working. It's definitely not impossible if you can somehow assemble the pieces.

>Crypto miners?

I think mining crypto with these would burn far too much energy compared to the ASICS in use.

Not every crypto can be mined with ASICs. For example, Monero is ASIC and GPU resistant.

Crypto is no longer mined commercially with GPU type compute. When ETH switched to PoS, it decimated the entire GPU mining industry. It is no longer profitable. The only people doing it now are hobbyists.

Some crypto is ASIC- and GPU-resistant.

Monero is one of them.

Sure, but you can get (much) better price-performance-power out of a CPU which isn't approaching a decade old, when mining ASIC and GPU-resistant cryptocurrencies like Manero. I don't know it'd be worth the effort to buy E5-2697-v4 CPUs which are running in such a specialized configuration over and above AMD Ryzen or EPYC CPUs in commodity, inexpensive mainboards.

Depends on the price this supercomputer sells at.

Monero is pegged to CPU.

I don't know why you downvoted me.

https://www.getmonero.org/resources/moneropedia/randomx.html

Recyclers.

any I've dealt with definitely wouldn't touch the 'you need to hire professional movers costing you $10k's of dollars to get it out of the facility' stipulation - they seem to prefer the 'where's the location of the storage shed' situation.

There's 8,064 E5-2697v4's in this. Those go on ebay for ~$50/ea. That's $400,000 of just CPUs to sell.

If the winning bid is $100k, you spend $40k to move it out of there, another $10k warehousing it while selling everything on ebay, and you're still up $250k on the processors alone.

> That's $400,000 of just CPUs to sell.

But do you crater the market for those CPUs? What's the demand for 2016-era Xeons and how much of their price comes down to supply?

I presume no one is building new motherboards for those processors either. While there is old stock laying around, you really need to run those systems close to as-is for them to be useful.

> I presume no one is building new motherboards for those processors either

That's actually far from the truth, LGA2011 is quite popular as a budged gaming system precicely because CPUs are so cheap on the 2nd hand market.

https://aliexpress.com/w/wholesale-X99.html

These are high spec cpus for the socket though. Lots of room for people with compatible boards that want to upgrade.

There's a lot of low budget hosting with old Xeon systems (I'm paying $30/month for a dual westmere system; but I've seen plenty of offers on newer gear); you can still do a lot with an 18 core Broadwell, if the power is cheap.

And how much labor costs to earn that $250K? Once that is factored in, I’m guessing fair price is zero or negative.

Plus knowing a bit about warehouse costs … your $10K is a bit on the low side don’t you think?

It's for the processors alone - a scrapping company dedicated to this stuff would be able to actualize more from other components - and they often have warehouse space available that they already own.

Let's come back and see if the auction failed; I doubt it will.

Academic departments with low budgets and cheap electricity who can make due with old CPUs

Reply on news.ycombinator.com