3M Novec 7100 Fluid Used in Largest Two-Phase Immersion Cooling Project (2015) | Hacker News Reader

3M Novec 7100 Fluid Used in Largest Two-Phase Immersion Cooling Project (2015)

investors.3m.com

33 pointsdeegles13 comments

I did some research with Novec back in high school, really interesting stuff. It was a hassle to get my hands on, but after going through a chemical supplier that actually flew out a staff chemist to ensure my high school had proper facilities to handle the vapors from Novec I purchased 1 gallon haha. Here's a video summary of my project, essentially just comparing efficiency differences between immersion cooling and air cooling with server hardware. https://vimeo.com/37127378 (Yes, I know the "efficiency savings" numbers at the end are incorrect)

One of the coolest attributes of Novec 7000 (now outdated haha) is that it's viscosity is so low near it's boiling point that it can actually slip through teflon gaskets and even dissolve thermal paste between a CPU and heatsink.

Not sure if Zack Glander is still running Vapor Phaze, but back in 2011 his startup was one of the only private companies I knew of developing hardware to use this cooling tech at scale. He had some cool demos at CES for a few years.

Wow, 250kW per rack is pretty much unheard-of. The highest density stuff I know of is 40-50kW per rack @ Ebay. They use 480v 3ph PDUs!

Boiling, like they do here, can remove a ridiculous amount of heat. Witness your ordinary electric kettle dissipate 2kW into a single 20 sq.in. surface, which it can do indefinitely without damage if you just hold the "on" button down.

I have wanted to get my hands on this stuff as a hobbyist computer Builder since about 2015 it's a shame because I live not even a few miles away from 3M Oakdale where the fellow Phil Tuma makes videos of delidded CPUs cranking away submerged in this stuff.

A few years back I was able to purchase through a commercial chemical distributor. It's hard to find in 1 gallon glass containers which on their own sell for $300 each. I know hobbyists on bitcoin forums have managed to purchase without an academic institution to back them.

3M and Gigabyte had a demo of an immersion cooled server this year at CES: http://www.anandtech.com/show/11046/gigabyte-server-shows-tw...

The 3M guy at the booth said they were partnering with Gigabyte because the latter can build servers with the requisite non-standard high-density form factors necessary to not have excess empty space being filled by the expensive cooling fluid.

So cool to see the "expanded copper" heat spreaders on these servers. In many cases those heat spreaders are fused to the CPU with indium solder. Cool stuff.

So nobody here is surprised this massive cooling infrastructure is getting built for custom ASICs for blockchain computation?

Allied Control was recently acquired by BitFury Group, a leading Bitcoin Blockchain infrastructure provider and transaction processing company, which builds its own fully custom Application Specific Integrated Circuits (ASICs), PCBs, servers and data centers.

Novec fluid [is utilized] in the Georgia data center to deploy 28 nm and 16 nm ASICs more efficiently.

“250 KW per rack"

I sort of realize this isn't being done on people's basement GPU rigs these days but with the drama with the ASIC vendors a bit ago I thought the bitcoin hardware market crashed.

I guess only on the hobbyist end.

Similar approach: http://www.grcooling.com

2015.

Are there updates on the project results?

everyone realized its impractical to drain a rack to replace hdd and psu every day and moved on.

You can pull SATA cables 1m, and PSU cables much longer. Upgrade to SAS, and you can do 8m cable runs.

So why not build your system such that only the parts that a) produce lots of heat, and b) break rarely, are immersed? I.e. just the CPUs, motherboard, memory, GPUs and ASICs? Sounds like the logical thing to do.

This is what I came here to ask. Seems a bit silly to fill the whole tower case with cooling fluid. My guess is that the incremental benefit you get from the redesign doesn't outweigh the costs. It may at very large scale, and I would guess that Amazon, Facebook and Google have all experimented with it.

Reply on news.ycombinator.com