AWS building 414 exaflops AI computer exclusively for Nvidia's internal research | Hacker News Reader

AWS building 414 exaflops AI computer exclusively for Nvidia's internal research

aboutamazon.com

19 pointsbelter7 comments

This is a water-cooled system, so AWS is going to have to figure out a new datacenter architecture to install and run it. It’s a big investment and commitment from AWS and I expect that part of the deal was that NVIDIA would be a guaranteed customer at scale for the system, in return for things like AWS getting preferential early access to the technology.

Based on the 4x 66kW (albeit an A+B design) power shelves in these, this # of systems is basically an entire datacenter for AWS assuming they stuck to ~35MW buildings as they’ve done historically. It will likely be a little larger even than a normal building…

It’s only 288 racks of GPUs, based on 72/rack as NVIDIA disclosed elsewhere. Then whatever ancillary network is required. Pretty wild to use that much power over that few racks.

Seems like critical load when at 100% for the GPU racks is supposed to be 34.5MW and assuming 10% for 800GE networking to plumb it all together and a 1.2 PUE gets you to 45MW overall?

I wonder what is NVidia's play here. Do they want to build their own models and compete with the likes of MS and GOOG?

Elsewhere the CEO was saying that NVidia wants to build it's own platform (whatever that means), so I'd have expected they'd build an SC like this on their own.

My guess is that by having AWS build it, they have a deal to then allow it to be used in the spot market when idle, something they wouldn’t be able to do if they ran the platform.

Spot usage could make a decent profit in addition to training anything they want.

Healthcare is one of the last places I think about for GenAI but maybe I have that totally wrong. Seemingly factual accuracy is a premium in that industry. I guess protein generation is a little different.

I still don't know if you guys just don't use the new chatbots, or you don't have a speciality where you can see what I'm describing. But if you have a peculiar question and you ask it, as someone experienced in the field, you will know if the answer passes the sniff test or not. You either save yourself 15 minutes dredging up the correct answer or you do it the hard way anyway.

I had the exact same reaction.

Some cursory searching shows that it's being pitched for things like patient engagement to improve patient satisfaction, as well as reducing staff burnout.

Reply on news.ycombinator.com