New breed of supercomputer aims for the two quintillion mark

https://www.top500.org/lists/top500/2021/06/

So 6k A100s at just under 10 TFlops of FP64 => 60 PFlops. I don't remember if they said what the interconnect was, as that can hold you back from achieving those 60-ish PFlops.

It's too bad NVIDIA's blog referred to the FP16 value as "flops" in the same breath, as most people at NVIDIA know that Top500 focuses on achieved FP64 LINPACK results. I swear there was some talk of using MLperf or similar to make a Top500-for-ML/reduced precision, but can't find it now.