Hacker News Reader

Comment by brianolson | Hacker News Reader

> Why aren’t these AI companies submitting to the TOP500 to show off their computing prowess?

my knowledge is 10+ years out of date, but once upon a time if they'd chosen to, Google could have had _several_ entries in the top 10 of the TOP500 list

It's just poker, they didn't want to tip their hand

View on HN

I’ve worked on several systems that had enough flop/s to make it in the top 5-10, but for which we never submitted benchmarks. Sometimes their backend network layout technically would make them several smaller clusters for an HPL run, sometimes it’s because the cluster is too heterogeneous to get a good benchmark result, and sometimes it’s because the employer wants to keep a low profile.

Most of the time, it just that it’s a hassle. It takes a while to prep and tune a big hero run for benchmarking, and if you spend a billion dollars on a cluster, it’s making you a lot more than that. Taking it down for a day or two stops the money printers.

What programs were yours running to print money?

Kubernetes

Also, would those 550k Blackwell have good FP64 performance? How would one even compare them?

Yeah, that's true. GPU flops are really impressive in fp16, and more recently fp8 and fp4. When the 40xx GPUs came out, Tim Dettmers had a really cool blog looking into the numbers, and for fp8 one 4090 GPU had enough flops to match the best supercomputer somewhere in the 2000s, with an 8x 4090 build being top for a few years more. It's insane. But it gets nowhere close on fp64, where most of the physics simulations and other usual supercomputer tasks shine.

Other than weather simulations and nuclear explosions, what other supercomputer tasks are out there?

I was surprised to find out that material science uses up a significant amount of the compute time on the big research machines, at least here in the UK. It's both potentially lucrative research and computationally intensive. Combustion is the other big one that hasn't been mentioned but that's... Controversial.

Although probably underfunded compared to the two that you listed, cosmology is probably up there as well. Things like early universe and galaxy formation, megastructures, etc.

There's likely also some need for fusion plasma containment and other related simulations.

Based on the latest attempts to simulate (in LTSpice) Analog Devices or Texas Intruments ICs together with some common transistors, i would say that supercomputers might allow a simulation to converge in a finite amount of time.

Weather modeling? Physics?

Cloud computing is not a supercomputer. Different architecture, bandwitch, interconnectivity and latencies.

That's not nearly as true when you look at AI training clusters. They're basically supercomputers but without an FP64 focus.

(These are the systems to which GP was referring at Google.)

Even before AI training clusters became important, Google has had an outstanding custom fabric (there's papers about it) together with the ability to tune NICs for their own cases, and "their own cases" meant nearly everything engineered within Google. Ethernet hardware has had low kernel latency and DMA for a long time; it's the rest of the stack that hurts. But as far back as the early 2010s (if not further back, that goes beyond my knowledge horizon), you could just make it not hurt, if you had the software engineers to do it.

All that would not help you with an AI training cluster interconnect. See Amin Vahdat's keynote at HotInterconnects 2025. Everyone is building a fabric for this stuff from scratch (Google/Falcon, Amazon/EFA, Azure/MANA, Cornelis/CN5000, and obviously Mellanox).

I thought TPUs couldn't reasonably run LINPACK at all because TPUs do not acknowledge that FP64 exists.

I know Google wants to compare their stuff to El Capitan or whatever but the comparison does not seem valid to me.

Historically there have been a bunch of clusters on the Top 500 that weren't used for HPC. The tell is that they used Ethernet (this was before RoCE). It's less efficient but you can still get an OK Linpack score.

My sense is you only submit if you are in the business of selling supercomputing cluster (IBM, Cray). If you are a consumer or build to consume internally, you would care less.

It's also good for recruiting. They're building a new supercomputer in Edinburgh and I'd imagine it's a pull for certain researchers.

This! Hackers like their intellectually stimulating and interesting very expensive toys

Is there international value to these designations? As in, would it be worth it for the U.S. to pay a bonus to anyone who qualifies into the TOP500, to offset the cost of the run?

Most of the US systems in the TOP500 are funded by the US government. It isn’t considered a meaningful demonstration of capability by most people in the know.

I have joked more than once that my teams might unintentionally have wandered into top-500 territory a couple times in the last 10 yeats.

Why would the scientific computing people want to tip their hand? It’s an open secret that the main point of these mammoth FP64 compute machines is to simulate nuclear weapons detonations to comply with the CTBT you’d think that crowd would really not be fans of broadcasting their capabilities.

In adversarial scenarios, there are varying strategies in communicating one's capabilities, just as one might do in a poker game.

Sometimes you want to show off what you can do to dissuade others from fucking with you. Sometimes you want to undersell your capabilities to hide your true ability. Sometimes you want others to think you are underselling your capabilities when you are actually at a disadvantage.

Might also be smart to make the adversary think you are overselling when you actually have the capability.

It is partly this and partly a funding vehicle for American next-gen computing. It is not that hard to estimate FP64 ballpark from a whole bunch of public statistics. And it takes a looot more than raw FLOPs to get a simulation working. And presumably a looot more to translate it into practice. And the openness makes it easier to talk to different vendors and not get in the way of them having all the H1Bs it takes to get these things to work.

Plus one think I like to say is that if a bullet is flying towards you, you could know everything about the chemistry of the gunpowder and the composition of the alloy without it affecting what happens next.

What for? You only need to match the performance that existed in the 1950s. In the Soviet Union. Everything else is a lack of knowledge rather than computing power.

Also you should read the second sentence of the CTBT Wikipedia article to find out why it's not even in force (spoiler: US hasn't ratified it).

At some point, you will get diminishing returns no? I don't think compute is the bottleneck right now for mechanical engineering if you don't count AI.

Reply on news.ycombinator.com