El Capitan: New supercomputer is the fastest
spectrum.ieee.org
spectrum.ieee.org
> El Capitan uses AMD’s MI300a chip, dubbed an accelerated processing unit, which combines a CPU and GPU in one package. In total, the system boasts 44,544 MI300As, connected together by HPE’s Slingshot interconnects.
Seems like a nice win for AMD.
Yep! They've been part of the Exascale project for a long time, and it's good to see their commitment on HPC actually succeeded unlike Intel's during the same time period.
The first sentence statement in the article mentioning that United States and other nuclear powers committed to the Comprehensive Nuclear-Test-Ban Treaty in 1965 is wrong since the treaty was only signed in 1996 not in 1965 [2].
[1] The Algorithm That Almost Stopped The Development Of Nuclear Weapons:
https://www.iflscience.com/the-algorithm-that-almost-stopped...
[2] The Comprehensive Nuclear-Test-Ban Treaty:
Said sensors can also track sonic booms from secret supersonic planes, but governments don't like to talk about that.
Precursors of FFT have existed in various mathematical works, starting with Gauss, but such algorithms were neglected before the existence of automatic computers that could apply them to problems big enough for the dependence of the solving time on the size of the problem to matter.
In comparison, the iPhone 15 Pro Max cellphone, released in 2023, delivers approximately 2150 gigaflops.
I once drew the chart backwards. I think my PC in 2013 would have been the fastest on Earth in 1990. And faster than every computer combined in about 1982.[0]
[0] might not be accurate
The Gflop/s quoted in the past for supercomputers and now for big HPC clusters are FP64 Gflop/s.
The difference in cost between FP64 and FP32 is currently extremely high, much higher than it has ever been in the history of computer technology.
The reason is not technical, but of market segmentation, because NVIDIA, then also AMD, have removed the ability to do FP64 operations at a competitive rate from their "consumer" GPUs, so the best that one can get in performance per dollar is AMD 9950X (which can approach 2 FP64 Tflop/s per CPU, i.e. 256 FP64 FMA per clock cycle multiplied by around 4 GHz, with 2 flops per FMA).
It looks good, but I miss the blinking lights of the olden days. Of course, they don't make sense on a supercomputer that's not designed to be seen, but they are still cool.
Are they trying to model every single atom?
Is this a case where the physicists in charge get away with programming the most inefficient models possible and then the administration simply replies "oh I guess we'll need a bigger supercomputer"
While that's true, the information that is online is surprisingly detailed.
For example, this series "Nuclear 101: How Nuclear Bombs Work"
The alternative is to literally build and detonate a bomb to get empirical data on given design, which might have problems with replicability (important when applying the results to rest of the stockpile) or how exact the data is.
And remember that there is more than one user of every supercomputer deployed at such labs, whether it be multiple "paying" jobs like research simulations, smaller jobs run to educate, test, and optimize before running full scale work, etc.
AFAIK for considerable amount of time, supercomputers run more than one job at a time, too.
So in order to verify that the weapons are still useful and won't fail in random ways, you have to test them.
Which either involves actually exploding them (banned by various treaties that have enough weight that even USA doesn't break them), or numerical simulations.
* It's a jobs program to avoid the knowledge loss created by the end of the cold war. The US government poured a lot of money into recreating the institutional knowledge needed to build weapons (e.g. materials like FOGBANK) and it's preferred to maintain that knowledge by having people work on nuclear programs that aren't quite so objectionable as weapon design.
* It helps you better understand the existing weapons stockpiles and how they're aging.
* It's an obvious demonstration of your capabilities and funding for deterrence purposes.
* It's political posturing to have a big supercomputer and the DoE is one of the few agencies with both the means and the motivation to do so publicly. This has supposedly been a major motivator for the Chinese supercomputers.
There's all sorts of minor ancillary benefits that come out of these efforts too.
There are two kinds of targeting that can be employed in a nuclear war: counterforce and countervalue. Counterforce is targeting enemy military installations, and especially enemy nuclear installations. Countervalue is targeting civilian targets like cities and infrastructure. In an all out nuclear war counterforce targets are saturated with nuclear weapons, with each target receiving multiple strikes to hedge against the risks of weapon failure, weapon interception, and general target survival due to being in a fortified underground positions. Any weapons that are not needed for counterforce saturation strike countervalue targets. It turns out that having a yield greater than a megaton is basically just overkill for both counterforce and countervalue. If you're striking an underground military target (like a missile silo) protected by air defenses, your odds of destroying that target are higher if you use three one megaton yield weapons than if you use a single 20 megaton yield weapon. If you're striking a countervalue target, the devastation caused by a single nuclear detonation will be catastrophic enough to make optimizing for maximum damage pointless.
Thus, weapons designers started to optimize for things other than yield. Safety is a big one, an American nuclear weapon going off on US soil would have far reaching political effects and would likely cause the president to resign. Weapons must fail safely when the bomber carrying them bursts into flames on the tarmac, or when the rail carrying the bomb breaks unexpectedly. They must be resilient against both operator error and malicious sabotage. Oh, and none of these safety considerations are allowed to get in the way of the weapon detonating when it is supposed to. This is really hard to get right!
Another consideration is cost. Nuclear weapons are expensive to make, so a design that can get a high yield out of a small amount of fissile material is preferred. Maintenance, and the cost of maintenance, is also relevant. Will the weapon still work in 30 years, and how much money is required to ensure that?
The final consideration is flexibility and effectiveness. Using a megaton yield weapon on the battlefield to destroy enemy troop concentrations is not a viable tactic because your own troops would likely get caught in the strike. But lower yield weapons suitable for battlefield use (often referred to as tactical nuclear weapons) aren't useful for striking counterforce targets like missile silos. Thus, modern weapon designs are variable yield. The B83 mentioned above can be configured to detonate with a yield in the low kilotons, or up to 1.2 megatons. Thus a single B83 weapon in the US arsenal can cover multiple continencies, making it cheaper and more effective than maintaining a larger arsenal of single yield weapons. This is in addition to special purpose weapons designed to penetrate underground bunkers or destroy satellites via EMP, which have their own design considerations.
I've seen speculation that Russia's (former Soviet) nuclear weapons are so old and poorly maintained that they probably wouldn't work. Not that anyone wants to find out.
Less need of 9 megatons against a hardened silo if you have a 1.2 megaton weapon with a 120m CEP.
Citation needed.
1 gram of Uranium 235 contains 2e21 atoms, which would take 15 minutes for this supercomputer to count.
"nuclear bomb simulations" do not need to simulate every atom.
I speculate that there will be some simulations at the subatomic scale, and they will be used to inform other simulations of larger quantities at lower resolutions.
https://www.wolframalpha.com/input?i=atoms+in+1+gram+of+uran...
I would like a citation for this.
> Hypersonics do however end up dealing with simulating subatomic particle behaviours
And this.
---
For example, you could choose to cite "A Study on Plasma Formation on Hypersonic Vehicles using Computational Fluid Dynamics" DOI: 10.13009/EUCASS2023-492 Aerospace Europe Conference 2023 – 10ᵀᴴ EUCASS – 9ᵀᴴ CEAS
At sub-orbital altitudes, air can be modelled as a continuous flow governed by the Navier-Stokes equations for a multicomponent gas mixture. At hypersonic speeds, however, this physical model must account for various non-equilibrium phenomena, including vibrational and electronic energy relaxation, dissociation and ionization.
(younger generations are worse at it, because the problems that forced elder ones into more complex approaches can now be an overnight job on their laptop in ANSYS CFX)
So unfortunately my only source on that is bitching of post-docs and professors, with and without tenure (or rather its equivalent here), at premier such institutions in Poland.
Given all nuclear physics happens inside atoms, I'd hope they're being more precise.
Note that a frontier of fusion physics is characterising plasma flows. So even at the atom-by-atom level, we're nowhere close to a solved problem.
What are you basing this on? Plasmas don't flow like gases even absent a magnetic field. They're self interacting, even in supersonic modes. This is like saying you can just model gases like liquids when trying to describe a plane--they're different states of matter.
I wrote a previous HN comment explaining this:
Tl;dr - Monte Carlo Simulations are hard and the NPT prevents live testing similar to Bikini Atoll or Semipalatinsk-21
Modelling a single nucleus, even one much lighter weight than uranium, is a captital-H Hard Problem involving many subject matter experts and a lot of optimisation work far beyond 'just throw it on a GPU'. Quantum systems get non-tractable without very clever approximations and a lot of compute very quickly, and quantum chromodynamics is by far the worst at this. Look up lattice QCD for a relevant keyword.
So it involves very small time scales, chemistry, fission, fusion, creating and channeling plasmas, high neutron fluxes, extremely high pressures, and of course the exponential release of amazing amounts of energy as matter is literally converted to energy and temperatures exceeding those in the sun.
Then add to all of that is the reality of aging. Explosives can degrade, the structure can weaken (age and radiation), radioactive materials have half lives, etc. What should the replacement rate be? What kind of maintenance would lengthen the useful lives of the weapons? What fraction of the arsenal should work at any given time? How will vibration during delivery impact the above?
Seems like plenty to keep a supercomputer busy.
I'd assume computing atomic behavior at 0K is a lot simpler than at 800,000,000K, over the same time step. ;)
A while ago there were a few labs in China in top 10 and they all attracted sanctions / bad attention. Now no Chinese lab report any data now
(after the nuclear test ban treaty, they run a LOT of simulations)
The NNSA—which oversees Lawrence Livermore as well as Los Alamos National Laboratory and Sandia National Laboratories—plans to use El Capitan to “model and predict nuclear weapon performance, aging effects, and safety,”
The basis for the ranking was a cumulative tracking of benchmark results that were required as part of commissioning bespoke computers. A contract would be written to buy a computer that could achieve a certain performance in operations per second, and in order to satisfy that the benchmarks were agreed to and codified in the contracts. Government contracts are to a certain extent public information so the goals and clout of successive performance were tracked in this way.
If you don’t need to satisfy a government contract, or don’t need the clout to attract engineers or funding, submitting results draws unwanted attention to what you’re cooking up.
Keep in mind the average hyperscalers cloud is not a particularly good setup for the top500. HPC tends towards more bandwidth, lower latency, and no virtualization.
The experimental fusion approach used by the NIF [1][2].
It's conveniently simultaneously an approach to fusion power, a way to study fusion plasmas and a tiny nuclear explosion.
[1] https://en.wikipedia.org/wiki/Inertial_confinement_fusion
[2] https://en.wikipedia.org/wiki/National_Ignition_Facility
I wonder what would happen if Apple offered people something like iCloud+ in exchange for using their idle M4 compute at night time for a distributed super computer.
The interconnect has been Cray's bread and butter for multiple decades: Slingshot, Dragonfly, Aries, Gemini, SeaStar, numalink via sgi, etc. and those for the less massively parallel systems before those.
I believe 306 of the top 500 clusters used Infiniband. Pretty sure the advance topologies like dragonfly are supported on IB as well as Slingshot. From what I can tell slingshot is much like ultra ethernet, trying to take the best of IB and ethernet and making a new standard. From what I can tell slingshot 11 latency is much like I got with omnipath/pathscale way back when dual core opterons were the cutting edge.
There are a lot of intricates, but at a high level they require different compute approaches.
Most parallel research, especially at this scale is more about different balance of operations to memory bandwidth, and much more worried about interconnect latency.
I wouldn't assume that just because various corporations have large training clusters that they could dominate HPC if they wanted to. Hyperscalers have dominated throughput for many years now, but HPC is a different beast.
What are you basing this on?
> it's not like more deadly nuclear weapons would change anything
We haven't been chasing yield in nuclear weapons since the 60s.
Our oldest warheads date from the 60s [1]. For obvious reasons, the experimental track record on half-century old pits is scarce. We don't know if novel physics or chemistry is going on in there, and we don't want to be the second ones to find out.
I can safely say that nuclear simulations are one of the major drivers for HPC research globally.
It is not the only one (genomics, simulations, fundamental research are also major drivers) but it is a fairly prominent one.
Think about a baseball-size device able to take out a city block.
Then think about an escadron of drones able to transport those baseballs to very precise city blocks...
If you are afraid of nuclear war, the thing to fear is a nuclear state's capacity to retaliate being questioned. These supercomputers are the alternative to live tests. Taking them away doesn't poof nuclear weapons, it means you are left with a half-assed deterrent or must resume live tests.
> the abandonment of the course of nuclear disarmament treaty
North Korea, the American interventions in the Middle East and Ukraine set the precedent that nuclear sovereignty is in a separate category from the treaty-enforced kind. Non-proliferation won't be made or broken on the back of aging, degrading weapons.
> repeated talk of a coming war against certain Asian powers
One invites war by refusing to prepare for it.
That's why nuclear capabilities and capacities are best reduced by universal compact. It is certainly not helping when they are being _enhanced.
Also, I'm worried about the US initiating more than I do about it retaliating.
> North Korea, the American interventions in the Middle East and Ukraine set the precedent that nuclear sovereignty is in a separate category from the treaty-enforced kind.
I don't understand this sentence because I'm not familiar with the combined term "nuclear sovereignty".
Regardless - it is certainly the case that non-proliferation won't be made on the back of aging and degrading weapons; there must a continued commitment to NP in the sense of not developing new weapons.
> One invites war by refusing to prepare for it.
https://en.wikipedia.org/wiki/Begging_the_question
Anyway, the US initiates most of its wars, so the "invitation" is irrelevant.
https://www.techtarget.com/searchdatacenter/news/252468294/C...
> "The Russians are fielding brand new nuclear weapons and bombs," said Lisa Gordon-Hagerty, undersecretary for nuclear security at the DOE. She said "a very large portion of their military is focused on their nuclear weapons complex."
> It's the same for China, which is building new nuclear weapons, Gordon-Hagerty said, "as opposed to the United States, where we are not fielding or designing new nuclear weapons. We are actually extending the life of our current nuclear weapons systems." She made the remarks yesterday in a webcast press conference.
> ...
> Businesses use 3D simulation to design and test new products in high performance computing. That is not a unique capability. But nuclear weapon development, particularly when it involves maintaining older weapons, is extraordinarily complex, Goldstein said.
> The DOE is redesigning both the warhead and nuclear delivery system, which requires researchers to simulate the interaction between the physics of the nuclear system and the engineering features of the delivery system, Goldstein said. He characterized the interaction as a new kind of problem for researchers and said 2D development doesn't go far enough. "We simply can't rely on two-dimensional simulations -- 3D is required," he said.
> Nuclear weapons require investigation of physics and chemistry problems in a multidimensional space, Goldstein said. The work is a very complex statistical problem, and Cray's El Capitan system, which can couple this computation with machine learning, is ideally suited for it, he said.
---
This isn't designing new ones. Or blowing things up ( https://www.reuters.com/article/us-usa-china-nuclear/china-m... ) to see if they still work. It is simulating them to have the confidence that they still work - and that the adversaries of the US know that the scientists are confident that they still work without having to blow things up.
The Armageddon scenario is some nuclear states conduct stockpile stewardship, some don’t, and those who do discover that warheads come with a use-by date.