Shouldn’t they instead figure out how to run cooler and with less power in general? That’s where it seems everyone else is going…
But as the article points out, if 40% of your DC's power consumption is in cooling, then you'd be foolish not to target that slice.
Liquid and immersion cooling allows higher power density, which all things being equal (I know there's a lot of heavy lifting being done by this...) will be preferred. Why distribute your components over a rack if you could fit it into a single 4U board? Why distribute your components over an aisle if you could fit it into a rack?
https://www.google.com/search?channel=fs&client=ubuntu&q=chi...
No but seriously why don’t they build a 128-core atom server. That’s really all anybody wants. I don’t need the fastest most immersed cpu ever, just a bunch of decent ones at 30W or less.
30 watts is low power mobile and "edge" compute.
After M1 and Graviton, there's a big rush for making denser server CPUs, pretty much bifurcating the market. Much more powerful than old Atom cores, but more power-efficient than regular ones, and targeted mostly at hyperscalers. AMD should have a dense variant of Zen 4 out next year, going up to 128 cores and reducing per-core power significantly. Intel will likely have a competitor out in 2024, and AMD might hit 256 cores then, although rumors of things that far out aren't very specific. On the ARM side of things, NVIDIA is launching CPUs like this next year. Qualcomm bought Nuvia, which was working on similar server CPUs, but probably switched focus to PCs. There are some other oddballs like Tenstorrent as well.
One interesting part with pushing per-core power so low is that communication between cores starts to be a serious part of total power consumption.
They were an uncomfortable middle ground though, between normal CPUs and GPUs. My benchmarks showed that there wasn’t much of an advantage over 20-ish normal xeon cores (for my HPC workloads).
(Memory is a little fuzzy - that was 4-6 years ago).
https://www.intel.com/content/www/us/en/products/details/pro...
Though, unless if you 100% need X86, there is the Ampere Altra 128 core Cortex-N1 chip.
Part of that, I think, was lack of parallelism in applications: in order to fully take advantage of those cores, you need to have a nearly-embarrassingly-parallel problem. Otherwise, you're not going to get the performance that you'd expect (but you'll get power efficiency!).
The fact of the matter is, that's not "all anybody wants."
They keep designing chips that use less and less per core. Why not cram as many cores as possible into a single server? Lower density means more wasted material, and once you go below 2GHz you're not saving very much power any more.
Manufacturing yields
Not that it matters either way for this discussion? Whether it's "one server" in that 2U box, or "twenty servers" in that box, you're still shoving in lots of cores and lots of watts to get that density up.
Not sure we need that, except in niches. At scale you often want at least some efficiency, which is certainly not max TDP per core (because the best efficiency point is with lower frequencies and higher width, not the max freq you can achieve). So remains the question of large number of cores, but at some point the area of silicon also goes stupid high. And you can put multiple packages, without sacrificing overall system density too much, and without departing from simpler, and probably lower TCO pollution.
For small systems it depends, but you actually often have even more limited thermal budget, except again in niches if you are ready to tolerate the drawbacks (stupid power req it even becomes hard to have just a few machines on a basic electrical network in standard homes or offices, high noise under load, obviously high TDP so heating up a lot). But you have less space constraints so if you really want absurd systems you already can.
So do we really need to e.g. double or triple the (electrical/thermal) power density at scale? Do we need 2 kW chips? Do we need to sacrifice the efficiency now, and increase the nominal consumption now, instead of waiting just a few years for node improvements? (And I could even ask: do we really need that much increase of processing power, shouldn't we start to optimise for the total ecological cost instead? and I've not tried to do some prospective in that area but maybe this would mean slowing down the processing power growth...)
Will we ever be able to double or triple our (general purpose) compute per cpu any other way? Moore's law is essentially over. Node improvements aren't really happening outside of TSMC, which doesn't have enough manufacturing ability to supply everyone, and even then those node improvements are getting more and more incremental.
And regarding power consumption, I think we really need to be consuming more energy across most sectors of human activity. The most likely explanation for the "great stagnation" is that our energy consumption has basically flatlined since the 70s. It appears that on a civilizational scale, reaching greater levels of development and expression simply requires more Joules. If you disagree, I highly recommend the book Where is my Flying Car?
> computers are incomparibly more efficient
Yet to be clear the total world energy consumption for computers has increased, which is sad because we could certainly cope with 1/2 of the current total speed capacity but way more efficiency. Trying to get very high TDP chips and/or density is likely going in the other direction (but I could be wrong for the datacenter).
This future probably won't happen, but it should.
[0] https://whatisnuclear.com/blog/2020-10-28-nuclear-energy-is-...
[1] https://www.sciencedirect.com/science/article/abs/pii/S00945...
Linus Torvalds said that ARM needs to be widespread on the desktop, because you need a critical mass of developers targeting ARM. ARM server vendors don't want that critical mass, they want special deals with a handful of big companies which is obviously doomed to fail.