Dell exec reveals Nvidia has a 1,000-watt GPU in the works

Yeah, there's this weird notion that OSS work on running LLMs on consumer hardware, and researching more efficient architectures, is somehow in opposition or competition to the Evil AI Wielding Corporations. It's not. Like always, whatever tool an individual has, a corporation can afford to use more of it, and use it better.

> This is like saying we shouldn't have any new CPUs after the the PDP-1 because quicksort got invented

I didn't assert anything here, I merely pondered. However, lets pretend that I did say that we shouldn't need heavy GPUs anymore. I don't think that's an entirely bad take because it would be fantastic if we could run larger models with more parameters on lower spec hardware. Especially phones that aren't flagship models.

If you find a way to extract compute from the work done by a compressor, sure. But then you'll be able to afford the heatpump and any number of GPUs, for the money you get from overturning laws of thermodynamics.

Considering that heat pumps have a coefficient of performance of greater than 1 (meaning they are moving more than 2.6 kw of heat), no. You'd probably need 3 to 5 GPUs to get comparable levels of warmth.

I mean, we have these, but changing to a bigger CDU just because you decided to get a rack of GPU servers is not always fun/easy/doable.

Even if you just upgrade them, then there's the primary loop's capacity.

I can't find it anymore, but I remember reading a few years ago about a watercooled server farm that used an outdoor water fountain. It wasn't a single rack, but a few 100s of them

There's a lot of work being put in to make LLMs run on consumer hardware, albeit it's being done by the open source community and not the companies themselves.

>"developers are expensive, hardware is cheap(ish)"

Is it even true?

H100 is worth 1 year of salary of mid engineer in eastern eu - 40k usd before tax

Yeah so they should increase the efficiency instead of the power.

As long as we insist on doing the same thing, we will never use less in total. Point me to any sizable consumption field where gains in efficiency didn't translate to more usage.

The best way to enjoy HN is to just ignore these comments. There are a lot of bored, angsty, frustrated, lonely people, many struggling with mental illness that like to make comments like this for no other reason than getting attention.

We'll blow ourselves up before climate and scarcity are solved. AI might help with this.

Depends what the externalities of that resource usage are, and who bares them.

Currently while NVidea shareholders reap the reward, it's people in the global south that suffer the immediate consequences (although the rest of us will eventually too)

Renewable energy does deplete a lot of natural resources. Even assuming it is carbon neutral (it isn't). Even assuming we're still capable of constructing those without fossil energy (we currently can't). Even assuming mining of various minerals is done without fossil energy (it isn't). Even assuming it doesn't require any mining (it does). It still largely requires "natural surfaces" where we have to disrupt local environments. It might be net benefits (offshore wind farms seem to improve diversity), but we have no renewable energy growth that doesn't require first for the natural resources to take a toll.

Don't get me wrong, I do think this is where we're heading towards, and I feel it's okay. But let's not be blind that yes even assuming we go towards fully renewable energy (still a big if, or rather very long when), any growth of energy consumption comes with a cost to natural resources.

There’s zero evidence for that.

Maybe. The increased competition and other side effects of increased resource usage also increase the chance of us nuking ourselves, so it's liable to be a close-run thing.