Hacker News Reader

Comment by jl6 | Hacker News Reader

If you are comparing the energy cost of LLMs, consider that the "training cost" of the human brain is the accumulated evolutionary activity of the last 4 billion years.

Human inference is much more efficient though - we run at about 100 Watts[0], probably a lot less for the thinking alone.

[0] https://physics.stackexchange.com/questions/431522/humans-ha...

View on HN

Remember, though that LLMs also need the accumulated evolutionary activity. They're not that interesting if you don't feed them human created content. Contrast that with the human computer that builds itself and then infers language and complicated physical and mental tasks.

Not really. We had millions (and now trillions) of copies of brain feeding each other.

So who trained the first brain and what was the costs?

Big bang. Imagine the energy.

Given I can do token generation on a 30 watt laptop, the language/Joule part at least seems to be the computer winning. (Separate to any discussion about quality).

Not so for images, as I can imagine visual scenes in what feels like real time, while even the fastest image generator I've seen was 200-300 ms at 512x512.

You're not really imagining visual scenes, the resolution just isn't there if you think about it. It's more like you're imagining low resolution encodings, which is definitely the smart and efficient way of doing it that evolution would choose over a wasteful full sim.

Encodings perhaps, but I assure you the subjective experience of the resolution is indistinguishable from my eyeballs.

It sounds like you're subjective experience of visualization is wildly different than the average, then. I think I have an excellent imagination and visualization ability, but it doesn't compare to just looking out my eyeballs in terms of resolution or cognitive load.

Look up aphantasia on https://en.wikipedia.org/wiki/Aphantasia they have a sidebar describing the spectrum of the ability to visualize images. The highest level is considered photographic. The other side is the inability to visualize anything.

I mean, a bunch of rocks falling on a keyboard can also generate tokens. So can an autocomplete software running on a 10W phone. But the quality is so far apart to the point you would find most people do not consider them on the same categories of "inference" as an LLM, not to mention a real human's mental capacity.

By the way, rocks falling is potential energy turned to kinetic energy so it is quite a lot of energy used. Didn't fact check this but if you drop 10 100g rocks per second from 2m height it is 30w according to GPT.

Although I guess you could use one rock fall to generate A LOT of tokens if you don't use keyboard, but just use the variance data from falling rock to generate many tokens at once.

E=mgh, so 10*0.1kg*9.8m/s*2m (per second) = 19.6 J/s = 19.6 W

You are correct, although there might be lost efficiency lifting rocks back up.

From the 100W consumed by the brain most of it is to keep the autonomic functions (respiratory, cardiac, digestion, processing senses [eyesight, hearing, touch, etc.]), actual thinking, executive function, language processing, etc. is a small percentage of the 100W in use.

Yes, 20 watts or so for just the brain.

But those 30 watts on the laptop also cover the screen, hard drive, wifi, etc.

If you want to count the brain alone, then it must be compared with just the silicon alone; if you want to count the whole body, then it must be compared with the whole machine.

Sounds fair if the laptop is capable of most of what the body can do an vice versa.

Maybe fair when we have bipedal robots doing work for us.

Depends what you want with the thing.

If the question is "I want to get this much mental stuff done", then you need the brain and the cost of keeping the brain alive and connected to the outside world — which for a human brain is the body, and for a silicon "brain" the computer/network.

If you want a humanoid robot to do be a drop-in replacement for human labourers in all professions, then sure (modulo the open question of how much brain power all our jobs actually require) a whole robot body has to be ready and you need to compare that body to our body.

If you want a special purpose machine that fully automates one specific product line but doesn't need to move, the "brain" might just be punched cards: https://en.wikipedia.org/wiki/Jacquard_machine

Okay I surely must have aphantasia then if you are describing seeing images like that.

Same, imagining objects feels like a fever dream for me, with illogical and warped proportions.

For me, the only visual parts of a dream that have ever been wrong like that, has been text — but the last time I recall dreaming of text was the late 90s, so I don't know if that's still the case.

Odours are a thing I can't "visualise", and those are usually entirely absent from my dreams.

I doubt your imagination runs at a full 500x500 resolution.

Imagination has been found to activate the same V1 cortex that vision does. The V1 cortex is retinotopic, so individual neurons (or at most small groups) act like pixels. There are approximately 280 million V1 neurons, so the neuroanatomical resolution of the visual cortex is much greater than 500x500.

This may not be the resolution we perceive as we imagine, but perception involves many additional systems and bottlenecks (notably, one can train oneself to have a more acute imagination, and there are people who report their imagination to be as acute as their vision).

I have no idea at which level of processing in which of my brain's visual areas my visual imagination is generated, however I can say that the subjective experience is "retina resolution".

Based on my own experience, it's only high resolution because when I focus on an area my mind generates the details. Until then, it's all broad strokes. JIT image generation in other words.

Computers can do this too! It’s called “foveated rendering”.

Well said, this precisely matches my experience. I can imagine and visualize scenes in real-time, but it's not like looking at landscape photography.

> 4 billion years

Well the computers/CPUs haven't been evolving for a few million years (https://en.wikipedia.org/wiki/Timeline_of_human_evolution); let's assume from Mammals onwards.

If a CPU has been developed so much within a few decades, then with the capabilities provided on ML/LLM/AI/and other-such-acronyms, in a million years we should at least have a "Lucy" (https://www.imdb.com/title/tt2872732/) evolved. In the meantime, I expect that in a few hundred years we will have a Borg Queen (perhaps without the interconnectivity).

Reply on news.ycombinator.com