Hacker News Reader

Comment by awesomeMilou | Hacker News Reader

Reducing the capability of the human brain to performance alone is too simplistic, especially when looking at LLM's. Even if we would assign some intelligence to LLM's, they need a 400w GPU at inference time, and several orders of magnitude more of those at training time. The human brain runs constanly at ~20w.

I highly doubt you'd be able to get even close to that kind of performance with current manufacturing processes. We'd need something entirely different from laser lithography for that to happen.

View on HN

The problem isn't the manufacturing process, but rather the architecture.

At a low level: We take an analog component, then drive it in a way that lets us treat it as digital, then combine loads of them together so we can synthesise a low-resolution approximation of an analog process.

At a higher level: We don't really understand how our brains are architected yet, just that it can make better guesses from fewer examples than our AI.

Also, 400 W of electricity is generally cheaper than 20 W of calories (let alone the 38-100 W rest of body needed to keep the brain alive depending on how much of a couch potato the human is).

> Also, 400 W of electricity is generally cheaper than 20 W of calories

Are you serious? I don't think you have to be an expert to see that the average human can perform more work per energy intake than the average GPU.

> The problem isn't the manufacturing process, but rather the architecture.

It's very much a problem, good luck trying to even emulate the 3D neural structure of the brain with lithography. And there are few other processes that can create structures at the required scale, with the required precision.

> Are you serious? I don't think you have to be an expert to see that the average human can perform more work per energy intake than the average GPU.

You're objecting to something I didn't say, which is extra weird because I'm just running with the same 400 W/20 W you yourself gave. All I'm doing here is pointing out that 400 W of electricity is cheaper than 20 W of calories especially as 20 W is a misleading number until we get brains in jars.

To put numbers to the point, at $0.10/kWh * 400 W * 24h = $0.96, while the UN definition for abject poverty is $2.57 in 2023 dollars.

As for my opinion on which can perform more work per unit of energy, that idea is simply too imprecise to answer without more detail — depending on what exactly you mean by "work", a first generation Pi Zero can beat all humans combined while the world's largest supercomputer can't keep up with one human.

> It's very much a problem, good luck trying to even emulate the 3D neural structure of the brain with lithography.

IIRC by volume a human brain mostly communication between neurones; the ridges are because most of your complexity is a thin layer on the surface, and ridges get you more surface.

But that doesn't even matter, because it's a question of the connection graph, and each cell has about 10,000 synapses, and that connectivity be instantiated in many different ways even on a 2D chip.

We don't have a complete example connectivity graph for a human brain. Got it for a rat, I think, but not a human, which is why I previously noted that we don't really understand how our brains are architected.

> And there are few other processes that can create structures at the required scale, with the required precision.

Litho vastly exceeds the required precision. Chemical synapses are 20-30 nm from one cell to the next, and even the more compact electrical synapses are 3.5 nm.

I think the power efficiency is very relaxed as you just need to consider the value of a single performance digital brain that exceeds human level.

>The human brain runs constanly at ~20w.

Most likely because any brains that required more energy died off at evolutionary time scales. And while there are some problems with burning massive amounts of energy to achieve a task (see: global warming) this is not likely a significant short falling that large scale AI models have to worry about. Seemingly there are plenty of humans willing to hook them up to power sources at this time.

Also you might want to consider the 0-16 year training stages for human which have become more like 0-21 year training stages with at minimum 8 hours of downtime per day. This does adjust the power dynamics pretty considerably in that the time actually thinking daily drops to around 1/3rd the day boosting effective power use to 60w (as in you've wasted 2/3s the power eating, sleeping, and pooping). In addition that model you've spend a lot of power training is able to be duplicated across thousands/millions of instances in short order, where as you're praying that human you've trained doesn't step out in front of a bus.

So yes, reducing the capability of a human brain/body to performance alone is far too simplistic.

Some of that 8 hrs of sleep includes fine tuning on recent experiences and simulation (dreams).

> We'd need something entirely different from laser lithography for that to happen.

Like, you know . . . nerve cells.

Care to explain?

I was being facetious in that brains already exist.

Reply on news.ycombinator.com