Hacker News Reader

Comment by ypeterholmes | Hacker News Reader

“The high connectivity [in a human brain] is very different from that found in the central processing unit of any digital computer, where one transistor typically connects to a handful of other transistors.”

Has he been living under a rock? Modern AI models already outpace the connectivity of the human brain, and are only getting bigger.

View on HN

Assuming that GPT4 has 1T+ parameters, you’re incorrect.

Modern estimates are that there are 100T neuronal connections in the adult human brain [0]. And that’s neuronal connections alone.

Astrocytes also make direct connections with neurons and can modify and induce neuronal activity [1]. There are 100B neurons [0] and ~20B astrocytes in the adult human brain [2, 3].

So this 100T connections estimate is only a small slice of the picture of human brain activity.

[0] https://medicine.yale.edu/lab/colon_ramos/overview/#:~:text=....

[1] https://neuraldevelopment.biomedcentral.com/articles/10.1186....

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5063692/

[3] https://link.springer.com/article/10.1007/s00429-017-1383-5

I was basing my number on Wolfram's work here: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

"In human brains there are about 100 billion neurons (nerve cells), each capable of producing an electrical pulse up to perhaps a thousand times a second."

So maybe that's wrong. Thanks for the links. Even so if we look out 10, 100, 200 years from now, that level of complexity will be greatly surpassed by AI.

Indeed, the computing power of cells is nothing to blink at.

It is astonishing that we can mimic human reasoning so well with these LLMs, but the essence of cognition seems to still be missing.

I agree with your belief that human processing will be surpassed in the future. We do live in some exciting times :)

Speed relative to environment is important in determining if something is intelligent. If it takes N flops to produce one token but I carry those out in 10 years, that is not intelligence.

The machines will be much faster than us, and soon. So what's your point?

Consciousness or at least intelligent behaviour is how fast a system is relative to environment.

AI is just software. He's talking about the physical transistors in a computer.

That's precisely my point. The database connections involved in a neural network are the corollary, not transistors. What am I missing?

The author is clearly talking about PHYSICAL connectivity in the brain. Thus why he says AI is unlikely to become conscious without the "advent of new technology". The brain is a physical neural network. AI is a software simulated neural network. Nowhere in the article does the author confuse the two.

The physical number of transistors is irrelevant. If in software you were able to perfectly simulate the 100T physical neuronal connections in a real brain, then you would perfectly recreate that brain's conscious experience.

Granted, you would be doing it with a frame rate limited by the processing power of the computer, but that just means that a thought that takes a human 1 second to arrive at might take much longer for the AI (for now).

I don't think that's what the quote is saying. I realize you are putting a shim layer between implementation and computational level, which was popular for many years: https://en.wikipedia.org/wiki/Level_of_analysis#Marr's_tri-l...

But at this point the TYPE of computation performed by neurons, which is unlike what modern computers do-- for example, brains do not appear to have addressable memory units separate from compute units-- do seem to be differing enough to perhaps explain some of the gaps between computers and minds. Some even think neural computation is a different category of computation altogether: https://pubmed.ncbi.nlm.nih.gov/23126542/

What? No. Consciousness isn't magic. Consciousness doesn't emerge purely from computation, that's just nonsense. Do you realize computers can have encrypted memory? How the heck can you possibly hope to perfectly recreate conscious experience with encrypted memory? How about virtual memory? Is consciousness seriously parsing the virtual memory table to reassemble the memory of a process, dereferencing pointers, parsing UTF8 bytes, interpreting computer code? Be real.

It's not correct to say modern AI models outpace the connectivity of the human brain.

No? The human brain has ~100 billion neurons and ~1 trillion connection weights. Google’s PaLM uses ~540 Billion nodes with ~100 trillion connection weights.

And the key point is this- these models are the worst they will ever be, and are gaining size at pace. So even if you we grant the argument that our brains are still a bit more complex, hopefully we can agree that will not be the case in 5 years. Heck, how about 20 years, or 100? Let's be real.

If you took the "no it won't" side of every argument about "how in X number of years, AI is sure to Y", you'd be way ahead.

In any event, raw parameter/weight count to me seems like a very primitive way to judge "complexity" in comparison to the human brain. Looked at most ways, our brains are for more efficient at doing the incredible things they do than LLMs. Consider how little language young children are exposed to in comparison to LLMs given their abilities to figure out how to produce language.

If the brain doesn't work like an LLM, you can expand the size and "complexity" of these models to the moon and they won't outperform the brain. Current models can write impressively well, but they can barely do math. It's clear they don't reason as we do.

nodes and weights are different from neurons and connections. Neurons are also not the only components in the brain which contribute to intelligence.

google recently scanned a 1mm cube of human brain which was 1.5Petabytes of raw data. The AI hardware that Google trains on is multiple racks.

I think a better analogy would be between an entire google datacenter (including all the networking, storage, sensors, processors, and memory) and a human body although even then it's a stretch.

Reply on news.ycombinator.com