Hacker News Reader

Comment by throw310822 | Hacker News Reader

Hey, heads up. It's now 2025, machines have mastered language, displaying better linguistic abilities than those of a large percentage of the population. They also do so using a small fraction of the neurons and synapses found in the human brain, despite being able to express themselves in tens or hundreds of languages and across an extremely wide range of subjects (much beyond the average general knowledge of human beings), dispelling the myth that artificial neurons are much less powerful than the real ones.

View on HN

LLMs are interest case, but they are nearly flat. Neural system of mammals is structured, as I understand, neocortex consists of about 100 millions structures each ~ 100x400 neurons, something like this. Anyway, just from calculation of human thinking delay appear human NN is just about 500 layers.

Second difference, natural I are feed-forward network, not back propagated as typical AI NN, because natural use some chemical method to "calculate" parameters.

I think, with right structure and some FF method, GAI is already technically achieved.

PS FF NN already exist, but problem that it now using classical method of differential equations which is prohibiting real use, because too much computation need.

> LLMs are interest case, but they are nearly flat.

They are less specialized than brains, but compensate millions of years of evolutionary adaptation with a larger corpus of text, about the same size as 11,000 people use in their lifetime, assuming 1B words/human.

Interestingly, humans needed 10 million times more text for cultural evolution, based on an estimate of 110 billion humans who ever lived.

So making progress is millions of times harder than catching up.

> humans needed 10 million times more text for cultural evolution

Could you provide source, where exists this 10 million times more text?

- Text which is not stored on some material carrier is just forgotten.

Largest old text system I know is Confucianism. It is huge, but very far from 10 million times 1B.

Other known text systems are all much smaller than Confucianism.

https://en.wikipedia.org/wiki/Confucianism

Estimating the number of words used by humanity: starting from total number of humans who ever lived 110B, multiply that with 80 year * 365 days * 35000 words ~= 1B words/human lifetime, you get 1.1*10^20 words.

Now, the training set of GPT-4 was around 14T tokens, let's say 10T words. If you divide them you get around 10 million.

> total number of humans who ever lived 110B

But why do you think that all 110B individuals have made significant contribution to overall intelligence?

I think, at best case, in every generation exists few significant contributors, but all others are just carriers of genome diversity and nothing more, and all they have done, just disappear as a breath of wind.

So, for 200k years, if one generation 40 years, will be just 5000 generations, ok, lets consider significant 1000 persons from each generation, will be just 5millions, 5 magnitudes less than your estimation, and BTW much closer to known estimations of humanity knowledge and size of datasets used to train largest existing models.

BTW one of the largest in world collection of texts - Library of Congress, old estimation considered to about 100B characters, or approximate 100B LLM tokens.

Plus, they have thousands of multimedia carriers (cinema, music, etc) and one time was archived all tweets for history preservation.

But all multimedia and tweets are much less volume than texts, but added few additional dimensions, hard to express with text.

No one said artificial neurons are “less powerful.” They just work differently, and the way we “train” them is much less efficient and extensible. So of course there’s value in trying to understand how our language mechanisms work.

“Express themselves”: such a wealth of wrongness packed into two short words. There is no self involved. There is no expression involved.

Reply on news.ycombinator.com