Hacker News Reader

I think what’s fascinating about GPT developments is it may very well emulate part of our left brain hemisphere. As McGilchrist pointed out, with lots of research and studies on people with brain damage strokes etc. is that it is the left hemisphere that sees the world from an internal representation. Right hemi. sees a car as the "thing in itself" in direct experience (colours etc). The left hemisphere makes it into a category of "car", now removed from direct experience.

And just like gpt’s data set isnt necessarily truth, so is our own image of the world which as we know can be deeply distorted through abusive childhood, cults etc. In fact, all of human knowledge is simply beliefs, agreed stories about reality. For example "red" is a word/sound that points to an experience. The word alone only has meaning in context (what GPT can handle), but can never substitue for a conscious experience.

Crucially imho, software will never be able to do what the right hemisphere does. And I find it dumbfounding that even Lex Fridman doesnt see the fundamental difference between conceptual thought / language based reasoning, and direct experience aka consciousness.

There's billions of years of compressed knowledge in those 4 year olds. Lots of useful priors.

You basically landed on Chomsky's universal grammar. And this only proves the chatgpt critics: we have no idea what those priors are, how they evolved, why they are so effective and thus we are not even sure they exist. Until this is demonstrated I think it is very fair to say chatgpt is applying very different reasoning to what humans are applying.

Also language is a fairly recent development in human evolution (only 60-70 generations ago) which makes it much more puzzling how a mechanism that is so efficient and effective could evolve so quickly, let alone pondering how actual languages evolved (almost instantly all over the world) given how hard it is to construct an artificial one.

Universal grammar only speaks to the ability to parse and form sentences, not to internal logic.

That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in, even if the underlying reasoning might be flawed, goes against the argument of something such as universal grammar existing.

At the very least, it shows that a neural net is capable of parsing and producing coherent grammar without any assistance from universal grammar. It does not prove that humans don't have it, but it does make a compelling case that it's clearly not required for humans to have it either.

That ChatGPT is capable of producing human-like sentences from pattern recognition without any universal grammar baked in

You didn't address or missed the main point: chatgpt requires something in the order of a trillion tokens to be capable of producing what you mentioned in one language.

There are 40 months old babies that are fairly conversant in both Chinese and English, and are able to detect sarcasm with something like 0.0000001% of the tokens, doesn't that give you pause that part of language acquisition is innate to humans and is not entirely acquired the way chatgpt is taught?

It requires so much to be able to generate answers tot hings asked about a wide variety of subjects, not to be able to generate grammatically valid sentences and parse them, which other a.i.'s also did before with far fewer tokens.

60-70 generations ago

More like 1000+ considering the Chauvet painters certainly had speech.

let's even make it 10,000+ generation, this still makes it quite magical to see how these priors could evolve to make language acquisition so trivial to humans relative to chatgpt. Chatgpt requires on the order of gazillion of epochs and tokens, and can still confidently express elementary mistake that a 4 year old doesn't.

Yes, per Wikipedia 10000 seems like it. Plus the human brain and anatomy was ready for it (even neanderthal is thought to have spoken).

I don't see GPT's blunders as mistakes. They are to us for sure but would not be to another GPT instance in that it would produce the same continuation to a prompt and thus agree.

Plus the human brain and anatomy was ready for it

We have no idea how evolution "readied" a deeply complex organ like the brain over many thousands of years, then almost instantly repurposed it for language acquisition and generation. To further hypothesise that what it was "readying" was something that trains from data in a way similar to how chatgpt is trained from data makes it even more astonishing and until this is demonstrated it is more scientific to not accept this hypothesis.

ChatGPT is not 4 years old, it's a newcomer, give it some more time, ok? It's still developing, it's not a final product. There is already a huge difference between GPT-3.5 and GPT-4. Next iterations will be able to do more and more, especially if they get tools, self-reflection and vision.

You missed the point, we are not saying that GPT isn't powerful, we are saying that it is different to human intelligence. Of course GPT will get more powerful and generate more interesting sentences.

But until someone demonstrates a GPT that can learn from a tiny dataset what a multi-lingual blind 4 year old learns it is very fair to challenge the hypothesis that humans learn the way a deep learning network learn.

Try taking ChatGpt, and teaching it on a new language that wasn't in it's training data. It will probably learn it faster than any 4 year old.

You might say that's not fair because we are comparing a pre-trained LLM with a blank slate newborn. But human hardware is also pre-trained by billions of years of evolution. We are hardwired to understand language and certain world concepts. It is not fair to compare hardware that is designed for language and reasoning to the hardware used for ChatGpt.

Another line of thinking: why does the amount of training matter? LLM and humans are completely different implementations.

See my responses to rest of the thread. Nobody understands or proposed a decent model of how an innate deep learning neural network was trained via evolution. Language acquisition in humans is in evolutionary terms an extremely recent phenomena, practically yesterday. Whatever the mechanism that processes language in humans, it was trained in very little number of evolutionary iterations, far less iterations compared to what chatgpt needs.

> So whatever is driving reasoning and intelligence in humans...

Probably multiple brain-areals that work differently and in conjunction. "Left-brain" like language functions working with serial information, "right-brain" function that tend to work on images (= parallel information), combined with symbolic-logical reasoning, an extremely strong programmable aversion system (the emotion of disgust) and the tendency to be lazy = optimizing in- and output.

The 4yo language abilities are a combination of spoken language and visual/manual interaction with the physical world. So there's a lot of multimodal learning going on and visual and tactile input undoubetedly counts for a lot.

I've just had GPT-4 generate a lot of Golang code. Boilerplate, but real code nonetheless. Did it perfectly, first time round. No typos, got the comments right. Much faster than any intern. No 4yo can do that.

Exactly we have established there are things blind four year old can do and learn many orders of magnitude more efficiently than chatgpt (acquire and manipulate human language), and there are things chatgpt can do that even an average forty year old would struggle to learn (write perfect boilerplate Golang).

The two intelligences are not the same, the way they are trained in particular is vastly different.

Also the fact that humans learn some language manipulation (or that it gives them such tremendous efficiency in learning language) from tactile experience is superficially plausible but it hasn't been demonstrated yet to any interesting level.

Why does feeling the weight of a stone in your hand make you better at parsing and understanding grammar or envision abstract concepts? Also, most animals have as much or even more tactile experience (including primates which have similar brains) and yet this doesn't seem to provide them with any kind of abilities similar to manipulating human language.

It's well known what stages kids start to solve certain reasoning tasks.

Like putting water from a tal glass in a wide one. They understand this quite late.

Just because humans have additional more/different inputs doesn't imply chargpt can't start to learn to reason like us.

It could easily be than the fine-tuning we do (thinking through things) is similar to reading a huge amount of text like chargpt does.