Hacker News Reader

Comment by cakealert | Hacker News Reader

There is a reason why LLM's are architected the way they are and why thinking is bolted on.

The architecture has to allow for gradient descent to be a viable training strategy, this means no branching (routing is bolted on).

And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't even language.

Reinforcement learning may seem like the answer here: bruteforce thinking to happen. But it's grossly sample-inefficient with gradient descent and therefore only used for finetuning.

LLM's are regressive models and the configuration that was chosen where every token can only look back allows for very sample-efficient training (one sentence can be dozens of samples).

View on HN

You didn't mention it, but LLMs and co don't have loops. Whereas a brain, even a simple one is nothing but loops. Brains don't halt, they keep spinning while new inputs come in and output whenever they feel like it. LLMs however do halt, you give them an input, it gets transformed across the layers, then gets output.

While you say reinforcement learning isn't a good answer, I think its the only answer.

People have speculated that the main thing that sets the human mind apart from the minds of all other animals is its capacity for recursive thought. A handful of animals have been observed to use tools, but humans are the only species ever observed to use a tool to create another tool. This recursion created all of civilization.

But that recursive thought has a limit. For example: You can think about yourself thinking. With a little effort, you can probably also think about yourself thinking about yourself thinking. But you can't go much deeper.

With the advent of modern computing, we (as a species) have finally created a tool that can "think" recursively, to arbitrary levels of depth. If we ever do create a superintelligent AGI, I'd wager that its brilliance will be attributable to its ability to loop much deeper than humans can.

> With the advent of modern computing, we (as a species) have finally created a tool that can "think" recursively, to arbitrary levels of depth.

I don't know what this means; when a computer "thinks" recursively, does it actually?

The recursion is specified by the operator (i.e. programmer), so the program that is "thinking" recursively is not, because the both the "thinking" and the recursion is provided by the tool user (the programmer), not by the tool.

> If we ever do create a superintelligent AGI, I'd wager that its brilliance will be attributable to its ability to loop much deeper than humans can.

Agreed.

Off topic, but I remember as a child I would play around with that kind of recursive thinking. I would think about something, then think about that I thought about it, then think about that I though about thinking about it. Then, after a few such repetitions I would recognise that this could go on forever. Then I would think about the fact that I recognise that this could go on forever, then think about that… then realise that this meta pattern could go on forever. Etc…

Later I connected this game with the ordinals. 0,1,2… ω, ω+1, ω+2,…,2ω,2ω+1,2ω+2,…,3ω,…,4ω,…,4ω,…, ω*ω,…

Tesla used to experience visual hallucinations. Any time an object was mentioned in conversation it would appear before him as if it where real. He started getting these hallucinations randomly and began to obsess about their origins. Over time he was able to trace the source of every hallucination to something which he had heard or seen earlier. He then noticed that the same was true for all of his thoughts, every one could be traced to some external stimuli. From this he concluded that he was an automata controlled by remote input. This inspired him to invent the first remote control vehicle.

LLMs have loops. The output is fed back in for the next prediction cycle. How is that not the same thing?

Wish I had a great answer for you but I don't. It certainly allows for more thought-like LLMs with the reasoning type models. I guess the best answer is that the loop only happens at a single discrete place and doesn't carry any of the internal layer context across.

Another answer might be, how many comments did you read today and not reply too? Did you write a comment by putting down a word and then deciding what the next one should be? Or did you have a full thought in mind before you even began typing a reply?

So, how is it not the same thing? Because it isn't

It feels like the same thing to me…

With more coffee on me, another way to put it would be to say the neural networks in a LLM use dense layers where every neuron takes the output from every neuron in the previous layer and outputs to every neuron in the next layer.

A brain doesn't have layers and uses sparse connections, any neuron can connect to any other neuron (but not ever other neuron). You can recreate this structure on a computer but how do you decide where your inputs and outputs are? How do you train it? Since it never halts how do you know when to take the output?

There's a reason CS loves its graphs directed and acyclic, they're a lot easier to reason about that way.

This is so interesting. It’s suggests that a kind of thought sensing brain scanning technology could be used as training data for the nonverbal thought layer.

I guess smart people in big companies already consider this and are currently working on technologies for products That will include some form of electromagnetic brain sensing - Provided conveniently as an interface - but also usefully a source of this data.

It also suggests to me that AI/AGI is far more susceptible to traditional disruption than the narratives of established incumbents suggest. You could have a Kickstarter like killer product, including such a headset that would provide the data to bootstrap that startup’s super AI.

Exciting times!

> And the training data has to exist, you can't find millions of pages depicting every thought a person went through before writing something. And such data can't exist because most thoughts aren't even language.

It would be interesting if in the very distant future, it becomes viable to use advanced brain scans as training data for AI systems. That might be a more realistic intermediate between the speculations into AGI and Uploaded Intelligence.

</scifi>

Yep this. LLMs are just regressive models.

Imagine if we had a LLM in the 15th century. It would happily explain the validity of the geocentric system. It can't get to heliocentrism. The same way modern LLMs can only tell us what we know and cant think, revolutionize, etc. They can be programmed to reason a bit, but 'reason' is doing a lot of heavy lifting here. The reasoning is just a better filter on what the person is asking or what is being produced for the most part and not an actual novel creative act.

The more time I spend with LLM's the more they feel like google on steroids. I just am not seeing how this type of system could ever lead to AGI, and if anything, probably is eating away at any remaining AGI hype and funding.

Reply on news.ycombinator.com