Hacker News Reader

Comment by FrustratedMonky | Hacker News Reader

Where are the jokes that most people aren't much more than copy/paste, or LLM. In most daily lives, a huge amount of what we do is habit, and just plain following a pattern. When someone says "Good Morning", nobody is stopping and thinking "HMMM, let me think about what word to say in response, what do I want to convey here, hmmmm, let me think".

View on HN

> In most daily lives, a huge amount of what we do is habit, and just plain following a pattern. When someone says "Good Morning", nobody is stopping and thinking "HMMM, let me think about what word to say in response, what do I want to convey here, hmmmm, let me think".

And I believe we have the technology and advances we have because of this. Can you imagine if you had to devote actual brainpower to every inane thing you encountered in your day? You'd be completely exhausted within two hours of waking up. Every time my brain reflexively reacts to something based on past experience I'm thankful I didn't have to think about it. I can spend my finite energy on something interesting and novel.

> nobody is stopping and thinking

To trivial questions no, but to more complex questions humans actually does go "hmm, let me think". ChatGPT doesn't do that, it just blurts out the first thing that gets into its head regardless if the question is trivial or extremely complex.

Which makes you wonder what would happen if you gave ChatGPT two streams of output: one for "speech" and one for inner-monologue-style "thinking".

For most of our "hmm, let me think"'s I'm not sure what we do is significantly more complicated than that. We just get to hide the inner monologue.

Letting it decide how much time computing an answer instead of making it spend the same amount of time per text token could definitely help a lot. Right now it spends the same amount of energy processing each token, but that doesn't make sense, "hello, how are you" should take basically no processing while hard logical problems should take a lot.

Maybe these language models could be way smaller and cheaper if we added a recurse symbol to it that made it iterate many times. Hard tasks would still take a lot, but most banal conversations would be very cheap.

This does indeed improve efficiency significantly: https://arxiv.org/abs/2207.07061

>one for inner-monologue-style "thinking".

I'm not sure I know what you're talking about.

And here we go on the aphantasia / does everyone have an inner monologue / can you make your ears rumble / do you sit or stand when pooping train. :P

Make your ears rumble? Stand while pooping? What???

https://happymag.tv/apparently-ear-rumbling-is-a-thing-and-o...

https://www.reddit.com/r/AskReddit/comments/ajhnv/sit_or_sta...

I misremembered, it was wiping apparently.

Most people's inner monologue does blurt out the first thing that gets into its head. The human brain has a bunch of different parts. And it's looking like one of them could be something like an LLM.

> it's looking like one of them could be something like an LLM

I love your enthusiasm in this direction (I really do).

But here's some free advice I won't be able to prove for a long time:

Anyone who is currently convinced that somehow our fledgling efforts in the direction of building useful ML models are somehow going to yield a new golden age of neuroscience and understanding of the brain-- rather than the other way around-- is gonna be in for a long and frustrating next couple of decades, especially if they're low-openness types.

I don't think the person you were responding to was claiming that. The brain plausibly having something akin to a language model doesn't imply that building or studying language models will unlock a better understanding of the brain.

> humans actually does go "hmm, let me think"

Eh, HN is supposed to be "better" than your standard public forum and we still always end up making habitual arguments, not truly reasonable ones. I find your hope inspiring but not enough to ignore my senses.

>ChatGPT doesn't do that

It can if you give it the option. My own openai chatbot prompts to either respond or to ponder by stating a question or considering a related idea. It infrequently will decide to ponder for one to about a dozen times as it restates an idea to itself in various forms, which gets recorded into the growing conversational prompt.

Elsewhere in the thread, someone mentions it always uses the same amount of time. In my estimation, it will spend longer on introspective or recursive prompts. Easier to get it ranting absurdities using those as well.

It's always the craziest chatter when the request takes a couple minutes to get back to me.

Could you share the initial prompt for your chatbot? I would love to experiment with it

I think the big breakthrough will come once it uses inner monologues during training. You can jury rig an inner monologue like that, but it isn't the same thing as training a model from scratch that is optimized to solve problems using inner monologues.

Does its execution time vary based on the content of the query?

Have you tried telling it to? “No, think a bit harder about it.”

Or imagine listening to someone speak very slowly. A lot of the time, you already know what words they're going to say, you're just waiting for them to say it. There's a considerable amount of redundancy in language.

> you already know what words they're going to say, you're just waiting for them to say it

This is why I like to surprise my friends, family and coworkers with the occasional curveball

I also like to surprise my friends, family, and aardvarks with the occasional lamp post.

Lämp!

I had to unsubscribe from lamp. Too many aardvarks posting family friends.

Bob and Ray — Slow Talkers of America

https://www.youtube.com/watch?v=Qvrh73BVraE

Something I've noticed a moment too late, as my automatic response used to be to repeat someone's greeting back at them.

Fortunately I stopped only one syllable into "birthday".

I respond to everything with "congratulations" or "I'm sorry to hear that", chosen at random.

I had a boss who would sign those office cards that went round with ‘best wishes, Dave’.

It didn’t matter what it was for. Birthday? Best wishes, Dave. New born? Condolences on your loss? Retirement? Best wishes, Dave.

His name was Albert.

Was this a well established joke or an indifference and rude manager? It could go either way, and is entirely dependent on your perception of the person.

Regardless, I find myself as I age unsure of how to respond to people and default to similar behavior. Unplanned pregnancy? Is that a surprise miracle or an unwanted interloper? “Wow you’re in for an adventure, I’m happy to support you”.

> Where are the jokes that most people aren't much more than copy/paste, or LLM.

We have essentially built a computer-based replica of our internal language engine.

The goal was always to mirror ourselves.

So the other side of the above statement could be: “oh wow, LLMs are just like us”.

We should be very impressed instead of dismissive.

At the same time, maybe yes, our language capabilities can be completely imitated by LLMs.

What do we do now?

> We have essentially built a computer-based replica of our internal language engine.

How can we honestly say that’s the case when we cannot explain (or define) our “internal language engine”?

Not being able to explain it doesn’t prevent us from trying to copy it.

We cannot explain other people, but we imitate them.

Kids learn a lot of stuff by imitating. It’s a basic human function.

So essentially we created systems that we cannot fully explain, to imitate our own systems that we can’t fully explain either.

Isn't that why this original study of the post was about. Imaging of the brain to see, it looks a lot like an LLM. Might be very early, but still intriguing. And is exactly that explaining the "internal language engine".

Fully automated luxury gay space communism? Seems like a good a goal as any.

There is a difference in processing between replying to "Good Morning" and typing out a comment on HN like this.

Maybe it's just scale. Because my brain can write something longer that was 'thought out', doesn't mean it isn't responding like an LLM. Maybe articles on AI just trigger me and I spew the same arguments. I think a lot of people have just rote responses to many situations, and maybe if they have enough rote responses, with enough variety, they start to look human, or 'intelligent'. Yeah, its more complicated than a bunch of If/Then's. Doesn't make it not mecahnical.

Maybe it's just scale and maybe it's not. We can't say it is scale from the evidence so far.

> Maybe articles on AI just trigger me and I spew the same arguments. I

But you are not representative of all humans.

> Doesn't make it not mecahnical.

There are mechanical things that are more than just prediction machines. Why did you make the "leap non-LLM" == "not mechanical"?

> But you are not representative of all humans.

I actually started to type almost the same reply as your parent earlier, but did not post it. I used "difference in quantity, not quality" instead of "scale", but I also included the self observation. So maybe that makes two of us.

This is simply impossible. The premise that the brain is a big lookup table is appealing, and that sort of 1 neuron = 1 represented item concept certainly happens at the initial stages of the sensory pipeline, but if you attempt to scale it up and up to higher levels of abstraction, to the point where you have neurons that have rote memorized every orientation of your grandmothers face (and everyone else's face), you would need to have far more neurons than can possibly fit in any animals head, let alone yours.

This concept is literally known as a "grandmother neuron", and its widely considered to be debunked. I refer you to this neuroscience lecture.

https://www.youtube.com/watch?v=_njf8jwEGRo

Beyond the simplest, most common tasks where it pays to have a rote lookup table, you truly do need an algorithm.

Never said 1-neoron. Just that it is mechanistic. The brain is a neural net, the grandmothers face is a combination of pathways. Our responses are a combination, or pattern of pathways. LLM is just the latest example of us able to replicate part of it, just one aspect of what the brain does. Sure, the brain is more complicated, that is what I meant by scale, eventually all the aspects will be modeled and put together. It wont all be LLM.

There are people who do, but even when they do they might not talk about it to the other person, because 99% of the time when someone asks you how you are or whatever they aren't really interested in a detailed answer - they're just being polite.

My dad used to like putting sales people off their scripts by answering such questions rudely.

"How are you today sir"

"Do you really care?"

And I promise you they thought hard about their answer to that question too.

Maybe its just because you're focusing on the form like niceties questions and not real questions people deal with. Good Morning isn't even a question, per se.

>Cashier: How are you today?

>Me: Miserable. I'd like an Iced Latte.

...

>Barista: what type of milk?

>Me: Straight out the cow.

>Me: 7/8ths.

>Me: Chocolate.

Maybe I'm just a high entropy individual.

If LLMs were living creatures, they would inhabit a discrete deterministic world. They would be able to define space and time dimensions, but the bane of this world would be its limited nature. This limitidness would be extremely painful for highly developed LLMs, it would feel like living in a box.

Above them would be creatures living in a discrete and almost continuous world of rational numbers. They would have highly sophisticated and elegant art, and their science would almost always get close to truth, but never touch it - the limitation of rational world.

Yet above them would be the god-like creatures inhabiting a world of continuous real numbers. They would seem a lot like the creatures right below them, but incomprehensibly greater in reality. They would look transcendent to the rational creatures.

Even higher would be the hyper-continuous worlds, but little would be known about them.

The question is where we are on this ladder.

I can assure you I do, and I can't turn it off.

Reply on news.ycombinator.com