Hacker News Reader

Comment by silisili | Hacker News Reader

> we understand language in much the same way as these large language models

Yeah, gonna need proof on that one.

First, LLM slop is uncannily easy to pick out in comments vs human thought.

Second, there's no prompt that you can give a human that will generate absolutely nonsense response or canceling the request.

If anything, it feels like it doesn't actually understand language at all, and just craps out what it thinks looks like a language. Which is exactly what it does, in fact, sometimes to fanfare.

View on HN

Hey, it's time you upgraded from GPT-3.5 :).

> First, LLM slop is uncannily easy to pick out in comments vs human thought.

So you think. Evidence says otherwise.

> Second, there's no prompt that you can give a human that will generate absolutely nonsense response or canceling the request.

Never been to a debate club? Or high school? Or talked with kids? 4 year olds don't even need a prompt, they'll ramble on and on until they get tired; if you listen in, you may even notice they seem to have a context window that's about 30 seconds worth of tokens wide :).

> it doesn't actually understand language at all, and just craps out what it thinks looks like a language.

I.e. much like every human on the planet. Or are you one of those aliens who actually learn languages by memorizing grammatical rules and learning to apply them? No one else does that.

> 4 year olds don't even need a prompt, they'll ramble on and on until they get tired; if you listen in, you may even notice they seem to have a context window that's about 30 seconds worth of tokens wide :)

As the parent of a 4 year old, this is spot-on, especially the 30-second context window. My kiddo will often embark upon a sentence with no idea of how to choose anything beyond the next few tokens, and then continue reaching for new stuff, merely to keep the sentence going.

I say this as someone with a deep respect for Humans. However, I also think most of us tend to operate in "default mode" (in the sense that David Foster Wallace described in his "What is Water?" commencement speech).

Overall, LLMs are a stark reminder to us that without careful and intentional living, we are in danger of being more like LLMs than we'd like.

> My kiddo will often embark upon a sentence with no idea of how to choose anything beyond the next few tokens, and then continue reaching for new stuff, merely to keep the sentence going.

Exactly right. Now, the "context window" observation hit me when I started to think not where the story is going, but where it's already been, and suddenly noticed that the repetitiveness in my daughter's rambling is meaningful.

I.e. she'd go on and on saying things like, "and my brother turtle has this, and his brother hedgehog, hedgehog the brother of my brother the turtle wore an apple, and the brother of the turtle which is my brother, turtle is my brother and his brother hedgehog did this and that, etc." - and I realized that this is her way of carrying context forward; anything that wasn't repeated in the past ~30 seconds would never appear again in the story, or would in a changed form ("the hedgehog, the sister of giraffe Betty, ...").

Over time, I could see how she was learning to be more economical with limited working memory, and faster at accessing the long-term memory (initially, recalling some detail would take a lot of visible effort, and take so long it would wipe half the context, leading to a huge shift in the story) - and I can't help but think that having to push hard against the limits of a small context window is important in development of language and intelligence in general. Figuring out different associations, faster associations, more indirect associations.

I wonder what the internal drive for this is, too - what made her improve her thinking to produce more interesting stories within the same memory limits, instead of getting forever stuck sounding like a Markov chain trained on the Book of Chronicles[0]. Some positive response to novelty, perhaps?

[0] - https://www.biblegateway.com/passage/?search=1+Chronicles+1&...

> Over time, I could see how she was learning to be more economical with limited working memory, and faster at accessing the long-term memory (initially, recalling some detail would take a lot of visible effort, and take so long it would wipe half the context, leading to a huge shift in the story) - and I can't help but think that having to push hard against the limits of a small context window is important in development of language and intelligence in general. Figuring out different associations, faster associations, more indirect associations.

Interesting…

I've wondered previously if the reason for childhood amnesia is that we need to learn how to form memories. I didn't consider the obvious implication of that idea, that if we learn how to form memories, we probably also need to learn how to access those memories.

Using the term 'slop' to deride it for its lack of originality is pretty funny to me.

Reply on news.ycombinator.com