Hacker News Reader

Comment by doctoboggan | Hacker News Reader

I asked the classic 'How many of the letter “r” are there in strawberry?' and I got an almost never ending stream of second guesses. The correct answer was ultimately provided but I burned probably 100x more clockcycles than needed.

See the response here: https://pastecode.io/s/6uyjstrt

View on HN

Well, to be perfectly honest, it's hard question for an LLM that reasons in tokens and not letters. Reminds me of that classic test that kids easily pass and grownups utterly fail. The test looks like this: continue a sequence:

Grownups try to find a pattern in the numbers, different types of series, progressions, etc. The correct answer is 1 because it's the number of circles in the graphical image of the number "9".

Damn I guessed the answer to be 9...

88880 = 9

I don't know if this is being done already, but couldn't we add some training data to teach the LLM how to spell? We also teach kids what each letter means and how they combine into words. Maybe we can do this with tokens as well? E.g.:

Token 145 (ar) = Token 236 (a) + Token 976 (r)

Repeat many times with different combinations and different words?

> but couldn't we add some training data to teach the LLM how to spell?

Sure, but then we would lose a benchmark to measure progress of emergent behavior.

The goal is not to add one capability at a time by hand - because this doesn’t scale and we would never finish. The goal is that it picks up new capabilities automatically, all on its own.

Training data is already provided by humans and certainly already does include spelling instruction, which the model is bind to because of forced tokenization. Tokenizing on words is already an arbitrary capability added one at a time. It's just the wrong one. LLMs should be tokenizing by letter, but they don't, because they aren't good enough yet, so they get a massive deus ex machina (human ex machina?) of wordish tokenization.

These tests always make me wonder: What qualifies as a valid pattern rule?

For example, why wouldn't "0" be a correct answer here (rule being "every other number on the right should be 0, other numbers do not have a pattern")?

Exactly, it's completely arbitrary. I like to frame it in terms of fitting n points (the existing data in the problem) to a polynomial of degree n+1, where there's an infinite number of ways to pick a solution and still satisfy the initial data.

Maybe the "solution with the lowest Kolmogorov complexity".

In a sibling comment, I replied that usually a repeating pattern can also be applied, but that one usually requires storing the n-sequence, rarely making it the shortest encodable rule.

I think it's better phrased as "find the best rule", with a tacit understanding that people mostly agree on what makes a rule decent vs. terrible (maybe not on what makes one great) and a tacit promise that the sequence presented has at least one decent rule and does not have multiple.

A rule being "good" is largely about simplicity, which is also essentially the trick that deep learning uses to escape no-free-lunch theorems.

This is exactly what I hated about these kind of questions!!

Or just, the pattern simply repeats for the n next number.. like come on, there is no objective metric for which rule is "better".

a possible lesson to infer from this example of human cognition, would be that LLMs that can't solve the strawberry test could not be automatically less cognitive capable that another intelligent entity (humans by default).

An extension of the idea could be that many other similar tests trying to measure and/or evaluate machine cognition, when the LLMs fails, are not precisely measuring and/or evaluating anything else than an specific edge case in which machine cognitions fails (i.e. for the specific LLM / AI system being evaluated).

Maybe the models are actually more intelligent than they seem, like an adult failing the number of circles inside the graphical image of the numbers, in the mentioned problem.

I think it's a "just so story" myth that children are better than adults at that task.

Well, I had to try just for fun of it. The immediate reply was the following:

[...]

Starting with 0 paired with 1. If I think about functions or some formula that could map these values, but that might be too complicated.

Alternatively, perhaps these numbers represent something specific. For example, maybe they're representing the number of certain features in the number itself, like the number of closed loops in the digit's shape.

Let's think about that:

0 has one closed loop. 5 has no closed loops. 6 has one closed loop. 7 has no closed loops. 8 has two closed loops. 9 has one closed loop. Wait a minute, that seems to match the sequence:

0 (1 loop) - 1

5 (0 loops) - 0

6 (1 loop) - 1

7 (0 loops) - 0

8 (2 loops) - 2

So, if 9 has one closed loop, then it should be paired with 1.

But let's double-check this theory with the given pairs.

0 has one loop, paired with 1 – matches.

5 has no loops, paired with 0 – matches.

6 has one loop, paired with 1 – matches.

7 has no loops, paired with 0 – matches.

8 has two loops, paired with 2 – matches.

Yes, this seems consistent.

Therefore, 9 should have one closed loop, so it should be paired with 1.

[...]

Then it continued trying another approach related to the number of angles in the digit shape, but it discarded it. Then "Alternatively, perhaps it's related to the number of segments that are 'on' in a seven-segment display." and so on.

I was about to give it some credit for being thorough, even if overly so. But then I noticed these gems:

   thinking about the phonetics, "strawberry" has three /r/ sounds...

   For example, in Spanish, it's "fresa," which has no "r"s, or in French, "fraise," which also has no "R"s...

   But to be absolutely certain, let's count them one by one: s-t-r-a-w-b-e-r-r-y. Okay, the first "r" is after the "a," then after the "b," and finally twice at the end before the "y." Wait, is that two or three "r"s at the end? Let's see: s-t-r-a-w-b-e-r-r-y. So, r-a-w-b-e-r-r-y. Okay, there are two "r"s before the "y." So, that's one after "a," one after "b," and two before "y," making it four "r"s in total. Wait a minute, now I'm confusing myself...

   For example, "copyright" has two "r"s, "arroba" has one "r,"...

   Wait, perhaps I'm confusing it with "arrowroot," which has two "r"s. Alternatively, maybe thinking of a word like "corroborate," which has four "r"s...

and

   Alternatively, thinking about Middle English "strawberie," still three "r"s.

It's not being thorough. It's just trained to sound like it's being thorough. It's producing the same garbage as any other LLM, this one just happens to get the answer right because it's been trained on data that includes discussions of the strawberry problem.

There's certainly a unique mechanism which might make it better at searching a space for answers, and that's pretty neat in its own way.

That's hilarious. It looks like they've successfully modeled OCD.

Yes, I thought that, too. And as LLMs become more and more "intelligent", I guess we will see more and more variants of mental disorders.

Wow this is fantastic, and I feel a little bit sorry for the LLM. It's like the answer was too simple and it couldn't believe it wasn't a trick question somehow.

Ha, interesting. FWIW the response I got is much shorter. It second-guessed itself once, considered 2 alternative interpretations of the question, then gave me the correct answer: https://justpaste.it/fqxbf

I mean that's less of a reasoning capability problem and more of an architectural problem as afaik it's to do with the way that words are broken down into tokens, strawberry becomes straw + berry, or something like st-raw-b-erry as per the tokeniser.

An LLM trying to get the number of letters will just be regurgitating for the most part because afaik it has no way to actually count letters. If the architecture was changed to allow for this (breaking certain words down into their letter tokens rather than whole word tokens) then it may help, but is it worth it?

I’m oscillating between “this is mind blowing” and “this is similarly impressive-looking-but-not-usefully-smart as other LLMs”.

The tone and expression is novel and it _looks_ like there’s something fundamentally different about reasoning but… also it keeps repeating the same things, sometimes in succession (a paragraph about “foreign languages” then another about “different languages”), most paragraphs have a theory then a rebuttal that doesn’t quite answer why the theory is irrelevant, and sometimes it’s flat out wrong (no Rs in “fraise” or “fresa”?).

So… holding my judgement on whether this model actually is useful in novel ways

Reply on news.ycombinator.com