Hacker News Reader

Comment by sva_ | Hacker News Reader

I don't understand how you can write a whole piece on Transformers failing to do arithmetics without mentioning the tokenizer used in GPT, which makes it more or less impossible for the model.

The reason is imo probably more along the lines of that the model can't see the actual numbers you're giving it.

When you provide GPT3 with the number "10011111110100011100011011110010", it actually sees [3064, 26259, 1157, 8784, 18005, 1157, 18005, 8784, 1157, 3064, 940] [see 0].

The number isn't even split up into even parts. Even if it knew the Peano axioms, it wouldn't have a chance because it can't 'see' the actual numbers that you're feeding it. Hence it can only estimate these operations based on combination pairs it has seen in training data, and it seems pretty obvious why this can't be a robust basis for doing arithmetic.

0. https://platform.openai.com/tokenizer

View on HN

It also completely ignores the developments around GPT tool use. If you’re evaluating the AI field as a whole, why are you restricting your analysis to the raw output of GPTs? Arguably the most impressive display of intelligence exhibited by these models is their ability to use tools like calculators to augment their own abilities.

It doesn’t matter if the model can’t do three digit multiplication without access to a calculator. Most people can’t mentally multiply large numbers either.

Most people can do 32-bit binary addition using a pen and paper. And the point of using addition as an illustrative example is that the rules of addition are extremely simple, yet a binary addition problem can easily be scaled to N bits of entropy. This means it's very easy to show that the system fails to apply simple logic rules in such cases.

It didn't fail logic, it failed to read your string. You didn't even notice that the terms being added in the 'show working' version don't match the given strings. And you are elsewhere talking about adding spaces to make the digits 'tokens'... you have no idea what you are talking about at all.

You can input the space-separated strings into OpenAI's tokenizer: https://platform.openai.com/tokenizer

You'll see that it does properly tokenize each binary digital as a separate token.

But again, the tokenization doesn't matter. I could say to you, please add these two binary numbers:

      101 0011 11001
    + 00 11 10 100011

And you could do it.

If your goal is to simulate a human doing addition with pen and paper it only seems fair to give the GPT read/write access to a text file. Outputting the answer directly from the model is much more akin to mental math.

ChatGPT does have read write access to a text file. It's allowed to show its work as part of its output, which it can write tokens to, and then read those tokens back as part of generating the next tokens.

Why does the tokenization matter? Also, ChatGPT doesn't "see" those numbers, it just sees smaller binary strings, and a human could add that series of binary numbers with knowledge of how to carry the result of the most significant bit.

Also, just to put this criticism to rest, I redid the addition example using space-separated bits and verified the tokenization was one token per bit (but again, it shouldn't matter).

    Let's add the two binary numbers provided, limiting the work on each bit to a single line:

      1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0
    + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1
    --------------------------------------------------------------
      1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1

The answer is still wrong. (It also didn't use a single-line per bit, but that's ok).

I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too.

Most humans can do it with a pen and paper once told the rules. But you are giving them tools (and purpose designed methods) to bridge the gaps in humans own tokenizer. Who knows what the equivalent of pen and paper would be for an LLM.

> I'd argue extremely few humans could do this in their head, which is essentially the bar you are holding GPT too.

The prompt was "Now add 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 and 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1. Limit work on each bit to a single line." So the model could use its own output to show its work. Again, the rules are really simple and a human could do this trivially.

All I'm saying, is that, to me, it looks like ChatGPT isn't capable for understanding logical steps. There are many, many examples online of people creating complicated logical puzzles that ChatGPT "solves." My hypothesis is that ChatGPT isn't actually "solving" these and ChatGPT would fail on any follow up examples that had sufficient entropy. To me, this is a significant difference. You might create a simple model of a problem and be amazed that ChatGPT seems to work, only to find that it fails with more complicated problems encountered in a real-world environment.

That doesn't mean ChatGPT is useless, but it would be limited to automation purposes (operating on known examples) and could never be a general thinking agent.

The reason humans can do it is because you give them a pen a paper so they can do it one step at a time. You are asking chatGPT to do the whole thing in one step.

Give it pairs one at a time and ask it to output the binary value and carry value. You'll see it can do it just as easily as a human doing it step by step with pen and paper.

As far as I know, GPT does the work one token at a time. It produces a single token at each iteration. So if you give it "abc" and want "def" as the result, you first give it "abc" and it returns "d", then you give it "abcd" and it returns "e", then you give it "abcde" and it returns "f", then you give it "abcdef" and it returns the stop token.

So just by outputting its work, it can provide its own context to memoize what the answer for each bit is.

Reply on news.ycombinator.com