Hacker News Reader

Comment by valine | Hacker News Reader

It also completely ignores the developments around GPT tool use. If you’re evaluating the AI field as a whole, why are you restricting your analysis to the raw output of GPTs? Arguably the most impressive display of intelligence exhibited by these models is their ability to use tools like calculators to augment their own abilities.

It doesn’t matter if the model can’t do three digit multiplication without access to a calculator. Most people can’t mentally multiply large numbers either.

View on HN

Most people can do 32-bit binary addition using a pen and paper. And the point of using addition as an illustrative example is that the rules of addition are extremely simple, yet a binary addition problem can easily be scaled to N bits of entropy. This means it's very easy to show that the system fails to apply simple logic rules in such cases.

It didn't fail logic, it failed to read your string. You didn't even notice that the terms being added in the 'show working' version don't match the given strings. And you are elsewhere talking about adding spaces to make the digits 'tokens'... you have no idea what you are talking about at all.

You can input the space-separated strings into OpenAI's tokenizer: https://platform.openai.com/tokenizer

You'll see that it does properly tokenize each binary digital as a separate token.

But again, the tokenization doesn't matter. I could say to you, please add these two binary numbers:

      101 0011 11001
    + 00 11 10 100011

And you could do it.

If your goal is to simulate a human doing addition with pen and paper it only seems fair to give the GPT read/write access to a text file. Outputting the answer directly from the model is much more akin to mental math.

ChatGPT does have read write access to a text file. It's allowed to show its work as part of its output, which it can write tokens to, and then read those tokens back as part of generating the next tokens.

Reply on news.ycombinator.com