Hacker News Reader

Comment by bigDinosaur | Hacker News Reader

Asking it to convert 32 to binary was a bit of an eye opener. It gave me multiple completely incorrect ways of doing so (resulting in 0100 or something each time) despite me informing it it was wrong. It'd say things like 'the 24th place, then the 23rd place are 0's which just made no sense in terms of the query (I suppose technically correct if you really want to pad left) - but I had to think about it because it was saying superficially plausible things (like '24' seems somehow relevant because it's 8+16, but of course that's totally irrelevant to the actual question).

That said in other respects it is pretty amazing, so I'll definitely not dismiss it, it can replace Google for a fairly decent set of queries so long as you know the subject matter well enough.

View on HN

Most LLMs are crippled at tasks regarding character level manipulation because of how the inputs are tokenized (words and numbers are split into common substrings rather than the original characters, 101 -> “10” + “1” vs “1” + “0” + “1”).

If you tried this binary conversion task with a model that takes in raw bytes (for instance, ByT5) it would perform almost perfectly. In GPT’s case, there’s a trade off between having a long context window vs being good at character level tasks, and OpenAI picked the former.

I tried to use it for help with a wordle. I think the word was EJECT and I had E?E?T. It seemed to understand what I was asking, but came up with 5 and 6 letter suggestions that didn't fit.

Frankly that's like asking a calculator to write you an essay. It's simply not the right tool for the job if you want to do mathematics. It doesn't actually understand mathematics or logics, it just knows patterns in its training data and how to apply them to input. It can go through the motions of answering your question but it can't know whether the answer makes sense.

I think no matter how much breathless media outlets and futurist fanboys like it, we need to stop pretending conversational AI is "generalized AI" just because it can perform language. It's a fully automated Dunning-Kruger machine. That's amazing a lot of the time, but it's a specialized tool with specialist applications.

I just wrote this comment in copilot (rider/c#):

// convert 32 to binary

copilot suggestion: 00000000000000000000000000100000

Then I tried:

int val = 32; // convert 32 to binary

Auto-generated:

binary = Convert.ToString(val, 2);

Reply on news.ycombinator.com