Hacker News Reader

Comment by jszymborski | Hacker News Reader

Worth noting some of the criticisms of the methodology that are being made on Twitter

https://twitter.com/jeremyphoward/status/1662687099685044225...

I believe "Can an LLM do X?" is often the wrong question. One should ask "How much prompting does it take to get it to do X?", and more specifically "How specific to the task does the prompting have to be" and "How close is the mental effort to create the prompt to just solving the task".

And in this case clearly GPT-4 can do the task with minimal prompting effort.

Building off of this, "how much experience do I need in task X to be able to use this LLM for task X effectively?"

Yeah it feels like the issue is that the researchers expect the AI to be smart enough to solve nonsensical Python puzzles but not smart enough to figure out that they are nonsensical and do the sane thing instead.

Having said that, I did try telling ChatGPT explicitly it is a puzzle and not normal Python code, and every time it explained that it was a trick because the builtins were swapped... and then gave the wrong answer anyway.

Definitely an interesting case.

I think the issue is just that, however silly or pointless the puzzle is, it's clearly understandable by any human who knows Python. You just use "len" any time you mean "print", and "print" any time you mean "len". And these large language models, despite being language models, simply cannot handle this concept.

Going to paste together snippets from comments on this thread.

This bit is from the paper.

> Typical programming languages have invariances and equivariances in their semantics that human programmers intuitively understand and exploit, such as the (near) invariance to the renaming of identifiers.

This is bullshit lol. Identifier renaming may be a no-op for the code, or even for the compiler/interpreter, but it's absolutely a meaningful change for human programmers. If it weren't, we'd all be calling functions named f0001, f0002, f0003, ... etc. to save space. Like us, LLMs process many different associations at the same time. A function named "print" has a lot of associations that together reinforce the understanding that it will make something the output of the program. Renaming it into len() makes the meaning go against all the association that go with that word.

Swapping print() and len() to use in the same program? That would trip any human up, it's tailor-made to be difficult for us to process. And, as it again turns out, so it is for LLMs, because they really seem to process associations the same way we do (at the gut/intuition/first reaction level).

Language Models can absolutely handle the concept. Proof is all over this thread. They just handle it very similarly to humans.

The problem here is, "scientists" believe GPT models couldn't possibly be in any way exhibiting forms of thinking or intelligence similar to humans, and so they assume they must be doing some specific kind of computation humans are not, despite every interaction with ChatGPT being evidence to the contrary. They design a test that a human would not pass (despite their claims), but their imagined specific computational system would - and then act all proud and mighty after ChatGPT fails it too. The only real insight from this event is: the authors of this paper imagined ChatGPT to be something it isn't, then demonstrated it in fact isn't it, and think they've discovered something.

That's just shifting the goalposts. Someone gets LLMs to do something impressive? "Ooh, LLMs are intelligent!". Someone shows LLMs falling flat on their face? "Lol of course LLMs can't do that you're using it wrong".

I'd also like to call out the preposterous claim that a human would not pass the example test illustrating the article. While in the normal course of coding and if they hadn't noticed the swap a human programmer could well be tripped by it, presented with the snippet of Python code in the Introduction of the article, most human programmers would have no trouble giving the correct answer.

Because as humans, they understand what the code says and can use that understanding to find the correct answer even when it doesn't agree with their expectation. Humans can guess, but they can also reason. LLMs can guess. The question is whether LLMs can reason. This article shows one case where they can't.

Sure that's the posts that are being shifted here.

If the same model can do exactly what you say it can't with so little effort added that you can do it minutes or hours within your paper release then the model can get very much do what you say it can't.

There's nothing inherent about falling flat on your face that makes it some quality reserved for beings that can't reason. Humans do that fine on their own al the time.

A paper that doesn't even pass the try rest is a paper that should be questioned at the very least. Invalidates all the claims it makes when the results it's founded on are patently false or misleading.

> That's just shifting the goalposts. Someone gets LLMs to do something impressive? "Ooh, LLMs are intelligent!". Someone shows LLMs falling flat on their face?

It's still intelligent! It may have weird failure modes that are different to typical human failure modes, but intelligence isn't restricted to entities that never get anything wrong.

@TeMPOraL and @og_kalu are the same people right?

I'm not lol. Which is exactly why I explicitly say I'm posting comments that have already been posted.

No, we're not.

> Swapping print() and len() to use in the same program? That would trip any human up, it's tailor-made to be difficult for us to process. And, as it again turns out, so it is for LLMs, because they really seem to process associations the same way we do (at the gut/intuition/first reaction level).

I would simply write the code I intend to write, then execute three replace operations in vim: %s/len/temporary/g, %s/print/len/g, %s/temporary/print/g.

It's not tripping me up at all. Maybe I don't think the same way that these models do?

Reply on news.ycombinator.com