Hacker News Reader

Comment by skepticATX | Hacker News Reader

That's not really true though. LLMs are abysmal at planning, for example. Something that comes quite naturally to humans.

View on HN

They're really only abysmal if you attempt to one-shot it and probe with tasks that would require a human a scratchpad to accomplish.

Humans can't one-shot non trivial planning tasks either. It's the one problem i have with all the papers that try to evaluate planning for LLMs.

Step away from that approach and they're ok.

https://innermonologue.github.io/

https://tidybot.cs.princeton.edu/

I'm curious as to your source, or particular examples, since they (or at least GPT-4) seem to me to be rather decent at planning. E.g., for writing code.

Ask it to write a backtracking sudoku solver with coroutines and/or fibers and let me know how it performs in your language of choice.

We are nowhere near generally intelligent software systems.

I was curious where GPT-4 would come up short on the problem and I was surprised -- it seemed to solve it pretty well whether or not using coroutines. (I dropped both solns into a python interpreter and both appeared to solve the problem.)

There could def be bugs I missed tho.

https://chat.openai.com/share/ef77507e-cb75-4112-97f1-a16cfc...

That's a good attempt but the coroutine solution is incorrect. See if you can figure out why and how to improve it. You can also ask it to propagate constraints and see what happens.

This isn't the dunk you think it is since GP is a human (I presume), and he thought the solution worked.

Sigh classic LLM -- without you, the expert, I can't quickly tell from the code / output how the answer the LLM produced is wrong. I also asked it to solve sudoku by "propagating constraints" and the answer seemed to work for me :/ Again, I'd guess the soln produced is wrong because I trust you more than the LLM but I don't have the mental horsepower to figure it out without resorting to tests & debugging.

If you are happy with the tool then continue using it. My point was a simple one, any generally intelligent software system would have given you an optimal solution by accounting for what people know about backtracking and constraint propagation but the solution presented did not account for how to optimally use coroutines to propagate constraints.

I'll repeat what I said previously in a different way, LLMs are useful but they are nowhere near what is required to achieve generally intelligent software systems. I'm sure they will continue to improve as engineers and companies learn how to utilize them in their workflows but let's temper the hype a little bit because statistical autocompletion is not enough to achieve general intelligence.

Ask a random college freshman in computer science to write a backtracking sudoku solver with coroutines and/or fibers in their language of choice. Oof, only 1 out of 10 can do it...? I guess 9 out of 10 college freshmen aren't generally intelligent? (I mean, I certainly wasn't very intelligent as a college freshman, but I'm pretty sure that's not the way you're using that word here).

Anyways, these kinds of strawmen always baffle me in regards to AI.

Insert random pseudo trivia that most of the general population wouldn't be able to do, see the AI fail at that specific task. "What did I tell you? The AI definitely isn't generally intelligent yet!"

Everybody is out to prove AI isn't intelligent without first defining what intelligence even is. And when other people rightly point out it can do a lot of stuff, they point to some specific task that it gets 90% of the way there but doesn't get perfect and then triumphantly declare AI isn't intelligent. Crazy.

I think everyone should have a test in their back pocket for testing claims made by companies making AI tools. Mine is backtracking sudoku solver with constraint propagation but that's because I know LLMs are incapable of recursion and backtracking. Usually people get the point I'm making but sometimes I have to elaborate further.

LLMs are cool toys but calling them intelligent is stretching the definition of "intelligent" way too far. It's important to be clear about what the words actually mean because if people start thinking these software systems can be substituted for their own thinking then we end up with all sorts of unnecessary confusion around what they're actually capable of achieving.

Any test a lot of humans will fail is simply not a test of General Intelligence. That should be obvious enough.

And GPT-4 does do this in a couple iterations.

Humans are not generally intelligent.

Then you have a wildly different bar than the vast majority of the people in any agi conversation and should state this upfront.

What exactly do you want me to state up front? A general intelligence can solve general problems when those problems are formally specified. Sudoku is a good example of a formally specified problem that can be solved by most people but not by LLMs. This is because LLMs and all neural networks are simply DAGs of function which do not support recursion or backtracking.

Adding recursion to neural networks has been tried a few times but no one actually knows how to stabilize their dynamics so the industry has settled on feed forward networks with constrained function blocks which have stable dynamics with respect to back propagation of errors.

I don't understand your question.

> What exactly do you want me to state up front?

Exactly what was in your comment that I told you to state up front.

Most people in a discussion about AI and replacing people are working with a definition of general intelligence that includes humans to a large degree.

> This is because LLMs and all neural networks are simply DAGs of function which do not support recursion or backtracking.

Without adding external state I can't solve a sudoku puzzle.

Ok great, next time I run into you I'll make sure to remind you that large language models are not intelligent as I conceive of the word "intelligent".

If you're saying it's not perfect, then sure: it required one iteration before the Python program it spat out worked. It also when asked gave a 2-page description of the code.

And how many humans can do this? Those poor exhausted goalposts.

I have strongly believed the best way of evaluating the state of AI is to look at what people complain it can't do.

It's taken a few short years to go from "keep a coherent story over more than a sentence" to "write a multithreaded sudoku solver in one shot".

Many programmers would fail to do this. I'm not even sure a randomly selected human would understand the question. And I'm wondering if you've setup any loops letting it write tests, search the internet, inspect and debug? If not what you see as an output is essentially it whiteboarding off the top of its "head". Programmers routinely fail to solve simpler things in interviews, and their skills are much narrower than current llms (you can easily argue deeper, but I feel comfortable saying broader).

> write a multithreaded sudoku solver in one shot

I did not say anything about threads but I did hint at the fact that coroutines can be used creatively to solve problems which require backtracking and constraint propagation. The fact that this is still controversial means a lot of people are unaware LLMs do not support backtracking, recursion, and constraint propagation. Sudoku is just an obvious example of a problem that is easy to solve if you know about these concepts and next to impossible if you don't. Such problems can also be expressed as integer programs but even fewer people know how to do that so I don't usually bring it up but it would be another good test for any software system that is claimed to be intelligent by the corporate marketing department.

Llms don't have to support something internally to know about those concepts, those are entirely different things.

> Such problems can also be expressed as integer programs but even fewer people know how to do that so I don't usually bring it up but it would be another good test for any software system that is claimed to be intelligent by the corporate marketing department.

It's very telling if the level of testing is "can it do something few people in the field can?".

What exactly are you confused about? Plenty of people can learn about constraint propagation and integer programming but LLMs do not have the capability to solve integer programs because LLMs (and neural networks in general) are not generally intelligent.

I'm not confused I'm saying you're conflating two things - understanding something and being able to perform it in a single pass. Neither llms nor humans have to be able to solve a sudoku puzzle to write code that can. I can be awful at mental arithmetic and write code for a calculator.

Ok, let me know if you can get an LLM to solve sudoku puzzles by prompting it with information on backtracking and recursion.

No prompting on backtracking or recursion, that was chosen by the LLM. All I did was tell it to break the problem down and reason about it, then afterwards asked it to write the code. Examples were added by me at the end, from a random sudoku site for "easy" and "hard".

https://gist.github.com/IanCal/9817f8b21b2ea6d77940966ee399d...

You misunderstood what I meant. Prompt the LLM with the sudoku puzzle and ask it to solve it and not write python. Human can do it, LLM can not.

They are probably better than 10% of people

You meant 70%, right?

Reply on news.ycombinator.com