Comment by lisper | Hacker News Reader

Comment by lisper | Hacker News Reader

Por que no los dos?

Thirty-five years ago they gave me a Ph.D. basically for pointing out that the controversy du jour -- reactive vs deliberative control for autonomous robots -- was not a dichotomy. You could have the best of both worlds by combining a reactive system with a deliberative one. The reactive system interfaced directly to the hardware on one end and provided essentially a high-level API on the other end that provided primitives like "go that way". It's a little bit more complicated than that because it turns out you need a glue layer in the middle, but the point is: you don't have to choose. The Bitter Lesson is simply a corollary of Ron's First Law: all extreme positions are wrong. So reactive control by itself has limits, and deliberative control by itself has limits. But put the two together (and add some pretty snazzy image processing) and the result is Waymo.

So it was no surprise to me that Stockfish, with its similar approach of combining deliberative search with a small NN computing its quality metric blows everything else out of the water. It has been obvious (at least to me) that this is the right approach for decades now.

I'm actually pretty terrified of the results when the mainstream AI companies finally rediscover this. The capabilities of LLMs are already pretty impressive on their own. If they can get a Stockfish-level boost by combining them with a simple search algorithm the result may very well be the GAI that the rationalist community has been sounding the alarm over for the last 20 years.

I've been thinking of the issue like this: a traditional approach to symbolic artificial intelligence frames problem solving as a search of a tree. Do you go depth first, dodging infinite branches? Or breadth first (do you have enough memory?). What about iterative deepening?

Sometimes the solution is in the tree, but it is too deep, and one runs out of time before it is found.

Statistical based learning could act as a branch predictor. Sometimes guiding the search to go very deep in the right place and find the hidden solution. Sometimes guiding the search to go very deep in the wrong place; one runs out of time as usual.

Notice the strength of hybrid approach. One isn't accepting the probably correct answer of the statistical part. It is only a guide, and if the answer is found, and the symbolic part of the software is correct, the answer will be reliable.

I think this is already being done with maths problems. The LLM is writing proof attempts in Lean. But Lean is traditional symbolic AI. If the LLM can come up with a proof that Lean approves, then it really has a proof. From your comment I learn that Stockfish has already got something like this to work very well.

What you're describing — search + NN — is presently under the term "test-time compute".

The rules / dynamics / objectives of chess ( and Go ) are trivial to encode in a search formulation. I personally don't really get what that tells us about AGI.

No, I don't think that test-time compute is the same thing at all. It's a little challenging to find a definitive definition of TTC, but AFAICT it is just a fairly simple control loop around an LLM. What I'm describing is a merging of components with fundamentally different architectures, each of which is a significant engineering effort in its own right, to produce a whole that is greater than the sum of its parts. Those seem different to me, but to be fair I have not been keeping up with the latest tech so I could be wrong.

I think search is a fairly simple control loop. Beam search is an example of TTC in this modern era.

It is a very wide term, IME, that means anything besides "one-shot through the network".

I think the thing about the search formulation, which is amenable to domains like chess and go, but not other domains is critical. If LLMs are coming up with effective search formulation for "open-ended" problems, that would be a big deal. Maybe this is what you're alluding to.

> search is a fairly simple control loop

That's like saying that Darwinian evolution is simple. It's not entirely wrong, but it misses the point rather badly. The thing that makes search useful is not the search per se, it's the heuristics that reduce an exponential search space to make it tractable. In the case of evolution (which is a search process) the heuristic is that at every iteration you select the best solution on the search frontier, and you never backtrack. That heuristic produces a certain kind of interesting result (life) but it also has certain drawbacks (it's limited to a single quality metric: reproductive fitness).

> Beam search is an example of TTC in this modern era.

That's an interesting analogy. I'll have to ponder that.

But my knee-jerk reaction is that it's not enough to say "put reactivity and deliberation together". The manner in which you put them together matters, and in particular, it turns out that putting them together with a third component that manages both the deliberation and the search is highly effective. I can't say definitively that it's the best way -- AFAIK no one has ever actually done the research necessary to establish that. But empirically it produced good results with very little computing power (by today's standards).

My gut tells me that the right way to combine LLMs and search is not to have the search manage the LLM, but to provide search as a resource for the LLM to use, kind of like humans use a pocket calculator to help them do arithmetic.

> If LLMs are coming up with effective search formulation for "open-ended" problems, that would be a big deal.

AFAICT, at the moment LLMs aren't "coming up" with anything, they are just a more effective compression algorithm for vast quantities of data. That's not nothing. You can view the scientific method itself as a compression algorithm. But to come up with original ideas you need something else, something analogous to the random variation and selection in Darwinian evolution. Yes, I know that there is a random element in LLM algorithms, and again I don't really understand the details, but the way in which the randomness is deployed just feels wrong to me somehow.

I wish I had more time to think deeply about these things.

Did you name that law by yourself?

https://flownet.com/ron/papers/tla.pdf

>In retrospect, in the story of the three-layer architecture there may be more to be learned about research methodology than about robot control architectures. For many years the field was bogged down in the assumption that planning was sufficient for generating intelligent behavior in situated agents. That it is not sufficient clearly does not justify the conclusion that planning is therefore unnecessary. A lot of effort has been spent defending both of these extreme positions. Some of this passion may be the result of a hidden conviction on the part of AI researchers that at the root of intelligence lies a single, simple, elegant mechanism. But if, as seems likely, there is no One True Architecture, and intelligence relies on a hodgepodge of techniques, then the three-layer architecture offers itself as a way to help organize the mess.

Reply on news.ycombinator.com