Wouldn't that be an overkill to train a model specifically for that?
I wish someone could come up with an algorithm that would outperform a trained model. This way we would actually understand how it works.
I wish someone could come up with an algorithm that would outperform a trained model. This way we would actually understand how it works.
You could probably tweak the heuristic used by A* to the problem to get similarly good results (at much lower compute cost than running what amounts to an LLM with huge context length). But this is a toy problem. The interesting thing about the paper is that you might be able to get good-performing search algorithms for any problem with a cookie-cutter process, instead of spending many man-hours tweaking a hand-written search algorithm. It might also be possible to easily adapt it to much more complex search problems, like cooperatively evading other agents navigating in the same space.
I suspect that the actual execution time would be far faster with the A* implementation than any of the transformers.