Hacker News Reader

Comment by bonzini | Hacker News Reader

Do I understand correctly that it's brute forcing a small grid rather than learning the algorithm?

> it's brute forcing a small grid

If by small grid you are referring to the attention matrix plot shown, then that is not a correct interpretation. That diagonal-like pattern it learns, is 3x3 convolution, so it can compare the neighbours of a given cell.

Edit: and note that every grid it is trained on / runs inference on is randomly generated and completely unique, so it cannot just memorise examples

My interpretation is that while it did learn the exact computation and not just a statistical approximation, it's still limited to a grid of a given size. In that sense the attention matrix is brute forced and the network did not learn a generalization. The article itself says "The largest grid size we successfully trained was 16x16".

They're using learned positional embeddings for each grid cell, so there's no straightforward way to extend a model trained on a small grid to a larger grid. If you go from large to small, I think it would do better than chance, but get the periodic boundary condition wrong, because the period changes with the grid size.

Using 2D RoPE instead would in principle allow scaling up as well, and maybe even period detection if you train it across a range of grids, but would eventually hit the same issues that plague long-context scaling in LLMs.

Yes, it does not understand (or has learned) the rules of the game then. For that being the case, it should be able to apply the rules correctly in a slightly different context.

Would it be possible to train an LLM on the rules how we would teach them to a human?

Reply on news.ycombinator.com