Transformers can easily be trained / designed to handle grids, it's just that off the shelf standard LLMs haven't been particularly, (although they would have seen some)
For LLMs we only have one axis of position and - more importantly - the vast majority of training data only is oriented in this way.