Indeed, the two papers came out within months of each other iirc. The GAT paper discusses Transformers in the context of stabilizing the learning of attention mechanisms.

Of course, this connection may be trivial to most people, but I hadn't seen a post on this before. So I decided to write one for myself as I studied these architectures.

Maybe a better way to put it would be "Graph Neural Networks are a Generalization of Transformers". The formulation of "X are Y" indeed suggests that you could just have stuck with Y instead of using the special case X, whereas in reality they took Y and added something to make it more general.

Yeah but even without attention, they're pretty similar.

Sentences can be reasonable modeled as a mostly 1 directional graph.