Yeah, I guess this is fine if by "Transformers are Graph Neural Networks" we mean Transformers < GNN, rather than Tranformers == GNN.

"Sentences are a fully connected graph". Ok fine, but that's a graph with basically no information embedded in its structure. GNNs are supposed to be useful for graphs that have interesting structure, right?