The core contribution of "Attention is All You Need" is logistic regressions

Is the tweet correct?

'Deep neural networks, as you probably know, are sandwiches of linear regressions with elementwise nonlinearities between each layer. The core contribution of “Attention is All You Need,” which led directly to the LLM/GPT explosion, is to throw some logistic regressions in there'