Speculations about Transformers and Compositionality

As a non academic this is an example in how to express a simple opinion with structure and references. My first reaction was to think this is much ado about nothing, but then I saw it provides a small map one can use to start to navigate this topic.