This gets repeated by the cynics every time like prayer, but it shows a very poor understanding of the current state of research.
If you actually follow the literature, you'll find that there is tons of evidence that the seemingly "simple" transformer architecture might actually work pretty similar to the way the human brain is believed to work.