I think a better simplification would be to remove the position-encoding altogether, and just discuss transformers as set models.

I'm talking about bringing up a specific outdated details, not about simplifying it