And the key takeaway of the T5 paper was also how "instructions" could vary massively.

But the main instructions were two types - summarise x, and translate y to z.

Translation in many ways was the root of seeing that multiple tasks/instructions could fit a model and not just in the context of multiple training loss methods, like NSP vs Gap filling(a modus of training difference, not actual task itself).

And the special tokens for BERT etc trace their origins back to enabling the task above and positional embeddings(and encodings). Which largely trace their origin to translation work.

Yes, but the translation model in the “Attention is All You Need” was trained on parallel sentences (input: English sentence, output: French). LLMs such as Claude are not trained on any labeled data. Yet, they still surpass the special-case models.

Yes, it was primarly for translation. I don’t know how OP come to the conclusion that it was accidental

As far as I know, it wasn’t just primarily for translation — it was entirely intended for machine translation. It only because clear later on that it was a more generally applicable architecture.