I would say it was discovered, not invented. People were messing around with some algorithms, intruiged by their results. Eventually researchers discovered that with using certain training algorithm with certain data can lead to really wonderful outputs. But this is pure empirical discovery.

No AI researcher from 2010 would predict that transformer architecture (if we could send them the description back in time), SGD, and Web crawling could lead to a very coherent and useful LMs.