two years ago it didn’t do anything that made me want to use it during work.

The data wall idea of needing 10 more internets is like the peak oil theory of the 2000s, imho. I don’t think 10 more internets are required, and all those tasks like proper security and i18n, none of these require creativity of any kind. These are ideal candidates for LLMs to solve quite soon.

But to talk about the data wall more: Current SOTA LLMs are trained on circa 30-50T tokens. First, that is not a full internet‘s worth; it is estimated that Meta owns more like 200T tokens of user data etc. Second, the work in optimizing for data efficiency is just beginning.