GPT-What: Intro to the next big thing

> What are those 175 billion parameters?

> According to Geoffrey Hinton — the godfather of ‘deep learning’ — parameter can be thought of as a synapse in a human brain ...

> They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.

So his estimate is now off by a factor of about 100? How long will it take for the next couple of 100-fold increases in parameter count?