AI scientists are studying the “emergent” abilities of large language models

I've always loved emergence. The entire universe is founded upon it, from physics to chemistry to biology to neuroscience and genetics, and beyond. There's a field of math called complex systems analysis which basically just studies emergence, and if I were more mathematically capable, I would definitely want to make that a source of income. Whenever people say something is "too complex" to be chance or physics, I'm just like, "no, it's not; emergence happens all the time". I would even consider emergence a fundamental property of the universe: scale anything up enough, mix enough simple bits together, and you get something complex and more capable than the sum of the parts.

I've been following transformer-based AI research quite a bit since the first announcement of GPT-3 (though admittedly not as much as some people, for sure), and these emergent spikes have excited me. Especially when you consider that a rat neuron is nearly identical to a human neuron, it's only the number of neurons and the connectivity that separates a human brain from a rat brain. (By analogy to AI, that'd be the number of parameters and their learned values, respectively.)

Emergence: it's everything, everywhere, all at once :)