IMHO the prospect of an AI winter is 0%. As someone who has done research in ML I think ML technology is moving forward much faster than we anticipated. ChatGPT shouldn’t work based on what we knew. It’s incredible that it works. Makes you think what other things that shouldn’t work we should scale up and see if they would work. And then there are things that we think they should work. Each new paper or result opens the door for many more ideas. And there are massive opportunities in applying what we already have to all industries.

You can absolutely build high precision ML models. Using a transformer LM to sum numbers is dumb because the model makes little assumptions about the data by design, you can modify the architecture to optimize for this type of number manipulation or you can change the problem to generating code for summing values. In fact Google is using RL to optimize matmul implementations. That’s the right way of doing it.