The whole thing is essentially curve fitting. The ML field is essentially an art more than a science and it's all about tricks and intuitions on different ways of getting that best fit curve.
From this angle the whole field got way less interesting. The field has nothing deeper or more insightful to offer beyond this concept of curve fitting.
I think there are some really cool problems, such as:
1. Is synthetic data viable for training?
2. How do you make deep learning agents that can do task planning and introspection in complex environments?
3. How do we efficiently build memory and data lookup into AI agents? And is this better/worse than making longer context windows?IOW: to me, fitting a generalized linear model is very different than fitting a convolutional network.
From a technical standpoint, it’s not correct analogy either, because it assumes you have a curve to fit. What curve is language? What’s curve is images? No answer, because there isn’t one. Deep learning is about modeling complex behaviors, not curve fitting. Images and language for instance are based in social and cultural patterns and not intrinsic curves to be fit.
At best, it’s an imprecise statement. But I’d disagree entirely.