Also your point 3) isn’t quite correct either, often a “standard” architecture and training procedure (e.g. ResNet50 with Adam) will work on a new task with sufficient training data and minimal modification of the model.
Most of what the article talked about can be done with much simpler models, which is what I get peeved about.
Also, yes, you can transfer learn with resnet. But if I throw my bank statements at it, it’ll do bugger all.
Similarly, if I throw new images at resnet in a silly way, it won’t transfer properly.
The development of neural networks is a major contribution of the machine learning community, so even if you’d like to split hairs about whether the “computer is learning” (“learning” has a a precise technical definition by the way), NNs are not “just statistics.”
I never said anything about the term machine learning. Check my bio, see what I’m working on. Fully aware of neural network contributions.
I’m all for machine learning. Just not “AI”. “AI” is hype bullshit.
“Learning” when used by the people who spout this BS is not the technical definition version, and is what I was referring to.
Could probably have made that clearer, but I’m 1.5 days without sleep.
What does feeding test data into a network yield? Inference results. Inference seems vaguely familiar from probabilistic modelling?
Bayes rule applies to neural nets too. Two different models may give vastly different results. Whilst they can be very good approximators, they can also be very unreliable if care is not taken during training.
G(x) ~ f(w.f(w.x+b)+b) is literally a fancy weighted sum. A linear regression. It is some easy stats combined together with a few other things that aren’t explicitly necessary, eg activation function can be identity to cancel out f().
EDIT Both the parameters of a network and the training data are variables in the application of Bayes rule. Which inherently deals with likelihoods (probability). /EDIT
So at their fundamental, they are “just some stats” stuff. They may have a few more bells and whistles to make them complex (and better) systems, but they still output a classification/regression based on inference.
You can, of course, approximate many functions with them. I’ve built a network with only weights of +1/-1, for example.
But those examples have extremely specific use cases that are not applicable to anything the article discusses.