A new link to an old model could crack the mystery of deep learning
quantamagazine.org
quantamagazine.org
https://calculatedcontent.com/2019/12/03/towards-a-new-theor...
Using techniques from statistical mechanics and strongly correlated systems, we can compute the average-case-behavior of a real world DNN
We believe we can reproduce some of the results of the NTK by using a Gaussian Random Matrix. But if we use a more realistic, heavy tailed matrix, we get more practical results
A early fork of the theory has been published in JMLR https://arxiv.org/abs/1810.01075
and the empirical results in Nature Communications https://www.nature.com/articles/s41467-021-24025-8
and we have an open source tool , weightwatcher, which can be used in production
pip install weightwatcher
https://github.com/CalculatedContent/WeightWatcher
Please give it a try and let me know if it is useful to you
UC Berkeley / ICSI: https://www.youtube.com/watch?v=6Zgul4oygMc
Stanford ICME: https://www.youtube.com/watch?v=PQUItQi-B-I
and a couple of Mikes's
Institute for Pure & Applied Mathematics (IPAM) : https://www.youtube.com/watch?v=fmVuNRKsQa8
Physics Informed Machine Learning: https://www.youtube.com/watch?v=eXhwLtjtUsI
and our KDD workshop KDD: https://dl.acm.org/doi/abs/10.1145/3292500.3332294
many more talks and papers available
And my favorite, a podcast that has featured LeCun himself:
https://blog.rebellionresearch.com/blog/theoretical-physicis...
On the flip side, Nature often has bolder claims with less convincing evidence, which often reads similar to pop-sci articles.
In 2011 I was staggered to realise what a load of bollocks I had been bongoogled into believing in, and annoyed.
I am never going to fall for the "if we assume x, and y and only talk about this bit of the system then it's all the same" line again. It may be mathematically nice to prove things with simplifying assumptions, but the ICML community basically walked up a blind ally because of this desire for rigor and elegance in preference to reality.
ML is not mathematics, because it's tied to data, and data is a product of nature. The study of nature is Science. This is the difference between String Theory and Physics, and it's the difference between Computer Science and ML as well.
What do you (and parent) mean by "do"? Inference, or training, or both?
However, clever people (Hinton, Lecun, others v.smart and determined and all should get a lot of credit) invented some tricks to train networks faster (and carefully showed that they really worked, despite what people said about them, especially people at ICML), and computers got many faster, and it became obvious (which it was to anyone who both could do the sums and had the imagination to think about it - which I didn't, at all, sad for me) that broad networks are much much slower to train than deep networks for an equivalent number of neurons (if you use tricks like regularization and dropout) and this makes large deep networks feasible.
I did play with deep networks in like 1994, because I wrote a C program to let me write neural networks - and I could do anything I liked with it on my Sun workstation and my supervisors had no idea about anything. However, I set this "deep" network off training to see what would happen and went to the pub. When I came back I interrupted it and looked at the model file to figure out how much the parameters had changed since initialization (I didn't have any of the good small value initialization stuff, I think I set them to 0) and they had changed so little that I did some sums and decided that the model would not be finished training before I finished grad school if I finished at all.
So, I went off and built various types of booster on an MPI network that most of my colleagues didn't realize was running on their nice Sun workstations instead. Talking to other people in pubs at conferences has since convinced me that most of the grad students working on ML in the 90's did the same thing.
About thirty minutes later he came back and said something like: "Shit! That's all the stuff we are doing now!".
To which I said: "Yeah. Computers got faster and we have lots more storage and memory. Beyond that, I don't think we moved forward very much at all."
If you look though 30 to 40 year old literature on AI you will find pretty much everything you'd expect in a modern book on AI. Sure, yes, there have been advances, of course. I would challenge anyone to find 40 years of advances when comparing AI then and now.
The best hypothesis I can put on the table is that speed and massive storage made a range of applications possible over time. With those exhausted, we hit the wall because we are essentially working with 30 to 40 year old AI technology.
I don't think that there would be many games for mobile phones though.
Smart software engineers would not be dedicated to figuring out ways to get people to click on buttons or how to get people into virtual food fights for profit!
I can't even imagine what progress might look like if all those brains were put to better use.
I wanted to make a joke here that, sure you can, all you need is an infinite number of cat pictures and infinite time and compute, but actually you don't need that. Neural nets can do better at identifying cats in images than anything that has gone before and that has to be said.
The problem is that this better performance degrades precipitiously with any distance from the standard benchmark datasets that may contain cats. In the real world, with its infinite variation of not only cats but also cat poses, environments, lighting conditions, etc etc, it's a much worse deal and as I say in another comment, good luck knowing how well SOTA approaches do at it - we'll never know.
I think some researchers are refusing to accept the idea that machine learning is very much an experimental science today, and the (very cool) mathematics of kernels, SVMs, empirical risk minimization, bayesian statistics, etc. are simply no longer useful in the large scale regime.
My prediction is that there will be a useful "deep learning theory" in the future, but it will look a lot more like physics (such as Kaplan's scaling laws) than early 21st century machine learning mathematics/statistics.
As an example, just last week a (huge) paper [1] was put on arXiv that used these theoretical methods to analyze a bunch of common architecture building blocks (skip connections, normalization, etc), and then applied their theoretical findings to figure out how to train Resnet like models in similar training time without these seemingly "required" building blocks.
Deep Learning is still in its infancy in many ways, and this type of research takes time, slowly building on successive results.
team behind the paper is Deepmind/Google. It is probably worth a read.
The same statement could have been said of neural networks for a decades, but researchers poking at corners eventually found methods to turn them into the useful tools they are now. After all, those methods you downplay were created since neural networks were not useful at that time, whereas many of these were.
I'd not poo poo what researchers decide to poke at. Pretty much every breakthrough is people poking at the edges of understanding. If solutions or steps were straightforward, then it would be engineering, not research.
>My prediction is that t
Mine is that neural networks as we understand them now get replaced by much more solid methods, based on the principles from scientific machine learning, where sophisticated differentiable models that are designed to mimic the problem space get tuned. After all, even current neural networks are heading that direction. Neural networks are simply too simplistic to capture lots of the complexity that problems demand (hence the current move past them in many domains).
Gluing linear functions together ad-hoc is simply a low level approximation to what can be developed using centuries of powerful mathematics to make models.
Maybe so. But I suspect that scientific machine learning would be difficult to grasp by most working data scientists and ML users who don't have graduate-level training in a quantitative science or engineering, which implies an uphill battle for adoption. You can do a lot with ML and deep learning without to construct a sophisticated mathematical model of your problem space.
(I took one class in differential equations in 2004, touched up on that in a boot camp before starting a PhD in econ, and have probably forgotten everything I ever learned about the topic ever since. Why yes, I feel mathematically inadequate sometimes.)
Most of what those same people do now was difficult to grasp when they started. The next group will adopt whatever is the best tools for the problems they face.
>which implies an uphill battle for adoption
The same was true for current ML and data science. It was true for OOP. It was true for structured programming. This is always true, but tools that solve problems get adopted.
>You can do a lot with ML and deep learning without to construct a sophisticated mathematical model of your problem space.
True, but as such simple tasks are mined out, everyone doing much work will move past that. For example, pick any task on say paperswithcode.com and look at the top performing models. It's hard to find any that are simple old school networks. All of them I just sampled do much more sophisticated modeling of the precise problem space, and the trend is that all current edge and future work will involve modeling the problem much more carefully.
I expect the naive deep learning approach will be a tiny blip in the progress of using learning to solve problems.
Future progress looks like it relies more and more on making a good model of the problem with some untuned parameters, then using gradient descent or related to tune those parameters. The better the original model, the better it performs, and the less data one needs to train it since it has relevant structure already.
For example, learning basic physics in a net takes significantly more training, parameters, and cost than simply building in basic physics and tuning the small unknown parts. The net version also doesn't scale well since it is built from pieces (linear chunks) that simply don't match functional relations in the underlying problem.
This is like criticizing the physicists working on the principles underlying steam engines during the days when people were making empirical advances in building steam engines, but nobody had figured out all the core principles yet.
Of course they understand that it’s an empirical science today. That’s exactly why they’re doing the work they’re doing.
I respectfully but completely disagree.
All the enabling technologies of deep learning come from machine learning and statistical learning theory. Stochastic gradient descent, regularization, dimension reduction, bootstrap, bagging, boosting: these techniques remain the fundamental tools in the deep learning toolbox, and a constant source of inspiration for the latest innovations.
Physics has done next to nothing for deep learning in comparison. It's as marginal as the kernel SVM stuff. Just fiddling around with the same old stat mech / network theory / power law stuff the Complex Systems types been doing for the last few decades.
Stat mech works great for materials because materials are relatively simple things and we have relatively simple questions about them. We want to know how they respond to electricity, magnetism, heat, pressure, etc. When you turn to elaborate gadgets for machine translation or image recognition, sure, you can ask similar questions, but the questions just aren't as interesting. You'll get plenty of plots and histograms and power laws out of it, but it's all going to be very superficial. It's not going to tell you how the gadget works.
All that said, there's no way to be sure where the big advances in deep learning theory are going to come from. Your guess might be as good as mine.
I keep seeing this claim pop up but I've never seen a citation to support it. My understanding is that e.g. statistical learning theory lets you prove theorems of the form "IF it has this number of parameters or fewer, THEN it won't overfit". First year logic is enough to know this theorem tells you nothing in the scenario where the model has lots of parameters. Can anyone provide a solid citation to support the claim that models with many parameters "should" overfit?
BTW, another example of a model with lots of parameters which doesn't overfit is random forests / gradient boosting. I'm wondering if this "should overfit" claim is less of a global property of machine learning models in general, and more just a property of the particular models that this set of researchers have most familiarity with (SVMs).
1: https://www.microsoft.com/en-us/research/people/gregyang/
Here's a reddit thread by Yang discussing his research into why NNs are not Kernel Machines [1].
[1] https://www.reddit.com/r/MachineLearning/comments/k8h01q/r_w...
It seems to me that proving DNNs to be equivalent to SVM kernels won't improve them appreciably -- they won't become more accurate or precise, not faster, nor more economical. More importantly, the achilles heel of DNNs won't be fixed -- their inability to explain their reasoning or annotate their inner workings to enable tuning or repair.
Replacing black box linear algebra with black box math does not seem like much of a step forward.
Why should it have any other value?
If anyone has more insight into the actual idea, I would appreciate an explanation.
In the article, there are two advertisements to older articles. One is "New Theory Cracks Open the Black Box of Deep Learning" from 2017 and the other is "Foundations Built for a General Theory of Neural Networks" from 2019.
[0] https://www.quantamagazine.org/a-new-link-to-an-old-model-co...
SGD in a small number of dimensions is easy enough to understand, and as Hinton said in an online class I took years ago: to visualize very high numbers in dimensions, close your eyes and shout out the number. He was joking but his comment convinced me to disregard the concept of millions of billions of dimensions and instead concentrate on practical techniques that enable training many layers, architectures that are multi-headed, etc., and use model architectures that are known to work for different types of problems.
Tangentially, I don't believe GPT-N will be the way to achieve AGI, however we can learn much on said road travelled.
This is not looking to be the case right now.. Companies are doing quite well without explainable models. SDC companies like Waymo are hoping it will be sufficient to:
1. Produce a low failure rate estimate, based on millions of virtual miles/fake cities.
2. Combine several NN outputs in a logical way. That way there are at least some interpretable values that lead to the result. e.g. a separate model that produces candidate trajectories of surrounding vehicles, one to perceive sensor input, one to plan, etc.
Yes ?
> SGD in a small number of dimensions is easy enough to understand
Is it?
> Is there much mystery?
There is much mystery!
Why do some network architectures work better than others even with the same number of parameters?
Are all network architectures equivalent? In the sense of, say I have a choice of feed-forward convnet vs fully-connected vs MLP mixer vs transformer vs whatever else you want. Can I take my working transformer and get an equivalent convnet that trains the same way? Or is there something special about one or more of these architectures?
Are there non-network systems that have the same properties as deep networks? Maybe they're better?
What even is the space of non-equivalent models? Right now we basically just try stuff and hope to see some improvement. That's really unacceptable. If we had theory to tell us if two architectures are "the same" (there are many nuances here, just as there are when you are say comparing two Turing machines), we could talk about the space of all models.
When things work or don't work, is it because of some property of our optimization procedure or something inherent in the network initialization or architecture?
Why is optimizing some architectures so much easier than others? Transformers are great, but they're very unstable and tricky to train. As we add layers or modify a transformer, what do we do to update our training regime?
How much data do I need to train a network that does X? Can we predict this form some property of the task and the network? What do I know that I've saturated. If I've saturated what do I need to do to get improvements from more data?
Then we get to more serious problems. There are things that networks are terrible at, like long-range inferences and compositionality. Why is this? We have some fixes to some network architectures, but how do we generalize these?
And more broadly. We know basically nothing at all about recurrent networks. We thought they were useful, but then we essentially gave up on them completely with transformers. What's the deal? Are recurrent networks special? Are they approximated so well that they don't matter at all?
This all guides us toward a stranger mystery. We know the brain has recurrent connections. Actually, the visual system in your brain has more feedback connections than feedforward connections. Why don't we need to be able to build models that have these?
This just starts to scratch the surface about what's mysterious here. We're in dire need of some guiding theory, but it remains to be seen if this is it. So far theory has contributed nothing :(
This isn't true at all. You can only make this statement relative to a dataset. And VGG is almost always trained on ImageNet. A standard VGG16 has 138 million parameters. ImageNet has 1 million images each is 244x244 (~50k). There are ~431 input data points per parameter in VGG16. VGG is not overparameterized for ImageNet.
There are plenty of network/dataset pairs they could have given as an example, but this isn't it.
Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something.
To state your question more broadly. Data has structure that's why we can learn in the first place. Can we separately talk about the structure vs number of examples of the different structures in our data? In computer vision this would be sort of like talking about how much information is within an object class vs between object class.
We don't know. And we don't have the mathematical tools to talk about this today.
> Do we really have any sense of how "effective number of parameters" scales with network features? This article had me wondering if peoples' understanding of these things is backward or something.
It's not backward, it's non-existent. We basically try things and when we have too many parameters we see that performance isn't increasing (or isn't increasing enough to be worth it, or with unsupervised methods and massive datasets we just max out the size of network we can train).
I.e. total overfitting can be achieved by ignoring most of the input values and using just as many pixels from the top left corner as you need to uniquely identify the source image (which is far less than the total number of pixels) and map it to an arbitrary class, and for a million images that should be theoretically possible with million-ish parameters; i.e. no matter how many pixels there were in each image, 138 million parameters for classification of a million images is overparameterized from the classic perspective of what overparameterization is (some generalization forced simply because it's theoretically impossible to fit each sample individually).
I left the ML community in disgust at how unsavory these ML methods were.
I didn't expect HN to be basically so obscurantist and to act so similarly to medieval alchemists (and that's not being fair to alchemists).
So apparently theory is useless, all that matters is the "art" and empirical results ?
Especially since quite a few of the comments here dismiss this work, and related, with a quick "well it's trivial/ well known?" and then procede to give their own personal explanation that show a clear misunderstanding of the topic.
https://news.ycombinator.com/item?id=28817666
https://matloff.wordpress.com/2020/11/11/the-notion-of-doubl...
Can I filter this out?
So, what is meant is that such networks (let's call them complex networks for short) generalise well to test data. But what exactly is "test data"? Here's how it works.
A team has some data, let's call it D. The team partitions the data to a training and test set, T₁ and T₂. The team trains their system on T₁ and tests it on T₂. To choose the model to test on T₂, the team may perform cross-validation which further partitions T₁ to k validation partitions, T₁₁, ..., T₁ₖ. In the most common cross-validation scheme, k-fold cross-validation, the team trains on k-1 validation partitions of T₁ and tests on one of the k validation partitions, trying all combinations of k-1 training and 1 testing, partitions _of T₁_ (because we're still in validation, which can be very confusing). At the end of this process the team have k models each of which is trained on k-1 different partitions, and tested on one of k different partitions, of the training set T₁. The team chooses the model with the best accuracy (or whatever the metric they're measuring) and then they test this model on T₂, the testing partition. Finally, the team report the accuracy (etc) of their trained model on the testing partition, T₂, and basically claim (though generally the claim is implicit) that the accuracy of their best model on T₂ is a more or less accurate estimate of the accuracy of the model on unseen data, in the real world, i.e. data not included in D, either as T₁ or T₂. Such truly unseen data (it was not available _to the team_ at training time) is sometimes referred to as "out of distribution" data and let's call it that for simplicity [1].
So, with this background at hand, we now understand that when the article above (and neural net researchers) say that complex networks "generalise well", they mean "on test data". Which is mildly surprising given that various assumptions make it less surprising that they'd do well on training data etc etc.
There are two key observations to make here.
One is that complex networks generalise well on test data _when the researchers have access to the test data_. When researchers have access to test data, the training regime I outline above includes a further step: the team looks at the accuracy of their system on test data, find it to be abysmal, and, crestfallen, abandon their expensive research and start completely from scratch, because obviously they wouldn't just try to tweak their system to perform better on the test data! That would be essentially peeking at the test data and guiding their system to perform well on it (e.g. by tuning hyperparameters or by better "random" initialisation)!
I'm kidding. Of course a team who finds their system performs badly on test data will tweak their system to perform better on the test data. They'll peek. And peek again. Not only they'll peek, they'll conduct an automated hyperparameter-tuning search (a "grid search") _on the test data_! That is what's known in science, I believe, as a "fishing expedition".
Two, in the typical training regime, the training partition T₁ is 80% of D, the entire dataset, and T₂ is 20% of D. And when T₁ is partitioned for cross-validation, the validation-training partition (the k-1 folds trained on) is also usually 4 times larger than the validation-testing partition (the single fold tested on). Why? Because complex networks need Big Data to train on. But, what happens when you train on 80% of your data and test on 20% of it? What happens is that your estimates of accuracy are not very good estimates, because, even if your data is Really Big, 20% of it is likely to miss a big chunk of the variation found in the 80% of the data, and so you're basically only estimating your system's accuracy on a small sub-set of the features that it needs to represent to be said to "generalise well". For this reason, complex networks, like all Big Data approaches, are pretty much crap at generalising to "O.O.D." data, or, more to the point, estimates of their accuracy on OOD data are just pretty bad. In practice, deep neural net systems that have amazing performance "in the lab", can be expected to lose 20-40% of their performance "in the real world" [2].
Bottom line, there's no big mystery why complex networks "generalise" so well and there is no need to seek an explanation for this in kernel machines. Complex networks generalise so well because the kind of generalisation they're so good at is the kind of generalisation achieved by humans tweaking the network until it overfits _to the test data_. And that's the worst kind of overfitting.
__________
[1] Under PAC-Learning assumptions we can expect the performance of any machine learning system on data that is truly "out of distribution", in the sense that it is drawn from a distribution radically different than the distribution of the training data, to be really bad, because we assume distributional consistency between training and "true" data. But that's a bit of a quibble, and "OOD" has established itself as a half-understood jargon term so I let this rest.
[2] I had a reference for that somewhere... can't find it. You'll have to take my word for it. Would I lie to you?
So results on truly-unseen, OOD data are never published, if they are ever collected at all, and the field advances one meaningless published benchmark-beating SOTA result at a time. In the end, despite all the hype, nobody has any idea how well neural networks, er, work, in practice. We only know how well they do on test datasets.
Oh, alright. Some people who deploy large neural nets for google and friends have an intuition about how well they work. A ... feeling.