Outrageously Large Neural Nets: Sparsely-Gated Mixture-of-Experts Layer (2017)
arxiv.org
arxiv.org
I can’t tell if this is primarily intended to provide effective sharding of a largely homogeneous network, or if it’s intended to allow for incorporation of diverse networks and use the gating to classify and route the inputs to the appropriate networks.
I'm very interested in working on using this same sort of conditional computation to make reasonably sized neural networks easier to understand. Has anyone seen papers on this sort of work?
Tools like LIME[1] have been around for a few years, and work really well for inspecting and understanding the decisions made by NNs.
It's true that complex decisions are hard to explain, but LIME does a better job than (say) anyway of trying to explain a SVN+Human coded feature.
LIME has been out since 2016. It and similar techniques are widely used in production systems.
It's time this "NNs are hard to understand" idea died.
After reading papers like this it feels like while we make some progress towards understanding some aspects of NN operation, we uncover deeper aspects that we don’t understand. This process itself looks like gradient descent (saddles everywhere!), and we are still pretty far from the global minima. :)
But have you ever tried explaining a SVN decision to an actual human before?
I'm not convinced that a NN is any worse than that.
Yes, Joel's about to say, but Jarvis is back in spew mode. "No shit. It was running the subway system over there, perfect operational record, and then one day it just forgets to crank up the ventilators when it's supposed to. Train slides into station fifteen meters underground, everybody gets out, no air, boom."
"These things teach themselves from experience, right?," Jarvis continues. "So everyone just assumed it had learned to cue the ventilators on something obvious. Body heat, motion, CO2 levels, you know. Turns out instead it was watching a clock on the wall. Train arrival correlated with a predictable subset of patterns on the digital display, so it started the fans whenever it saw one of those patterns."
"Yeah. That's right." Joel shakes his head. "And vandals had smashed the clock, or something."
But for a system you don't understand you are at whim of cascading patterns of errors in underlying behavior.
1. A cpu you rely on is a well understood system yet unexpected failures do happen (eg pentium bug, spectre exploits, etc).
2. A human you rely on might fail unexpectedly (tired, drunk, heart attack, going crazy, embracing terrorism, etc).
After testing reasonable number of things, if NNs perform more reliably than those other systems we rely on currently, it will be increasingly harder to justify using the other systems, especially when that means more people accidentally dying every year.
For 1 - well, actually CPU's are not as well understood as they might be. There are verified CPU's and stacks, but the pace of Moore's law and the cost of verification has meant that the performance gaps between them and the commodity chips has been huge. In 20/30 years time I expect that that gap will be small and we will see investment in fully verified stacks for many applications.
If the argument about accidental death reduction were true then wouldn't we have abandoned the use of personal cars and instead be insisting on public transit only? I believe that deaths per mile travelled on rail is far lower than for automobiles (in the US - which appears to have the worst record for rail one death per 3.4 billion passenger-km, for roads one death per 222 million passenger-km, that's an order of magnitude.)
[https://pedestrianobservations.com/2011/06/02/comparative-ra...]
To paraphrase some reviewer, "whenever I feel my will to live growing to strong I read some of this guy."
Edit: Buy the Firefall compendium. Just do.
There are countless examples in the public domain of AI systems improperly discriminating on race and gender when processing job applications, aiding medical diagnoses or targeting ads for housing and career opportunities. These outcomes can vary from annoying to morally questionable to explicitly illegal, and are all the more common when you don't really understand what your model is doing.
Statistical inference is important, and it's hard to get right. Centuries of thought have been put into methods to explain the effect of X on Y, accounting for Z. It's not the same goal as maximising the AUC or minimising the MSE. In many cases too, it's far more important.
Why was I denied car insurance? How come I didn't get past the initial algorithmic screening phase of that job application? If you can't answer those questions properly you're in legal trouble in many jurisdictions.
So going back to the original comment, that's why people are trying to understand neural networks, and why statistical inference and not just raw predictive power is important.
While validation is a proxy for this trust, it's not the same because the modeling may be incomplete or we don't have the same evolutionary heuristics to understand aberrant behavior. So we tend to want to interpret the model explicitly.
Certainly recurrent networks in general are not obsolete, even if attention/convolution works better for some applications.
Perhaps one ought to try GRU before LSTM but there’s no reason to suppose that it would dominate in all cases.
https://arxiv.org/abs/1906.01603
This is also obvious: Without recurrence you cannot remember information that is not externally visible, but it may be computationally very convenient and often necessary to maintain information that is hidden.
The hard part is learning reps for hidden information as recurrences are plagued by vanishing and shattering gradients.