What matters is the percentage accuracy. A black box with a 10% failure rate is better than a fully explainable system that fails 20% of the time. Explanations make us feel better and they can be very important. But for most cheap, repeated processes they aren't necessary. Not to mention that neural networks can be tested and interrogated in ways that other systems cannot.
That's not true at all, it depends on the use case.
What actually matters is the desired percentage of acceptance.
For many critical path use cases you'd much rather have something fail twice as often but understand why it failed so that you can correct the issue and resubmit the input. Error observability is an important feature that's taken for granted in many systems. It all depends on what the system is used for—how important it is to be able to get to correct results, and what the consequences for failure are. The biggest danger of neural networks is in people that don't understand this nuance and apply them in a blanket way in all systems.
old adage: if a bug can be reproduced then it's only a matter of time before it's understood and fixed. If a bug can't be reliably reproduced (Heisenbugs) then repair time is unbounded.
(that said, humans are perfectly capable of creating inexplicable and irreproducible bugs - for example, in multithreaded code)
Much more so than fixing the 10% of a neural net.
There is no way to tell the Tesla vision NN, "hey, when you see this pattern and you're confused about which path to take, it is better to take one incorrectly than it is to run into a concrete divider". We know exactly what the problem is, but there is no interface with a NN to tell it to do something, other than to just keep training it with more data. And once you realize that your only interface to get better outcomes is to wildly manipulate the training dataset, then you haven't made software engineering better, you've made data engineering worse.
Take notice of something important: all of the domains where Neural Networks have been wildly successful are domains that are wildly underspecified. Take language for example. Grammar rules, vocabulary, pronunciation, and even meanings of words are constantly changing. There is no possibility of ever having a formal definition of any language, let alone all of them. Or vision...where the only formal definition of anything is what color of light it reflects in a particular angle. Again, no formal definition of anything.
But the shortest path from A to B? That problem has a formal definition, and no neural network has even come close to the accuracy of A star or Djikstras. The minimum cost solution to a Multi-Commodity Flow Problem? That problem has a formal definition, but no Neural Network has come close to the accuracy of a Simplex method's solution.
Tautological arguments about percentage accuracy might give the edge to Neural Nets in some domains, but not all of them, and for that reason they completely miss the point.
1. Percentage accuracy is only half of the thing that actually matters. Without the cost of being wrong taken into account, percentage accuracy will totally fuck you over hard. Here's a game: you can choose between two algorithms, one that has 90% accuracy with a 10% chance of smelling gross, or a 99.99% accuracy with a 0.01% chance of your body been shaved down to bone over a thousand cuts from a vegetable peeler. Which would you choose?
2. Sometimes absolute accuracy matters. We have formal systems for absolute accuracy. We have symbolic logic for absolute accuracy. We have deterministic systems for absolute accuracy. If the best that I can get from a neural network is a percentage accuracy, then it has already failed a test of general applicability. How many years and how many computers and how much data would we have to feed into how big of a neural network in order to get to E = MC^2 with perfect accuracy?
3. Even if percentage accuracy matters, time-relative accuracy matters even more. With a Neural Network, if you need to get better accuracy, how do you do it? You should see actual machine learning practitioners try to solve these problems. They literally try to deconstruct the black box, trying to figure out how different neurons are weighted, and what input data can be altered to result in a different weighting. It's a clusterfuck, and it slows progress to a halt. We've known exactly what was wrong with the Tesla Vision NN for over half a decade, but actually fixing it has completely stalled because of the fact that it is a black box, and can only be fixed like black boxes. This is systems theory 101: you can't fix systems that you can't understand.
If the "actual machine learning practitioners" can break down behavior to the neuron level, then I'd say they understand it to some degree.
The reason that many of these problems still exist is because chasing down individual errors is a seductive waste of time. If you have to tell the network "here's how you handle this one pattern and here's how you handle this other one", then you're building an expert system. Yes it's tempting to correct errors as you see them pop up. But it's better to construct a network that can learn from data to handle any situation that you didn't think of specifically.
Building a robust network requires lots of time and data. There will always be edge cases that cannot be fixed individually. The fact that Tesla or anyone else hasn't built a system for X with no embarrassing edge cases yet does not mean that we should go back to coding individual instructions and conditionals line by line.
“Yes it’s tempting to save and invest your money for your retirement, but it’s better to put your faith in god and he’ll solve all your financial problems, even the ones you didn’t know you’d have.”
If the system, with all of its data, can’t solve a common problem that happens every day, how the hell is it supposed to solve a problem so rare that the engineers don’t even know exists?