AI Failures in 2017
syncedreview.com
syncedreview.com
"Facebook chatbot shut down" doesn't really show anything. It was a everyday failure of training that got overblown in the media.
"Google AI looks at rifles and sees helicopters": This wasn't new at all. Black box attacks have been around for a while. The paper purported to show a much more efficient attack, but it's not really a huge advancement.
In general, most of these aren't really what I would call "failures" of AI. Some of them are illuminating parts of ML that are problems (copying society's bias, adversarial attacks), but most of the rest are simply things being overhyped/regular design errors (anything to do with Alexa/google home).
Overhyped marketing (Face ID, Las Vegas self-driving bus, HSBC voice id, "AI imagines a Bank Butt sunset")
Questionable design choices (Amazon Echo, Alexa)
Adversarial attacks ("Google AI looks at rifles", street sign hack)
Bias (Allo turban emoji)
wtf is this (Facebook chatbot)
ML does have real shortcomings, but the problems in this list does not exemplify them well.
Aka : If a gun could be mistaken for an helicopter just by altering a few invisible pixels, then the behavior of that system is by definition not similar at all to « vision » in its common sense. And so if it’s not, then what is it ?
Humans do have the advantage that our vision system can draw on a huge repository of real-world knowledge and has been fed with decades of high resolution training data.
My impression was that ML image recognition systems have exactly the same to draw on, non?
> Some of them are illuminating parts of ML that are problems (copying society's bias, adversarial attacks), but most of the rest are simply things being overhyped/regular design errors (anything to do with Alexa/google home).
It's a personal opinion that adversarial examples will always exist (they exist for pretty much all kinds of machine learning models), and that they might exist for human vision too.
Then they are not really AI failures “of 2017”.
More like “AI problems that are still unsolved”, at best.
It's good to be on watch for the bullshit (the news stories about Facebook's chatbots' "language", most futurist's predictions, any hype about general artificial intelligence, replacing x% of jobs in y years), but don't throw the baby out with the bathwater.
IMO, unlike blockchain, AI/ML is undoubtedly here to stay, and it will have a huge effect on everything.
I don't think it's a bad thing to try and apply new solutions to old problems but it just feels that we're trying to apply it too much to problems that are otherwise already solved.
That doesn't mean defense contractors, for instance, won't keep winning contracts based on that PR/marketing, while causing who knows how many innocent lives to be taken because of them over-stating the effectiveness of their AI.
That's just one example where AI is already "real" today, in the sense that many companies and government organizations have started deploying it and causing real harm to people because they think it's much better than it really is.
Other examples include anything from the AI being used in the justice system to AI "simply" being used to censor "porn" but censoring completely unrelated things in the process.
Speaking of which, there's a study by some RAND Corporation researchers (described on RAND's blog [0] and here [1]) about how it is likely a good idea to get the technology out even before it is perfected, not only to save lives now, etc. but also to speed up the perfection of the technology -- the rubber needs to hit the road, so to speak.
[0] https://www.rand.org/blog/articles/2017/11/why-waiting-for-p...
Ah yes, simple as that!
> a BBC reporter’s twin brother was able to access his account by mimicking his voice. The experiment took seven tries.
That's actually extremely impressive!
https://en.wikipedia.org/wiki/Category:Aviation_accidents_an...
AI and software engineering probably have a long, long way to go before they are in the same category of dependability.
Didn't someone point out in the thread for that, that backing up would've technically been breaking the law?
It's weird if you think about it, but from a logical standpoint, whether or not you broke the law is often undefined.
For example, what if a light breaks right in front of you and is now stuck on red?
Self driving sensors are better than eyes, but it is possible for there to be a situation where the car can’t know if it is safe. This probably has been thought about more by the designers than by all armchair critics combined, of course, but even if not then the law is the lowest common denominator of backside-covering.
But yes, engineers probably have thought about this, but there’s an infinite amount of stuff that could happen which is impossible to control for. What about falling trees across the road? Dirt roads that’s way behind on maintanance that would require you to drive off road a bit?
I’m not saying I believe AIs can’t handle this, but they would need room for improvisation to do it. When an entirely new situation occurs humans make decisions, argue in court, and then the ruling becomes precedent.
I remember a few years ago we were arguing about the fact that it's not a good idea to have a society where 100% of the people are punished for breaking the law 100% of the time, because then you can't have any progress. Who's to say the version of American society today is the the pinnacle of a modern society and that no law should ever change?
However, for change to occur, some people first need to break the law, and begin to make that new thing a culturally accepted thing before there's a strong enough movement to support a law change.
But if the AI automatically sees your first instance of breaking the law and then automatically punishes you for it, how is that change ever going to come about?
These things won't just "work themselves out" if by that you mean anything but people viciously fighting for their rights against an AI-totalitarian state over the next few decades. Just like AI today can sometimes make a mistake of seeing a helicopter instead of rifles, we'll need to keep "updating" the AI along the way to include these rights, and hopefully before too many people have to suffer in the process. Hopefully that process will be much shorter than the time it takes today from a law being passed by Congress to being shut down by the Supreme Court when found inadequate.
If you want a future-proof law, you should start from first principles and not use a path-dependent current laws, where selling alcohol is ok, but selling heroin is a crime.
This exploit isn’t practical as it requires 3D scanning the victim’s face. Similarly the touchID exploit isn’t practical so no one cares.
This method is much more secure than passwords that also get exploited by cameras or just brute forcing the 20% of pass codes that are “00000.”
When bouncers let people enter a bar with a photo ID that does not match the person in question they are not failing to identify a human. They are failing to give enough of a shit to carefully examine the picture in question.
My neighbor has an identical twin and one day his twin came visiting, but I had no idea he was coming. He was standing in front of our intercom that’s been broken for years looking bewildered when I got home from work, and since I’d heard about my neighbor’s identical twin before it didn’t take me long to piece together what was going on, even though I’d never met this guy before and I had no idea he was going to be there.
Pretty sad if you ask me, and - perhaps - it will never get better as it is tougher to tell a person by their voice rather than their looks.
And in any case, remember children: biometric data is user id and not password!
If that is the company's claim then certainly it was a failure.
If the HSBC voice ID has a similar failure rate for random voices than it is as secure?
You know why I love machine learning? It's immune to anti-intellectual fads. It speaks the same truth that children do. While you can make an algorithm spit out the answers you want, you have to explicitly train it to do so, making the selective blindness and hypocrisy we demand crystal clear.
I am really curious though, how did sexual selection help us evolve our brains?
Smarter mates were, or were able to make themselves appear, more attractive?
To me it's fairly evident that sexual selection played a large role in brain development.
https://www.psychologytoday.com/articles/201109/the-incredib...
What he calls formal verification is what we would call regular math in machine learning. Deep learning is sorely lacking hard bounds for all sorts of things (generalization, etc.). It's something that's gotten substantially better over the past year, but is something that needs a lot of work.
PAC guarantees are very different. They generally rely on:
- Data proportional to 1/eps^2
- Only reduce the “variance” component of the “bias + variance” (you’ll never fit a linear model perfectly to a nonlinear dataset, regardless of PAC)
- Get worse as you decrease bias
- Assume your data is IID and identically distributed to the test data. In TFA, several of the examples of bad behavior come from data which is distributed differently from training data (adversarial examples or automated cars not moving out of the way)
What I initially meant by "hard bounds" was any kind of mathematical proof more rigorous than "well, dropout kinda makes your neural network not rely on one feature so that's why it generalizes".
As for your points, I don't think they're really criticisms of PAC bounds.
I'm not familiar with the first point, but it'd be surprising to me if most PAC bounds had that, considering PAC is a framework and not a specific technique...
Your second point is irrelevant to generalization. You're looking for theory about capacity I think? I think learnability also comes into play.
3rd: that is indeed what the bias variance trade-off would imply. It's also why most classic PAC bounds are vacuous for neural networks.
4th: I think that's a fair assumption to make for any meaningful study of generalization.
For the common case of "The model misclassified a data point, no idea why.", formal verification doesn't help, but that doesn't make it completely useless. Probably not worth the effort, though.