Artificial Intelligence Is Setting Up the Internet for a Huge Clash with Europe
wired.com
wired.com
"Artificial Intelligence" => algorithms using lots of user data
"the Internet" => companies using this data like Facebook, Google
"Europe" => recent EU regulations
"Huge Clash" => a bit of a dust-up
Bowing down to the techno-gods and expecting them to make better decisions than we do has serious ramifications when the decision making is literally inscrutable. Right now the conversation seems to be hovering at the level of "shut up you luddites, and accept it." That doesn't mean the system should be trusted. In fact, it shows the opposite by illustrating those who would use the systems have nothing but contempt for those they would use the system on.
A simple example of how an inscrutable system could be considered benign is making it well-constrained and doing a failure modes analysis. But the question becomes: is the analysis preformed? thorough? and are the results acceptable? For things like self-driving cars: of course. It'd have to be to get on the road. When the failure modes aren't both blindingly obvious and unquestionably unacceptable to the consumer? "lulz, move fast and brake things!"
I fear the only way to improve the systems thinking around machine learning at fly-by-the-seat-of-your-pants companies are things like the EU regulation forcing the issue. That's a bit depressing, isn't it? Hopefully the level of discourse can be improved, even though the providers of such technology are incentivized to paint resistance as simpletons that just don't get it.
[0] Bruce Schneier explains better than I can, but with focus on security https://www.youtube.com/watch?v=KWD2v0SJAM8&feature=youtu.be... through 30:30ish
Also, given that such code bases tend to be masses of closed code, who performed the tests or audited them?
EU regulations on algorithmic decision-making and a “right to explanation”
Bryce Goodman, Seth Flaxman
We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for machine learning researchers to take the lead in designing algorithms and evaluation frameworks which avoid discrimination.
From the regulation:
Article 11. Automated individual decision making
1. Member States shall provide for a decision based solely on automated processing, including profiling, which produces an adverse legal effect concerning the data subject or significantly affects him or her, to be prohibited unless authorised by Union or Member State law to which the controller is subject and which provides appropriate safeguards for the rights and freedoms of the data subject, at least the right to obtain human intervention on the part of the controller.
2. Decisions referred to in paragraph 1 of this Article shall not be based on special categories of personal data referred to in Article 10, unless suitable measures to safeguard the data subject’s rights and freedoms and legitimate interests are in place.
3. Profiling that results in discrimination against natural persons on the basis of special categories of personal data referred to in Article 10 shall be prohibited, in accordance with Union law.
In this case, it doesn't necessarily mean there will be a HUGE CLASH. We will either turn to more interpretable AI, or continue developing more and more interpretability techniques for deep learning, which has been an area of very active research in the recent years.
[1] https://www.cs.princeton.edu/picasso/mats/Book_Schapire.pdf page 664
Yes, in general anti-discrimination law in the US, a disparate impact along a protected axis usually means that a decision must be justified on some neutral grounds. If there is no disparate impact attached to -- not just no express input related to -- some protected axis of discrimination, then, no, the outcome doesn't need to be explained.
Now, a self-driving car probably would be...
I bet this conundrum never crossed the minds of any of the people writing the regulation.
I expect it has, but they feel the ability for people to have an explanation for why they were denied insurance/credit/a job is more important than allowing companies to use processes they can't explain to make important decisions about your life.
You don't need the data. You need the model. And regulations like these are already in effect in the US. In banking, for example, it is prohibited to use protected classes (like race for example) or attributes that highly correlate with them, in your models.
The comment I was responding to was talking about training sets, I riffed on that. Depending on your definition of "full explanatory power", the model itself might very well not be enough, especially in the case of neural networks. Could you take a set of weights in a 5-deep neural network, look at an input vector, and have any kind of intuition about the output? It could be that there's some really interesting research that I'm unaware of here, please let me know if there's something I'm missing.
You'll need additional descriptive statistics at the very least, combined with some stochastic meta-models to relate a given record to the "whys" of their output. It's doable without access to training sets, but convincing to us and convincing to a lawyer reading the letter of the law regarding "all inputs to the decision" are 2 different things.
Speaking from experience, there is a massive shift away from ANNs in decision making. Companies (esp. in marketing) drank the Kool-Aid of some data scientists but figured out that they aren't all that useful. Turns out people care a lot more about inference and are willing to take a hit on prediction in order to be able to interact with the model. I'm not too familiar with ANNs but from the talks I've seen, I'm not even convinced they perform any better than other models.
So from what I've seen, ANNs have very limited application area which does not overlap with decision making, so I don't think there is a problem with them. And if there is, just ban them in decision making. That's not being luddite. We ban all sort of technologies because they are not appropriate.
For marketing optimization, FWIW, my experience is the opposite. For a big enough application, any 0.5% improvement is hugely welcome, as long as it works reliably and you're not just gaming the metrics (or, to be honest, even if you are, yay corporate).
Neural nets trained on personal data may be illegal wholesale, now or very soon.
Illicit NNs will then become VERY valuable.
As Isamu quotes from the paper: https://news.ycombinator.com/item?id=12079343
> We summarize the potential impact that the European Union’s new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user-level predictors) which “significantly affect” users. The law will also create a “right to explanation,” whereby a user can ask for an explanation of an algorithmic decision that was made about them. We argue that while this law will pose large challenges for industry, it highlights opportunities for machine learning researchers to take the lead in designing algorithms and evaluation frameworks which avoid discrimination.
Autocompleting a field is not something that feels like it would fall under this. Getting rejected by all your banks for a mortgage because they all use the same prediction company would be.