Machine Behavior Needs to Be an Academic Discipline
nautil.us
nautil.us
The experts need to justify their position by coming up with more and more reasons that people need to listen to them. People who are actually doing the work quickly become convinced that said experts are ignorant asses who don't know what they are doing. This automatically sets up grounds for conflict.
I agree that the ethics of AI is important. However I expect the best work to come from people who are experts in AI first, and only secondarily in ethics. And I expect that the self-appointed experts in ethics won't recognize this. (Particularly not when faced with anything that makes their position seem less important.)
Thus we get quotes from them like "overpopulation on Mars"
I'm trying to picture if the government went around to all highly qualifying AI PhDs asking them if they would like to join a prominent ethics panel to guide and aid in AI ethics regulation in industry. So, you're claiming they're going to decline that or accept and sabotage under the assumption that they're a. all members of a giant AI consortium and b. any ethics regulation is going to destroy their industry?
As bad of an example as this is - look at other industries. Banking officials in government who set banking policies come from the finance industry. While this certainly has its problems and how they get there comes into question, they actually understand the industry they're regulating.
We already have an ethics discipline. This is an ethics problem.
Do you also propose that for example self-driving cars accidents should be investigated by AI PhDs which worked on self-driving cars algorithms, and they should be the ones proposing regulations? Are you saying that a regular "car accident specialist" is ill equipped to handle these, and it's opinions aren't as valuable because he's an expert in "human car accidents" not "AI car accidents"?
There is a legal term for what you are describing - it is called "conflict of interest". No, I do not think professionals should be involved in making decisions in matters in which they have a conflict of interest. Not all AI researchers are stakeholders in Uber, Waymo, or Tesla, hence you would select those people to investigate, as you would analogously do in any other matter of human affairs.
In fact, I'll go a step further and say that an "AI specialist" should be working alongside a "car accident specialist" and "laywers". I am arguing that this is a better combination than "Ethics specialist", "car accident specialist" and "lawyers" to investigate the accident.
Think if this would work in finance: asking JP Morgan people to investigate the Barclays Libor scandal. Smart people understand that a scandal against one of their peers will taint them also.
Do you hear today Google/Amazon/.... or other big private data hoarders makeing noise against Facebook? The one which spoke, Apple, doesn't have and rely on private data as much.
I would say that a serious ethics panel would have opposed the way Facebook handles private data, in a way that engineers working on them don't. Because engineers didn't study ethical/political problems of the past, and they don't understand how being lax with this can cause harm.
BTW, in all universities, doing studies on humans/animals requires first getting approval from an Ethics comitee. Googling now a bit I see that they have people from all faculties - Engineering, Humanities, Law, ... So universities, which give those AI PhDs consider that it's valuable to have a diverse ethics panel, not just one composed of Engineering professors if they meet to discuss an engineering experiment ethics.
Further, comparing the reactions of large companies with divergent interests to scandals is only tangentially relevant; that does not imply anything meaningful of the bias of hiring autonomous individuals from divergent backgrounds in AI; they do not all work on autonomous cars or <sell your data> at Big N. Finance has different social conventions as well.
"engineers didn't study ethical/political problems of the past, and they don't understand how being lax with this can cause harm" - I would wager that the average CS professor at a top school is significantly more well-read in history, ethics, and many other subjects than the "AI Ethics major" being proposed. But okay, let's set up an AI McCarthy panel with "experts", we'll get bored of Trump scandals on TV eventually.
This sounds like the jokes of when a plumber comes in to fix the pipes of a physicist, and the physicist is like "Whats so hard here? It's just basic physics"
Wait, what? To become a professor you have to spend the vast majority of your time teaching and/or studying/producing research in your specific discipline. How do you reach the conclusion that someone who doesn't exclusively focus on a topic would be more well-read on it than someone who does?
As btilly is referencing above, you already have a clear example of what type of people this field will attract: people with poor intellectual ability and zero understanding of inefficient, large-scale bureaucracies, basic probability or statistics, or software systems, painting simple organizational ineptitude as deliberate malfeasance and racism with the intent of bolstering themselves and their social group financially. It's clear there is an issue and it needs to be addressed as a growing pain, but deliberately training moral police with power over the software industry may lead to something like a dark age in technology.
Context, please? :)
It is such a distant threat and yet such a focus, and all the while we sit in our lovely apartments 40 minutes from nuclear incineration.
the conflation of machine learning and AGI causes confusion yet again..
https://nickbostrom.com/papers/survey.pdf
>We thus designed a brief questionaire and distributed it to four groups of experts in 2012/2013. The median estimate of respondents was for a one in two chance that high-level machine intelligence will be developed around 2040-2050, rising to a nine in ten chance by 2075. Experts expect that systems will move on to superintelligence in less than 30 years thereafter. They estimate the chance is about one in three that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity.
I said that people whose expertise is in being experts on what others should do are going to not do a good job.
Let's take a concrete recent example.
http://time.com/5209144/google-search-engine-algorithm-bias-... decries how Google search results are racist and are biased against black girls. It is written by someone whose background is clearly not machine learning, but who qualifies as an expert in the ethics of machine learning. She has a PhD, tenure at a well-regarded university, and a book on the subject.
While her article complains about the results of machine learning algorithms, and how horrible they are, she said nothing that showed an understanding of how they actually work. Why they produce the results that they do. Her diagnosis is that Google is a racist and sexist company. Her proposed solution is that Silicon Valley companies should be forced to hire blacks, and particularly black women, for the express purpose of keeping companies accountable for their racist environments.
The truth is that Google's internal environment is neither racist nor sexist - quite the opposite in fact. The actual problem is in the data available to Google of what people click on. How you can modify the algorithm to be slanted towards socially acceptable results is a tricky problem that requires a CS background. How you can modify the incentives so that companies are motivated to produce good results is an even more tricky problem.
Yet if she got to create the regulations, what would happen? We would literally get a requirement to hire politically correct thought police!
How well do you think that would work?
What she did say is that it's not surprising that Google search produces biased results, given that minorities are underrepresented and topics like ethics are barely discussed in engineering schools.
It sounds to me like she is attempting to raise awareness of a systemic issue in the tech industry, which requires no qualifications other than knowledge of the issue.
> We need people designing technologies for society to have training and an education on the histories of marginalized people, at a minimum, and we need them working alongside people with rigorous training and preparation from the social sciences and humanities. To design technology for people, without a detailed and rigorous study of people and communities, makes for the many kinds of egregious tech designs we see that come at the expense of people of color and women.
So perhaps not a direct call to hire minorities, but a direct call to hire people versed in identity politics and various non-white studies.
- The company is bad because it doesn't hire enough black women.
- The company needs to hire people versed in identity politics to critique what they are doing.
- Programmers should be required to take the kind of course that she teaches.
- Programmers should be only allowed to work alongside the people versed in identity politics who can tell them what they are doing wrong.
In short, this is a power grab. "They done wrong, so people like me need to be given authority to make sure that they don't continue doing it."
That may seem like an extreme example, but it is along a natural progression for people whose profession is figuring out what should be complained about. Everyone wants to believe that what they do is important. That they should be given more power and authority. What that means in practice varies by profession. But that is always the tendency.
Also, why assume that Machine Behavior specialists would not be not AI experts?
but I also don't think someone trained in CS, who has a deep understanding of machine learning systems is necessarily equipped to understand on their own the problems their systems can cause when they're out in the real world, being used and manipulated by real humans. It's just a completely different domain.
so if we say only CS researchers can speak credibly about the ethics and consequences of AI systems, then the work simply won't be done at all, or else will be done poorly.
this is why I think the article's call for an interdisciplinary field is reasonable -- in the current world, most CS researchers blithely ignore consequences, and most people who care about social impacts of these technologies do not adequately understand how they work.
If an attempt is made by people with only a lay understanding of the subject, the work will be done poorly for a different reason. And the poorness of the work will be something more readily apparent to those in the field than not.
Of the two necessary skills to intelligently comment on the subject, CS is the harder one to acquire. Which is why I believe that the best work will come from those who studied CS as a primary, rather than those who studied it as a secondary.
Nobody is going to call their friendly neighborhood ethicist until they are at risk of losing something. Rogue companies like Uber will only be checked when there’s a credible threat of being sued into oblivion. Employees of rogue companies will only act if they have exposure.
For instance I was designing an app that would identify if the food had an allergen or not. Eventually I came to the realization that my program if deployed could actually hurt people. So I stopped working on the project. I wrote a blogpost about this (shameless plug) at https://medium.com/@zitterbewegung/making-computer-vision-sy... . The core issue was I was giving people information that they could perform medical decisions. This gave me a bad feeling and a bunch of lawyers told me when I referenced this post.
I eventually pivoted what I learned into this (yet another shameless plug) : https://steemit.com/twilio/@zitterbewegung/mms2text-let-your...
which doesn't have the issue of harming people.
Without knowing if his app was accurate, we cannot say whether building or not building the app was the right decision.
Also, I have a nut allergy so I have a complicated set of procedures to figure out if nuts are in the dish (that was the main motivation for the app).
ML algorithms work on statistical performance against loss functions or error rates. They aren't (yet) good at understanding the difference between a mistake that causes a missed dessert and a mistake that might kill you. Maybe they can guess correctly a higher percent of the time if shown flashcards, but that's small consolation from the hospital bed. They also aren't that good at the limits of their own knowledge, i.e. saying "I don't know".
A set of standards that automated algorithms should adhere to? What kind of standards are we talking about?
A new branch of philosophy that talks about the actions of automated algorithms? What would be the point? How do you actually turn that into something that progresses, and that people will act on?
More ethics classes in universities for CS students? How do you differentiate this from what's already happening? Is it just broader?
---
To try to steelman this a bit, I figure there are two issues at play.
The first is making sure that AI is working in the way we designed it to; it's free of biases, it doesn't endanger people, it adheres to laws. We're doing horifficially on this metric when it comes to long-term superhuman AI alignment stuff, but most people are talking about next-decade issues. Those are raw technical issues, and though we're still working on robustness and detecting biases I really cannot see much need for external guidance outside of the natural technical progress that the field is already heavily invested in. These are problems we want to solve already.
The second issue is use of these technologies as tools. This is where we talk about how large companies' algorithms affect public perception, or how automated militaries affects warfare, or how self-driving cars are litigated. These are not AI problems, they're social problems. Yes, the technology behind those examples looks kind'a similar behind the scenes, but these are distinct social, legal and economic problems.
This is kind of like seeing the advent of electricity, predicting its effect on society, and asking for people to study "ethics of electricity".
the second problem is one where social scientists could benefit from actually understanding how these systems work. yes, they are social problems, but the sociologists could do much better if they had a better understanding of the details of the technology.
The authors say they want "a consolidated, scalable, and scientific approach to the behavioral study of artificial intelligence agents in which social scientists and computer scientists can collaborate seamlessly". So, a new interdisciplinary category.
I get the sense they are especially concerned with the empirical study of behavior and social impact of big ML systems in the real world. In another post I compared this sort of thing to sociologists studying transit infrastructure.
Could you be more concrete about how you would like other fields to contribute?
> the second problem is one where social scientists could benefit from actually understanding how these systems work [...]
> The authors say they want "a consolidated, scalable, and scientific approach to the behavioral study of artificial intelligence agents in which social scientists and computer scientists can collaborate seamlessly". So, a new interdisciplinary category.
I don't really agree. You don't need to know how electricity works to study its societal effects, and the societal change from electricity-augmented manufacturing is a completely different problem to the health and safety regulation of indoor sockets.
The same is true for AI. You don't need to know what backpropagation is to work on the long-term ramifications of automated warfare, and that in turn is a largely irrelevant discussion for someone figuring out whether a company's hiring algorithms are racially biased. There is neither a clear need for top-down regulations to be grounded in the minutia of the systems, nor any obvious advantage of grouping these discussions under one umbrella.
There is a need for the disciplines already dealing with these problems to pick up more specialised knowledge as the systems get more common, but that is a far cry from what seems to be argued in the article.
In the world of fair machine learning, "What are the right criteria for fairness under [set of circumstances]?" is usually not easily answerable, and I don't think we'll find satisfactory answers without more involvement from non-CS researchers -- both people in economics, law, and the humanities who are sort of generally concerned with fairness in society, and people who know a lot about specific domains where systems are being deployed. This in particular seems like an issue of "machine behavior" as described in the article.
With regards to transparency/explainability of models, there's a problem of making sure the "explanation" is actually useful and intuitive to the user, where psychologists (and HCI people who are already in CS depts) may have a lot to contribute.
In both cases there is a little ad hoc communication and there ought to be more.
> I don't really agree. You don't need to know how electricity works to study its societal effects, and the societal change from electricity-augmented manufacturing is a completely different problem to the health and safety regulation of indoor sockets.
I think electricity is not the best analogy, because it has a really subtle theory that few people understand, but we're so familiar with its use that many of its interesting properties are too obvious to see. It's also not at all autonomous -- it is often fruitful to think of a specific machine learning system as an agent, whereas this doesn't make much sense for electricity. Most importantly, I think it's just broader and more general than machine learning (trivially so!).
> The same is true for AI. You don't need to know what backpropagation is to work on the long-term ramifications of automated warfare, and that in turn is a largely irrelevant discussion for someone figuring out whether a company's hiring algorithms are racially biased.
The commonality here is you care about understanding or predicting the empirical behavior of machine learning systems interacting with the real world, especially with humans. ("Long-term ramifications of automated warfare" might not qualify, but I think medium-term ramifications certainly could.)
I don't think CS researchers are trained or particularly interested in empirical studies of human behavior, the stock market, etc., nor should they be, so somebody else will have to help. That somebody had better know enough about CS to be able to collaborate with actual CS researchers, though, or the results are going to be poor.
There is nothing particularly machine learning specific about this. If I want to design an AI to detect bank fraud, you're right that I want to do cross-disciplinary research in its design, but I do not understand what AI ethicists would add to that.
My disagreement is not that AI will involve itself in other fields, and in the process we need to learn about those fields. It's with the idea that either those people should be talking about the raw technology or that there should be a general field about how AI specifically relates to the sum of every other field.
> The commonality here is you care about understanding or predicting the empirical behavior of machine learning systems interacting with the real world, especially with humans.
Which is just the technical domain of AI research. Narrowing it down to the commonalities has removed all of the interesting points we were going to study!
I agree that when AI is added to the social systems or the stock market, we need to involve people versed in social systems or the stock market. I still do not see why they cannot collaborate in exactly the same way that they have already. The only difference visible to me is that that the hammers are bigger.
i think machine learning systems are different, and any researcher in a new field of "Machine Behavior" would have to know much more about how they work than in the railroad example. but there's no reason to suppose that work done by people trained outside of CS departments will be nonsense.
and as the authors say, there are real blind spots where CS researchers might just ask the wrong questions entirely.
Yes -- and a good sociologist would also learn a lot about that stuff once they started their research, I'd imagine :). Not enough to be a railway engineer, but presumably enough to be able to speak meaningfully & intelligently to railway engineers. Not unlike science journalists.
why?
Or do you mean that study of some humans is generalizable to most/all humans, whereas machines lack this consistency and generalizability?
Anyway, why would this concern prevent "feature detection" tests that apply to all machines?
Obviously, like any other tool, AI systems are part of the purview of anyone studying how society works (and as a particularly complicated tool, deserve a particularly extensive examination.) But at this point, studying the ethics of a machine is like studying the ethics of a fruit fly.
What do people make of this claim? Is that a fair comparison?
Even with simpler algorithms, you might argue that in a complex environment they could have some emergent behavior that's hard to foresee based on the source code. For example, if one is not careful then even simple linear regression on some datasets can "learn" techniques like redlining -- using zip code as a proxy for race as a part of biased predictions.