Hacker News Reader

Comment by bem94 | Hacker News Reader

Depending on ones level of idealism, the best defense against malicious AI is actually educating programmers / hackers / engineers on the implications of their work and to have a sense of decency & foresight.

It baffles me that we just accept the kind of malice inflicted on people by programmers because "someone will always do it". As a profession / collection of skilled persons, we should really be better than that.

Obviously, one cannot see the future, nor would we want to be paralized by fear of doing anything. But there is a certain minimum requirement for collective responsiblity which I really don't think we are meeting at the moment.

View on HN

Decency and foresight is highly subjective. Actually encoding what we understand as "decent human behaviour" into AI is a huge problem: If you create a system that decides about giving somebody a loan in the US based on a financial data set it might come up with any of these hypotheses:

* Don't give loans to people living in [poor area].

* Avoid people with names that aren't similar to the most common ones in the database (i.e. foreign ones)

* When linking the customer data to their social media and you see the picture is dissimilar to [preferred ethnicity], do not give a loan.

Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1]. It knows that poor people pay back loans less frequently and makes the rational decision to not give a loan to that group. This of course reinforces the problem.

But how are we to solve this? Introduce an additional column "race" to the data and bias the results with it? Would that not be just as racist? How do we give the system an awareness not to discriminate against ethnic groups, if the data contains implicit clues? This comes down to giving such an AI human intuition about such questions.

[1] http://www.ssc.wisc.edu/irpweb/faqs/faq3/Figure1.png

I'm not saying teach people some universal moral code, or suggesting we can accurately encode a set of appropriate morals into an AI. Just that people stop and think if what they are doing is appropriate.

It seems silly to take problems which humans cannot solve with all the data, intuition and humanity that we have, and then use that data to train machines to make decisions, we know will be flawed.

>Now without any malice from the developer, the system has become racist: It saw the correlation that in the US blacks and hispanics live in poverty more often that others [1].

If the data really suggests that these factors (poor areas, foreign names, social media) affect liklihood of default, should they be completely ignored? I'm sure credit scores and salaries are correlated to race in the US too. Should those be ignored? Just give out loans indiscriminately?

They're not ignored now by loan approval. Worse, inaccurate stereotypes are used as a proxy for some of these factors that may be relevant to a borrowers ability to pay. We don't live in a perfect world where people ignore certain factors.

Are you familiar with correlation vs causation? Without a causal link, making an inference based on correlation is unsound. Encoding unsound reasoning in an AI model, particularly when it reinforces an existing social imbalance, would be less than ethical.

"Weapons of Math Destruction" by Cathy O'Neil delves into this in more depth. It's a very valid concern, particularly in the way non-technical people are trained over time to give deference to the algorithm.

+1 for Weapons of Math Destruction.

I'm not sure what you mean. AI doesn't involve "encoding unsound reasoning" into anything. That's more akin to how humans think and act, by making decisions based on heuristics learned from a lifetime of experience and societal influences. It would be unethical to encode a "race" factor to be considered in an algorithm, although even if it were included I think it would prove insignificant since race has no direct causation to ability to prepay when you consider all the other factors (e.g. poor white person in same neighborhood would have same default prob). However other factors might, such as the ones you have listed (where they live, credit score, etc)

I am familiar with the book you mentioned.

You still didn't answer my question. If credit score and income are correlated to race in America, should they be ignored from loan applications? What are valid factors to consider?

> If credit score and income are correlated to race in America, should they be ignored from loan applications?

Is correlation causation? Is it fair to encode a judgement based on correlation but not causation?

Or to make your position more clear, do you support racial profiling? If not, why not, and how do you justify that position, and how does that justification apply to judgements based on correlation but not causation?

(My position is that making decisions based on correlation but not causation is the general case for which racial profiling is a specific case.)

I already answered that. No race should not be a factor as it does not convey any info not explained by other factors.

What are factors that are causation? Presumably not where you're living and other considerations you determined to be implicitly racist

You know, if you code your machine right, and give it the real information it wants (income level), an inference machine will detect that race is a worst proxy than the direct value and will not become racist.

It's when you start to deny proper information into decision making that those bad proxies start to look good.

But a correlation between income and race might be detected and used by the system as well, making it at least partially racist. Maybe no better than humans today but we should at least aim to make our AI not overtly racist.

Ethics can get pretty subjective, so I think it would be hard to teach a uniform view of decency. Also I hate to sounds pessimistic, but there are a lot of smart people who do much worse things than create malicious software when money is at stake. I think educating the people is a step in the right direction, but I fear it will fall short of your ideal.

>> I think it would be hard to teach a uniform view of decency

Ofcourse, but we don't avoid studying moral philosophy because we can't find a uniform model. Nor does not being able to find a uniform model mean that there are not things worth learning from it.

>> there are a lot of smart people who do much worse things than create malicious software when money is at stake

Agreed. Two wrongs do not make a right, and as engineers in the know, we have a duty to stand up to such people when we have the chance.

>> I fear it will fall short of your ideal.

Me too, but the alternative is worse in my opinion.

The problem is that if engineers in the US don't take the job then someone somewhere in, say, Asia will. Ethical rules are not the same everywhere. You can already tell this by looking at experimentation with stem-cells, cloning, et cetera, which is happening faster in Asia than in the US because of regulations.

Also, refraining from building these systems means that other countries get ahead of us. Besides being an economic disadvantage, this could also threaten security.

So I do agree that engineers should adhere to ethical guidelines, but they should be considered in a global context.

this, we need to wake up to the fact that China is willing to do whatever it takes to dominate. They have done massive research on genetics and intelligence and the top schools in china make parents take IQ tests to determine potential for children when accepting students. They don't care about diversity or equality, only results.

The US and Europe need to wake up or we'll be facing a billion hypernationalist, gentically modified super-geniuses while the only thing we have is the moral highground. Based on what I've seen over the last decade we should probably start learning Mandarin.

I'm learning mandarin and have a chinese girlfriend, fully ready for the takeover when it happens ;)

I don't think that's the best defense, because even if 99.999% of programmers are perfect, ethical, decent people, all it takes is one psycho to go ahead and let the genie out of the bottle anyway.

Imagine there's a big red "end the world" button that anyone on the planet can push. No matter how idealistic and utopian your views are, do you really trust that there's not a single person out of 7 billion people that might push the button? That is the kind of situation I worry we may face once people figure out how to create strong AI.

Turning what I think you are saying on its head:

There is a 0.001% chance of something bad happening, so there is no point in the other 99.999% changing their behaviour in order to stop that bad thing happening.

I don't think that follows either. Second guessing human nature to the point where you resign yourself to catastrophe is surely less worthwhile than striving to create a sense of collective responsibility before it is too late?

I'm not saying there's an 0.0001% chance of it happening, I'm saying that if you give enough people a chance to create an AI, eventually someone will do it, even if 99.999% of people would choose not to. Right now, nobody knows how to do this, but my point is that eventually people will figure out how: once everyone knows how, then it's just a matter of time before someone does it.

I'm not second guessing human nature, just saying people do crazy things all the time. The vast, vast majority of people are not suicide bombers, for example, but it still happens sometimes. It doesn't matter how much collective responsibility we have; eventually you may just end up with a brilliant programmer with schizophrenia or something who writes an AI in his basement and unleashes it on the world.

I'm also certainly not resigning myself to catastrophe. There are probably ways to prevent or mitigate the negative impact of AI, like the one being suggested in the original article, for instance. I would hardly consider myself a cynic, but forgive me if I'm skeptical of the idea that every human in the world can somehow agree to never do anything that might create a dangerous AI. If it can happen, it eventually will happen.

By that logic the gamma knife that has saved many people with cancer would not exist - before practical applications comes laying the basic foundation, and once that foundation is down, the genie is out of the bottle - I'm working on an autonomous drone, I am of course deeply opposed to weaponizing what I know is a ridiculously stupid system, but I know that it will happen - implication is not action, and knowledge is mostly neutral, it's the application of that knowledge that might be problematic - whichever way you slice it, it's a really complex issue, and anything worth having is often capable of both great and terrible things, depending on the application. AI is way too primitive to be malicious on it's own and will remain that way for years - now, as for abhorrent malicious uses of what we have now, here's a great example -- https://goo.gl/HmBY9k

Sorry that's how natural selection works. When there's money at stake, someone will always do it.

edit: typo

And people like that are why we can't have nice things.

I'm not saying that education / provoking thought will stop everyone doing something (else) monstrously stupid with AI. Just that we should be doing that educating / provoking more than we currently are.

Even to the detriment of themselves... see "Tragedy of the Commons"

Reply on news.ycombinator.com