Comment by dcole2929 | Hacker News Reader

Comment by dcole2929 | Hacker News Reader

I think what he describes here is a fundamental flaw of human nature. We now have overwhelming evidence that humans are really good at taking incomplete, disparate pieces of information and coalescing them into a whole to reach a conclusion. We generally infer very well. The problem is that the conclusions we reach are also generally wrong. Given a very specific type of domain with deterministic answers (e.g. most engineering fields) we are able to reach more often than not correct answers. And the rate at which we are correct has only increased (nearing 100%). However, when it comes to the messy domains in which we live we are still pretty poor. Our instincts are generally good enough to keep us alive (don't walk down this dark alley, that guy looks shady, etc) but much more than that and we suck.

I for one am deeply uncomfortable with people training AI systems to form the kind of decisions we as humans do. These systems may have more information to go on but if it's using the same methods that we do, how accurate can it be? Thanks to the rapid advancement of the last 150 years we have quickly reached a place where society's technological knowledge and ability far exceeds our moral ability to reason about it, and it's implications. We are like an untrained child with a gun. Yeah we may understand the mechanics behind pulling the trigger and how it works but we don't yet have the maturity to understand when we should, and what truly happens when we do.

There's this attitude that because computer work on pure logic, anything a computer does must therefore be logical and rational. The problem is that the humans who program the computers will often encode their own illogical, irrational, biased, and flawed logic. The computer is not logical and rational, it just follows instructions well. If the instructions are illogical, irrational, biased, and flawed---but executable---the computer will follow them. And when they spit out a illogical, irrational, biased, and flawed result... well, it must be logical, rational, unbiased, and unflawed, because the computer is purely logical, rational, unbiased, and unflawed.

We code our biases into the algorithms we use every day. They inherit our flaws. The sooner we all wake up to this, the better.

> We code our biases into the algorithms we use every day. They inherit our flaws. The sooner we all wake up to this, the better.

And our values. And assumptions. In Arthur C. Clarke's "2001: A Space Odyssey" the homicidal behavior of HAL was ultimately based on the conflict between his secret instructions and what he was able to share with the human crew, Bowman and Poole.

This is actually a serious concerns when it comes to AI. People tend to think it's overblown but non deterministic behavior is bad because you don't know what the program will do. Maybe it crashes, maybe nothing happens, maybe it nukes your cpu. You simply don't know. The first rule of true AI has to be to ensure it values human life. It would be grossly irresponsible to create an entity more intelligent than us and do otherwise. But what happens when the military inevitably decides these new AI things would make really good drone pilots. Or when a sufficiently powerful enough AI comes to the conclusion that the best way to protect human life is to take human life. What's the end result of that conflict. What's stopping a sufficiently intelligent AI from rewriting it's own code to get around restrictions it doesn't like. We already have self modifying code. It's a scary thought, and the fact that people are basing these things off the way humans think makes it even scarier

On the other hand, if the instructions are clearly defined and machine "intelligence" is used, the computer will often fix the biases of it's creators.

https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

What you are pushing is nothing but the idea of (secular) original sin for algorithms.

Unfortunately, innumerate reporters imagining AI as somewhere between a human and a 5 line python script doing exactly what the creator proposed are unable to comprehend this.

Reply on news.ycombinator.com