Hacker News Reader

Comment by Forge36 | Hacker News Reader

I have this same problem with people. Just because we don't understand perfectly doesn't mean the output is not useful.

Sure, but humans are a relatively known entity. They exist with a level of variance that is not too extreme, and you can more or less, in a general way, predict their behavior to a tolerable risk level. We've seen billions of humans, we know what to expect of them. We might not understand how other humans work internally, but we understand their actionspace.

For AI models, that isn't really the case. We don't know how each individual model will act in the real world, and certainly not after having spent millions of hours training against itself or in a simulated environment. And as these models get more and more powerful, this growing uncertainty is something that I believe is worrisome.

> Sure, but humans are a relatively known entity.

We don’t know how brain works, why sleep exist, most of humans cultures and languages are not documented, hundreds if not thousands of psychological and medical things are unknown, etc. Machines and their applications are a few more magnitudes more understood than any human matter, by the sheer fact we created them. Their complexity is ridiculously low in comparison to human, their biology, psychology, etc.

on one hand i agree with what you're saying. we humans have done terrible stuff like wars, genocides, famines, destruction of ecosystems, extinction of entire species, etc. and that's only the things we did more or less deliberately. we might cause our own extermination or a mass extinction event "by mistake", and we don't even understand how basic parts of our minds work.

but all of that being said, i think it's also worth considering what a non-human entity with more power than us might be able to do. after all, even the worst examples of humanity are confined to at most "only" causing millions of deaths, but not the eradication of all life on Earth for example. it's not difficult to imagine that a non-human entity, with non-human goals, and with super-human powers, might act in a way that is contrary to our interests, or even to our existence.

This already applies to corporations.

AI is just the fruiting body of something that has already happened.

Corporate intelligence are still people, abiding laws and regulations for people.

Yes, I wouldn't want an army of self supervised learning military mini drones

That was not the original point. Humans are relatively predictable whether we understand how the brain works exactly. An AI model isn’t an actual intelligence in the normal use of the word. It’s a functional fragment that is able to emulate the ability to absorb a pattern of information for repetitive conditional recall in future processes. Until a researcher has had ALOT of time spent on observing the limits db particulars of a model, there are unforeseen behaviors it can have that can have incredibly real and serious consequences depending on the application. A more fair comparison would be training a new intelligent species to do something before learning about the characteristic behavior quirks of said species

Luckily we never have unforeseen behaviors with people that we then have to react strongly to after the fact ... We like to pretend that's true, but really, it isn't. Governments are built around the principle that single individuals and even small to medium groups can't be trusted to react reasonably, something that played out in human history again and again.

How many people have been in US prisons again? Oh right, about 3%. Clearly a high percentage of these people did something both unexpected that seriously damaged others.

So this is not a real difference between AIs and humans. One might even say, not a single AI has been convicted yet, so it might not be true for them. For humans, serious, bad, violent, illegal, even when totally irrational, ... behaviors are very common indeed.

(and frankly, if there ever is a true Human <> AI conflict, not allowing any kind of errors for AIs seems to me a strong contender for the casus belli)

Humans being unpredictable at the individual level although true does not invalidate the original point that employing a poorly understood neural net training methodology in critical systems of any kind is incredibly rash and presumptive

Then train the models on real world data? Verify outputs enough until confidence is achieved.

The computer can do whatever it wants. People can do whatever they want. The question will be what level of security access will they have.

The key difference today is people are really good at making rationalizations for individual decisions. Computers are not.

Sometimes decisions are generally important, when they are important they require trust, and if these models never generate a method of demonstrating success/trust they won't be adopted.

>> Then train the models on real world data? Verify outputs enough until confidence is achieved.

And therein lies the rub. Do people actually want to know the truth? They invent methods to obtain it, but is that their aim or is it to confirm their current understanding of the world? Unlike a human being that can be forced into silence through coercion or manipulation, a computational model, once proven with certainty, is never going to go back in Pandora's box. This contradiction of pursuing truth but suppressing the inconvenience of its conclusions hasn't disappeared for some reason despite greater and greater knowledge of ourselves. Will it ever?

Both? I'm not 100% following. I think people want to know the truth, change is slow and admitting mistakes is often looked down on. Humanity made significant changes based on what was learned in the past 100 years. How this will be used, and who will reap it's benefits is unclear.

Perhaps I wasn't very clear. Would the general public accept the truth of a data model that proved that a previously controversial/unacceptable idea was factually correct to the point it could not be denied by a reasonable person?

Several attempts at training with real world models have produced results that have been attacked for being "algorithmically biased". This isn't because there was anything experimentally wrong with the dataset or choice of model on their own, but because of the result's contradiction of a certain worldview. While, I agree with you that people come closer to the truth in the long-term, in the short- and medium-term, I expect certain people to pad their own data and quash the different findings out of ideological motivation for some time.

Very few individual humans can act in a way that affects hundreds of millions of people, computers can do that in a nanosecond.

Can't humans with computers affect hundreds of millions of people too?

Only with the computer's help, that's my point.

In a medical context, the doctor won't use it if he doesn't know where it gets its score from

As she should. Trust but verify. The biggest question is what will this be applied to. Without it we can't really discuss approaches towards developing that trust.

I'd consider The ability to output a rationalization in human understandable text (ie: english in my case) will determine success /failure.

Reply on news.ycombinator.com