Hacker News Reader

Comment by godelski | Hacker News Reader

Honestly, as someone that researches ML, this is my major concern. It isn't AGI that has the potential to kill us, it is current ML systems that can't handle OOD data and engineers put it in place because "that's the user's fault." Same reason we have Teslas crashing. AI safety might talk about AGI a lot, but their main area of research is modern systems and concerns over that.

OOD data is really hard to deal with FWIW. But personally I don't feel confident that adding more matrix multiplies won't generalize in a way such that OOD isn't of major concern.

View on HN

AGI alignment is a vastly bigger problem. Of course poorly built and deployed ML systems will kill and injure people - but these are tragedies of the kind humanity can endure and overcome and has overcome. Poorly aligned AGI is nothing less than the entire species at stake.

> AGI alignment is a vastly bigger problem

But also far less likely to happen anytime soon. A bigger danger is when someone thinks a machine is sentient or "semi-conscious" (whatever that means) and naively uses it to do tasks it shouldn't.

I don't think you nor anyone else knows when AGI is likely to happen. I also don't think that incorrectly believing a machine is sentient when it is not is a "bigger danger."

Again, an improperly aligned AGI could kill the entire human race. I'm not sure what harm incorrectly believing a machine is sentient might do, but I don't think it would be as bad as human extinction or enslavement which are both real possibilities with AGI.

You seem to be comparing only the worst-case impact and not the probability. To see why that's fallacious, consider that an asteroid could also kill the entire human race, but nobody would agree that asteroids are more dangerous than drunk driving.

I think there's a high probability of AGI within a century. Surveys show most experts share that opinion. It's hard to know the probability that the AI will be misaligned - but currently we have no idea how to align it. It's also hard to say how likely a misaligned AI would be to cause extinction. However, we have no reason to think that either of those things are unlikely.

>Again, an improperly aligned AGI could kill the entire human race

Lots of humans are "improperly" aligned GI. Rouge humans like Hitler or Kim Jong Un haven't been very successful.

> I don't think you nor anyone else knows when AGI is likely to happen.

Sure. But since I am an AI research I'd imagine I'd have a good leg up on the average person. I'm at least aware of the gullibility gap. Lots of people think it is closer than it is because they see things doing tasks that only humans can do but really your pets are smarter than these machines.

I think that's a reasonable stance, but only for some values of "soon". In a hundred years, we may well have AGI. At that time, we better have developed a robust science for how to control them. This is a somewhat unrelated problem to the current problem of machine/AI safety and both require more focus than they currently get.

We use the same scheme we use to control humans. The rich own all the valuable land and all the money. Ban robots from owning land. That way people can just use their shotgunsor call the police to kill them for trespassing.

If I were an AGI robot I would be scared of getting swatted for lols.

This is a classic case of undefined behaviour or memory unsafety. Your mistakes can have an infinitely bad outcome but people blame the programmer even though there are memory safe languages. Yes they sacrifice efficiency but who the fuck wants to consider the billion potential ways of operating a robot in a physically unsafe way?

This means we are going to have the equivalent of GC in robots that interact with other humans.

Genuine question: is Tesla’s autopilot crashing more often or more severely than human drivers?

There was a post recently with a video of a Tesla attempting to drive into the path of an oncoming train: https://youtube.com/watch?v=yxX4tDkSc_g

It’s not the big mistake at the end that stands out to me but the sheer volume of mistakes it makes along the way. Edging forward at an intersection when there’s a red light for example.

I don’t doubt that self driving tech will improve and be a safer alternative to a human driver eventually. It doesn’t seem like we’re there yet though.

Yeah, I fully expect Autopilot to have different failure modes than human drivers, but what I’m interested in is the different fatality rates (deaths per hundred million miles, adjusted for different types of roads i.e., highway vs city streets). If Autopilot can save hundreds of lives annually to human-error mistakes like falling asleep at the wheel, etc but at the cost of one life annually due to obscure failure modes like driving toward a train, I maintain that we should not only allow Autopilot, but probably even mandate it on new vehicles. Sacrificing hundreds or thousands of lives annually because we don’t like the specific failure modes seems absurd. Of course, if it doesn’t save lives, then we should block its deployment on those grounds (but the particular kind of failure mode shouldn’t affect the calculus).

To some degree, that doesn't matter. An underlying feature of a competent approach to safety in design is that the design must take maximal ownership of eliminating risk to all people in all scenarios that can be reasonably expected to result from the design.

The moment Telsa set expectations by proclaiming it as autopilot, they took the corresponding responsiblity to make sure it did not generate any scenarios which were unsafe. The moment they implemented features that allowed the attention of drivers to drift more than standard driving, they also took responsibility to make sure that the drifting attention of drivers did not place the system in an unsafe state.

This same issue applies to touch-screen interfaces in modern cars. Drivers could always stare down at their radio when there were tactile knobs and dials, but touch-screen interfaces now expect that because they've eliminated tactile feedback. Telling drivers 'just don't look down' misses the point, because it's the responisbility of the car manufacturer to not create a system where that added safety risk is not controlled appropriately.

Pretty much this. They could have called it Super Cruise Control or something and I'm pretty sure nobody would have anything to say, because it is expected that cruise control be supervised, but, I think people wouldn't be quite as willing to pay a lot of money for a feature that didn't sound so remarkable.

Self driving technology did seem to reach human parity in 5 years back in 2010s, and the growth was later revealed to be logarithmic than exponential, and Elon doubled down on a bad bet on it.

It’s not about whether they should have clarified the scope, the scope did include a completely automatic driving. It just that they failed to deliver(tbf no one truly made it).

I really don’t see how anything could matter more than “does it save lives, on balance”. If it saves thousands of lives annually, then why would we let tenuous marketing grievances forestall its deployment? How many lives should we sacrifice over branding concerns? Of course, if the technology doesn’t save lives on balance, then that’s reason enough to restrict deployment, but in any case marketing issues don’t seem like they should factor into the calculus.

Tesla publishes a Safety Report regarding this.

https://www.tesla.com/VehicleSafetyReport

Many, if not all publicized "Autopilot suspected" Tesla crashes were later found to happen because driver accelerated too fast, forgot that break pedal exists and lost control.

That is less possible with Autopilot, as it can't go faster than 90 mph

By OOD, do you mean Out Of Domain?

Out of distribution.

Not the parent but presumable “out of distribution” meaning being faced with inputs outside of the statistics of the training data set.

Yes, this

I suppose that's a nice way of saying "We never trained it not to break someone's finger".

These acronyms are destroying my will to live.

Are there any other major examples of modern ML ethical issues besides some Tesla cars killing their drivers?

Are ML driven robots in factories killing people or Something? Because I haven’t heard of anything else.

The only other modern AI ethics stuff I hear about is making image generators more politically correct and maybe some criminal sentencing algorithms that are being misused (which isn’t really an AI ethics problem but a judicial procedural one).

I think everyone here is justified in saying "do your own research."

Plenty of stories if you just Google a bit.

I was mostly being facetious, I have done my research. I don’t really expect many legitimately good answers (although I’ll keep an open mind).

I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality.

Maybe ML needs more script kiddy frameworks like KaliOS for these tools before the justifications to slow down R&D will finally be justified?

> I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality.

I'm a bit confused. There's plenty of propaganda written by ML. Here's some deep fakes with respect to Ukraine[0][1]. Manufacturing robots kill people all the time[2][3]. They are weaponizing ML. Like specifically GPT-3? Probably not but people do use these to write tweets and short form things.

[0] https://news.northeastern.edu/2022/04/01/deepfakes-fake-news...

[1] https://www.npr.org/2022/03/16/1087062648/deepfake-video-zel...

[2] https://www.theguardian.com/world/2015/jul/02/robot-kills-wo...

[3] https://www.techrepublic.com/article/robot-kills-worker-on-a...

> Manufacturing robots kill people all the time[2][3].

Your techrepublic article discredits your statement:

> While any death is a tragedy, it also must be put into perspective. Humans and robots have been working together in the manufacturing industry for decades with few grievous problems. According to a 2014 New York Times report, citing OSHA, at the time robots had been responsible for 33 workplace deaths over the past 30 years. According to the National Association of Manufacturing, there are 12.3 million manufacturing workers in the US, who account for roughly 9% of the country’s workforce.

> Are ML driven robots in factories killing people or Something? Because I haven’t heard of anything else.

All the videos on the YouTubes I’ve seen show industrial robots with crazy amounts of safety equipment where you can’t get close enough to it while it’s running for someone to get hurt.

I have seen people experimenting with robot arms next to their CNC machine where it could easily take off someone’s head if you piss it off but these are small shops where they expect the operator to keep on the robot’s good side, no inappropriate sexual comments and biology shaming.

They are using ML algorithms on CNC machines and it’s causing AI ethical concerns?

Building experimental robotics on a production floor with workers walking around sounds like a normal safety issue not an AI one.

I was watching one video where they had to train the arm what to do and am pretty sure they (the Silicon Valley robot arm startup) gave it AI magic sauce because even toasters have AI these days.

Not AI directly, but there is a talk by a coder who was asked to do triangulation targeting for mobile phones. It was an interesting problem so he went for it.

After a while he figured out that his code was used to target missiles on people using cell phones in Iraq.

Not any AI then?

Reply on news.ycombinator.com