Hacker News Reader

Comment by carbadtraingood | Hacker News Reader

The answer from the federation is unconscionable. They blamed the boy and then said they could not be held responsible. Fuck off, a kid made a reasonable kid movement. If the robot wasn't ready to be around children, it shouldn't have been deployed around children. And there should have been a big red button that immediately everything, within reach of every player.

View on HN

Honestly, as someone that researches ML, this is my major concern. It isn't AGI that has the potential to kill us, it is current ML systems that can't handle OOD data and engineers put it in place because "that's the user's fault." Same reason we have Teslas crashing. AI safety might talk about AGI a lot, but their main area of research is modern systems and concerns over that.

OOD data is really hard to deal with FWIW. But personally I don't feel confident that adding more matrix multiplies won't generalize in a way such that OOD isn't of major concern.

By OOD, do you mean Out Of Domain?

Out of distribution.

Not the parent but presumable “out of distribution” meaning being faced with inputs outside of the statistics of the training data set.

Yes, this

I suppose that's a nice way of saying "We never trained it not to break someone's finger".

These acronyms are destroying my will to live.

Genuine question: is Tesla’s autopilot crashing more often or more severely than human drivers?

There was a post recently with a video of a Tesla attempting to drive into the path of an oncoming train: https://youtube.com/watch?v=yxX4tDkSc_g

It’s not the big mistake at the end that stands out to me but the sheer volume of mistakes it makes along the way. Edging forward at an intersection when there’s a red light for example.

I don’t doubt that self driving tech will improve and be a safer alternative to a human driver eventually. It doesn’t seem like we’re there yet though.

Yeah, I fully expect Autopilot to have different failure modes than human drivers, but what I’m interested in is the different fatality rates (deaths per hundred million miles, adjusted for different types of roads i.e., highway vs city streets). If Autopilot can save hundreds of lives annually to human-error mistakes like falling asleep at the wheel, etc but at the cost of one life annually due to obscure failure modes like driving toward a train, I maintain that we should not only allow Autopilot, but probably even mandate it on new vehicles. Sacrificing hundreds or thousands of lives annually because we don’t like the specific failure modes seems absurd. Of course, if it doesn’t save lives, then we should block its deployment on those grounds (but the particular kind of failure mode shouldn’t affect the calculus).

To some degree, that doesn't matter. An underlying feature of a competent approach to safety in design is that the design must take maximal ownership of eliminating risk to all people in all scenarios that can be reasonably expected to result from the design.

The moment Telsa set expectations by proclaiming it as autopilot, they took the corresponding responsiblity to make sure it did not generate any scenarios which were unsafe. The moment they implemented features that allowed the attention of drivers to drift more than standard driving, they also took responsibility to make sure that the drifting attention of drivers did not place the system in an unsafe state.

This same issue applies to touch-screen interfaces in modern cars. Drivers could always stare down at their radio when there were tactile knobs and dials, but touch-screen interfaces now expect that because they've eliminated tactile feedback. Telling drivers 'just don't look down' misses the point, because it's the responisbility of the car manufacturer to not create a system where that added safety risk is not controlled appropriately.

Pretty much this. They could have called it Super Cruise Control or something and I'm pretty sure nobody would have anything to say, because it is expected that cruise control be supervised, but, I think people wouldn't be quite as willing to pay a lot of money for a feature that didn't sound so remarkable.

Self driving technology did seem to reach human parity in 5 years back in 2010s, and the growth was later revealed to be logarithmic than exponential, and Elon doubled down on a bad bet on it.

It’s not about whether they should have clarified the scope, the scope did include a completely automatic driving. It just that they failed to deliver(tbf no one truly made it).

I really don’t see how anything could matter more than “does it save lives, on balance”. If it saves thousands of lives annually, then why would we let tenuous marketing grievances forestall its deployment? How many lives should we sacrifice over branding concerns? Of course, if the technology doesn’t save lives on balance, then that’s reason enough to restrict deployment, but in any case marketing issues don’t seem like they should factor into the calculus.

Tesla publishes a Safety Report regarding this.

https://www.tesla.com/VehicleSafetyReport

Many, if not all publicized "Autopilot suspected" Tesla crashes were later found to happen because driver accelerated too fast, forgot that break pedal exists and lost control.

That is less possible with Autopilot, as it can't go faster than 90 mph

AGI alignment is a vastly bigger problem. Of course poorly built and deployed ML systems will kill and injure people - but these are tragedies of the kind humanity can endure and overcome and has overcome. Poorly aligned AGI is nothing less than the entire species at stake.

> AGI alignment is a vastly bigger problem

But also far less likely to happen anytime soon. A bigger danger is when someone thinks a machine is sentient or "semi-conscious" (whatever that means) and naively uses it to do tasks it shouldn't.

I don't think you nor anyone else knows when AGI is likely to happen. I also don't think that incorrectly believing a machine is sentient when it is not is a "bigger danger."

Again, an improperly aligned AGI could kill the entire human race. I'm not sure what harm incorrectly believing a machine is sentient might do, but I don't think it would be as bad as human extinction or enslavement which are both real possibilities with AGI.

You seem to be comparing only the worst-case impact and not the probability. To see why that's fallacious, consider that an asteroid could also kill the entire human race, but nobody would agree that asteroids are more dangerous than drunk driving.

I think there's a high probability of AGI within a century. Surveys show most experts share that opinion. It's hard to know the probability that the AI will be misaligned - but currently we have no idea how to align it. It's also hard to say how likely a misaligned AI would be to cause extinction. However, we have no reason to think that either of those things are unlikely.

>Again, an improperly aligned AGI could kill the entire human race

Lots of humans are "improperly" aligned GI. Rouge humans like Hitler or Kim Jong Un haven't been very successful.

> I don't think you nor anyone else knows when AGI is likely to happen.

Sure. But since I am an AI research I'd imagine I'd have a good leg up on the average person. I'm at least aware of the gullibility gap. Lots of people think it is closer than it is because they see things doing tasks that only humans can do but really your pets are smarter than these machines.

I think that's a reasonable stance, but only for some values of "soon". In a hundred years, we may well have AGI. At that time, we better have developed a robust science for how to control them. This is a somewhat unrelated problem to the current problem of machine/AI safety and both require more focus than they currently get.

We use the same scheme we use to control humans. The rich own all the valuable land and all the money. Ban robots from owning land. That way people can just use their shotgunsor call the police to kill them for trespassing.

If I were an AGI robot I would be scared of getting swatted for lols.

Are there any other major examples of modern ML ethical issues besides some Tesla cars killing their drivers?

Are ML driven robots in factories killing people or Something? Because I haven’t heard of anything else.

The only other modern AI ethics stuff I hear about is making image generators more politically correct and maybe some criminal sentencing algorithms that are being misused (which isn’t really an AI ethics problem but a judicial procedural one).

I think everyone here is justified in saying "do your own research."

Plenty of stories if you just Google a bit.

I was mostly being facetious, I have done my research. I don’t really expect many legitimately good answers (although I’ll keep an open mind).

I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality.

Maybe ML needs more script kiddy frameworks like KaliOS for these tools before the justifications to slow down R&D will finally be justified?

> I’m still waiting for those GPT-3 and deep fake horror stories we were warned about to come to reality.

I'm a bit confused. There's plenty of propaganda written by ML. Here's some deep fakes with respect to Ukraine[0][1]. Manufacturing robots kill people all the time[2][3]. They are weaponizing ML. Like specifically GPT-3? Probably not but people do use these to write tweets and short form things.

[0] https://news.northeastern.edu/2022/04/01/deepfakes-fake-news...

[1] https://www.npr.org/2022/03/16/1087062648/deepfake-video-zel...

[2] https://www.theguardian.com/world/2015/jul/02/robot-kills-wo...

[3] https://www.techrepublic.com/article/robot-kills-worker-on-a...

> Manufacturing robots kill people all the time[2][3].

Your techrepublic article discredits your statement:

> While any death is a tragedy, it also must be put into perspective. Humans and robots have been working together in the manufacturing industry for decades with few grievous problems. According to a 2014 New York Times report, citing OSHA, at the time robots had been responsible for 33 workplace deaths over the past 30 years. According to the National Association of Manufacturing, there are 12.3 million manufacturing workers in the US, who account for roughly 9% of the country’s workforce.

> Are ML driven robots in factories killing people or Something? Because I haven’t heard of anything else.

All the videos on the YouTubes I’ve seen show industrial robots with crazy amounts of safety equipment where you can’t get close enough to it while it’s running for someone to get hurt.

I have seen people experimenting with robot arms next to their CNC machine where it could easily take off someone’s head if you piss it off but these are small shops where they expect the operator to keep on the robot’s good side, no inappropriate sexual comments and biology shaming.

They are using ML algorithms on CNC machines and it’s causing AI ethical concerns?

Building experimental robotics on a production floor with workers walking around sounds like a normal safety issue not an AI one.

I was watching one video where they had to train the arm what to do and am pretty sure they (the Silicon Valley robot arm startup) gave it AI magic sauce because even toasters have AI these days.

Not AI directly, but there is a talk by a coder who was asked to do triangulation targeting for mobile phones. It was an interesting problem so he went for it.

After a while he figured out that his code was used to target missiles on people using cell phones in Iraq.

Not any AI then?

This is a classic case of undefined behaviour or memory unsafety. Your mistakes can have an infinitely bad outcome but people blame the programmer even though there are memory safe languages. Yes they sacrifice efficiency but who the fuck wants to consider the billion potential ways of operating a robot in a physically unsafe way?

This means we are going to have the equivalent of GC in robots that interact with other humans.

It's appalling the robot was designed to ever use that much force in the grip. Even if the chess pieces were made of lead I can't see it being needed. In general, more attention to failing safe.

But the kid is some kind of local chess champion, I can't fully fault the decision to have him play with the experimental chess robot. Is it more dangerous than a lawn mower or a blender or any other machine that 9 year olds might begin to operate?

Correct. The fact they made a robot that could crush a human hand means they paid no attention to this hazard. Competent execution of Safety in Design concepts would demand limiting the grip force to only what's necessary to reliably move the pieces, which almost certianly wouldn't break bone. If that isn't possible, then it would imply the requirement to find some other way to resolve this hazard in the heirarchy of controls.

Relying on a human is the last option, not the default, when it comes to safety. Human adaptability is not a licence to hand-wave away design responsibility. The most glaring example is Tesla, who is unforgivably guilty of this.

This is bog-standard competent engineering in almost all domains of engineering. It is the table stakes-level expectation of a reasonable approach to safety. I'd literally end up in jail if something went wrong and I had been found to not consider these factors.

Software- and computer-related domains of engineering are a conspicuous outlier when it comes to this philosophy.

You would think that much force would lead to broken actuators and chess pieces often enough during development that someone would land on the idea of setting an upper limit on all forces just to save money.

If that is the official statement of the federation, I agree it's abysmal. But i question whether Lazarev really blurted all of that in one go, or if he was asked a series of leading questions and then his answers were misleadingly pasted together to make him sound as callous as possible.

To be clear, any answer short of, "this was a failure on our part to protect the children who attended, and as the leader of the organization, responsibility falls on me. We will make this right for the family and I will tender my resignation" is not adequate.

>> I will tender my resignation

whats with this attitude?

He's responsible for a gross oversight in safety that led to an injured child. There were obvious and simple safety precautions that were missed.

This wasn't just, "whoops", this was fairly gross negligence.

Then you don't want a voluntary resignation, I think you want them ousted.

The kind of person to learn from their mistakes through honest self reflection (and hence would voluntarily resign) is probably the kind of person you might want to keep around, or at least not decide to make a public example of.

The kid tries to beat a piece before it was even placed and then the robot tried to place the piece on his piece while his hands were still covering it. This could have lead to a minor injury even with a human opponent. What is strange is that they insisted on using such a powerful robot arm without any compliance in the actuators.

Now it broke a finger, which sucks, but’ll heal. But what if it had taken out an eye?

That's not what they said. They said they weren't responsible for the robot, and which is true. Same as if a human player injured another player. The robot's operator is responsible.

I disagree, if they are running the event, they are responsible for ensuring the safety of participants. You go skydiving and the chute fails in a totally predictable way, the company who takes you up shouldn't just shrug and say, "well, we aren't responsible for the chute, that was provided by another company"

Are you really going to go for the "think of the children" argument...?

Although I do agree with you about having an e-stop.

"Day cares should be safe for children"

"OMG did you just use the 'think of the children' fallacy??? Invalid!"

There are in fact times when we need to think about children. If we don't, humanity won't have much of a future.

Quite the contrary, I think. Society has become "soft", for lack of better words. If we had been so risk-averse centuries ago, the industrial revolution would have never happened.

Why is it children injured by robots who need to toughen up rather than the people nostalgic for a past they never experienced?

That sort of argument might work well for a time when slavery was a recent memory and children frequently worked in factories instead of going to school. That doesn't mean it works equally well for the present day, or for the kind of future most of us would probably prefer.

I meant it literally. It's difficult to raise children without thinking about them, and if we don't raise any children...

'dont wish upon others that Wich you yourself cannot endure'

I'm confused. You think if we don't let robots break childrens fingers because we insist on proper safety, then the industry of physical chess playing robots might never get off the ground?

We could have had a perfectly good industrial revolution with more mechanical safeguards and less child labor.

If a machine is 20x faster than human labor, and making it safe knocks off 10%, that's fine. And it'll leave you with better employees over time. It's only a problem if you're in a race to the bottom that doesn't care about worker safety.

The industrial revolution and accompanying urbanization lowered birth rates.

Technological progress is not necessarily good for the species, although it can be (c.f. the advent of new agriculture methods creating more food post WW2).

"Think of the children" when used as an expression refers to the situation in which children are used as an excuse to implement rules which would be otherwise unpalatable. Not every case of protecting kids is a "think of the children" situation.

Yes, I'm going to make the outrageous claim here: children should not be expected to work next to industrial machinery with inadequate safeguards.

I know this is controversial to some, such as userbinator, who believe that industrial robotics without safeguards are just fine to mix with children.

There are tons of rules around these kind of robots in regular work environments for precisely this reason.

Usually there need to be either a physical barrier like a cage or a virtual one like a laser waterfall that detects foreign objects in the robots perimeter and emergency stops it.

These rules were disregarded here.

I used to work in a company were such machines were developed and even a very experienced engineer, working on a prototype, was once hit by it (no serious injuries and safety was improved afterwards) because they can move very fast and in unexpected ways.

These days there are better solutions available (so called cobots) which are designed to be work together in very close proximity with humans whiteout physical separation. They feature very sensitive force sensors and are severely restricted in the way the are allowed to move.

So yes "think of the humans/children" does apply here. This is a solved problems and the operators decided to disregard established procedures and went instead for "flashy and cheap" (cobots are more expensive and slow as molasses)

>a kid made a reasonable kid movement

What do you mean? There's no reasonable time where you and your opponent are touching the pieces at the same time. Nor is there a reasonable time where you reach for the same piece.

Seven-year-old boys are known for always doing things that are reasonable, and never doing anything unreasonable.

I agree especially since he "is among the top 30 chess players in Moscow under the age of nine."

It does not excuse such a reckless use of industrial equipment around people without appropriate failsafes.

A robotic arm like this not a toy, is a deadly machine with a lot of force and a lot of mass. If you were standing next to it for some reason it could unexpectedly swing around and break your neck in an instant. Engineers should be careful to design such systems with failsafes that account for any action the operator might take.

He’s still just a little kid. Even the most precocious kids are still kids.

He's "one of the top 30 chess players under nine in Moscow". Still the robot's fault, though.

So you're saying he's already done at least one unreasonable thing...

It's a kid, my friend.

The penalty for an illegal move is maybe forfeit, not a broken finger.

Sure there is, like if you move a piece, hit the clock, then realise your piece wasn't quite centered on the square. Maybe not technically correct, but reasonable.

A robot isn't just going to grab your piece since the rules of the game state it has to move its own piece.

Oh, that settles it. Robot cannot do things that are outside of the rules of chess.

Kid should have known better than to open with the classic, "break my finger" opening from the regulation standard rules.

Industrial robotics are incredibly dangerous. They operate without concern for anything in their path. Robotic arms like this can be lethal, and the idea that the robot wouldn't ever do anything unusual is, quite frankly, laughable naive.

The problem isn't that the robot did anything unusual, it did exactly what it was supposed to do which is crush everything in its path.

Is it normal in chess to break fingers of an opponent who breaks rules? I think not. If it was not a robot but a human being he would be considered guilty. Yeah, boy broke chess rules, and what? He probably should be punished by losing chess points or something, but not by the means of breaking fingers. But the human who broke fingers of his opponent would be disqualified from chess for a life and would face charges. It is robot, it cannot be guilty, so someone else is. Who is? His creators? Or organizers of the event? Or parents of the kid who allowed him to go to face the robot? Some adults are guilty, not the boy.

Dude, children get antsy sometimes. They literally have different brain development than adults do. They do not have the same controls and inhibitions.

Children are literally wired for physical experimentation and echopraxia.

Kids also want to be seen as helping. If this robot put a piece down poorly, the kid might be trying to straighten up after it.

All of which are eminently reasonable for a kid who has no concept of operating around dangerous machinery.

Reply on news.ycombinator.com