Hacker News Reader

Comment by SocksCanClose | Hacker News Reader

Great article, and the best is Musk's response. Just one more example of how, ala Allison and Zeilkow's "Essence of Decision," so much of what happens in industrial organizations ("industrial" in the philosophical sense -- meaning organizations with hierarchies, divisions, etc.) is motivated not by the overall interest of the organization (cf. Musk's response), but rather of the more parochial needs of the individual managers. Which is to say someone within the Tesla organization, fearful of an error they made, sought to retaliate against the guy -- even though the retaliation was unauthorized, and even counterproductive. An amazing use case for how Twitter, when optimally utilized as a total free-speech zone, can really help move the world forward, as things like this enable information to percolate directly to the top without winding its way through the "mittelebureaucracy."

View on HN

What would you do if your latest commits went live and they indeed did have new things in there, e.g. attribute values?

You might just roll the whole thing back a release, in the first instance to 'update' the hacker to a 'safe' version. If there were a need to roll everyone back then some type of patched new version would need to be released, or, if speed really mattered, just remote downgrade everyone to something safe. The major version numbers might not be the safe releases, the last release from a previous major version might be safer. Hence back to v.12.x.y for him. No malice be involved, just prudent reaction.

> No malice be involved, just prudent reaction.

It's not malice, it's panic and it's profoundly stupid.

Only those with root access to the car can access the secret info.

There are a handful of people out there who rooted their cars. They are individuals of extraordinary technical ability.

You think they don't know how to make a backup? Or how to stop remote access?

That's exactly what happened. He blocked them and then deleted the pending downgrade.

Let's assume there is a problem caused be the rooting and as a result the Tesla spins out of control and kills a bystander.

Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

> Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

No.

Cars have had software for a while. Tesla is the FIRST and ONLY company to do over-the-air updates. I don't think you should be criminally liable for turning a feature off.

Sorry, but once you push to production, the cat is out of the bag. I absolutely HATE this new attitude that it's OK to not thoroughly test code before pushing it to production, because hey, you can always fix it later or roll back, right?

> Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

No. There's no mens rea. He should be liable for damages in civil court.

there is if he is told the downgrade is for a critical safety bug that needs to be rolled back. Then he has shown criminal negligence for not allowing his car to be fixed

That's not what the OP said. He didn't say anything about downgrades. All he said was rooting and presumably, messing with the system. Neither qualifies as criminal negligence in my mind. I'm not sure I want to live in a society where gaining access to hardware I own gets me thrown in jail? I would also like to emphasize that this has never happened and is totally hypothetical.

Gaining access to safety critical hardware is a questionable right.

What if you are the owner of a building and you decide you want to "optimize" the fire detection/fighting system because you have some ideas in this area, or maybe you want to run an "open" software on it, with better reporting, and as a result some people die in a fire.

Yes, is totally hypothetical, but you seem to be saying that one is not responsible if he does that, because he was just "hacking", with no mens rea.

>Gaining access to safety critical hardware is a questionable right.

It happens every time I put more air in my tires.

(Serious question) But there's no law requiring tires to be able to be inflated by the consumer, is there?

By this logic killing someone while driving drunk is also not criminal since there is no mens rea.

There is such a thing as criminal negligence.

https://en.wikipedia.org/wiki/Criminal_negligence

In your case, the person was being reckless. I don't think gaining access to hardware you own can be considered reckless, regardless of the unlikely circumstances that may result. Your death-by-rooting example is entirely hypothetical and has never happened.

It hasn't happen only because the number of rooted smartcars so to speak is negligible compared to the total number of cars.

Just like drone accidents, each year we get closer and closer to an actual kill as their number increases (https://youtu.be/MvF49R_ZX5E). One day heavier drones will be required to run only certified software if flown over crowds, and rooting those drones will raise the same question.

Absolutely it could be considered reckless. You think some guy messing with his car's firmware understands it well enough to be as safe as the manufacturer? If an accident caused by manufacturer firmware can make the manufacturer liable for an accident, then someone tampering with it obviously can make them liable.

It has never happened? Who cares, what does that matter?

I never said the hacker shouldn't be liable. I said he shouldn't be criminally liable.

By your logic, if the manufacturer puts out software that is responsible for a death, would you throw the software developer who wrote in jail? That's silly.

All this stuff is limited to civil liability.

It really depends on what the developer did. If the developer knowingly removed all tests that weren't passing because the car, sometimes, did not recognize old ladies crossing the street, then this person should go to jail. This is evidently negligence.

Hmm, I see your point. But I don't know how you can draw a distinction between this and other forms of reckless endangerment. A manufacturer at least in principle isn't being reckless. But some random person that fancies themselves an elite hacker messing with things they definitely do not understand seems definitely reckless. And therefore criminal.

I agree entirely.

If I root my phone and accidentally brick it, nobody here would say, "Gosh, that's terrible, it's all Motorola's fault!" They would say, correctly, that if blame is to be assigned, it'd all be mine.

I think it's even clearer with something where even small errors could be fatal. Anybody rooting a car is betting both their life and those of the people they encounter on their technical skill. It might be an excellent bet, but they should be entirely aware that they're betting.

> Anybody rooting a car is betting both their life and those of the people they encounter on their technical skill.

Yes?

So is every guy who pops his trunk and starts working on the car? And people have been doing that since the very beginning. This isn't anything new.

Since upvotes aren't visible, I'll note that I was going to post the same thing. Person puts new brake pads on his/her car, brakes fail (or don't fail), same thing.

Sure, and if you are negligent in making modifications to your car, you will be liable. Mechanics regularly receive criminal charges when their actions cause deaths.

There are three new things here.

One is that automotive mechanical systems are relatively well understood. If you are a mechanic working on a car, it was literally designed for that. The manufacturer makes manuals explaining how to do it well, and they sell extra parts that are meant to be swapped in. There is a century or so of evolved practice an understanding here. This is not true with drive-by-wire software, which is certainly novel, and is frequently terrible.

The second is that mechanical stuff is generally linear and easy to understand. If I change a lightbulb, I have a pretty good understanding of what that will affect and what it won't. Software is highly non-linear. A novice working on, e.g., a device driver, can have effects they will find extremely surprising.

The third is that in software we have a long culture of consequence-free tinkering. We're used to being able to power-cycle something, to restore from backup, to just re-install the OS. Any software person is going to have a lot of habits and biases that become extremely dangerous when working with life-critical systems.

I'm not sure the law about drunk driving has much to do with logic anyway. If I can't drive a car under influence because that makes me unable to evaluate risks and in general to act responsibly - then how can I be considered completely responsible for the choice of driving, if I made it when I was already drunk?

It kind of shows that responsibility and liability, as they exist in our current legal systems, are flawed concepts that are in some cases too weak, and in others too strong. We could really use an update that takes modern behavioral science, neuropsychology, and intended/unintended consequences into account.

But in the case of drunk driving, you were negligent at the point of taking the first drink for not making safe transportation (or lodging) arrangements while sober.

Not all crimes require mens rea. Drunk driving, in particular, is a good example of a serious crime that is strict liability. Normally, however, strict liability is only applied to lesser offenses.

This is in principle no different to people modifying their cars, which they've been doing for about as long as cars have been sold.

Actually longer, since the first cars were the result of people modifying their horse carriages to not require horses :)

Yes, absolutely there should at least be the possibility of the owner being criminally liable. It's like if you drive with shit poorly strapped into the back of antruck and it falls off on the highway and kills someone. If it can be shown that the tampering caused the harm, it's really hard to argue there is no criminal liability.

One can envision various ways of showing that. Loading the custom firmware onto a different robocar, which then exhibits previously-unseen unsafe behavior related to the mishap in question, would be convincing.

Unfortunately from this thread it seems clear that we'll just have some blowhard Tesla manager swearing that "our code is perfect and any change to our code would automatically cause vehicular Armageddon, never mind the fact that we change it on a regular basis!" They won't want me on the jury...

Or you know, you do it like Apple. You make no comment whatsoever, keep the software and let the rumor machine keep you on top of tech news sites.

Reply on news.ycombinator.com