Man hacks Tesla firmware, finds new model, has car remotely downgraded | Hacker News Reader

Man hacks Tesla firmware, finds new model, has car remotely downgraded

arstechnica.com

239 pointsantman94 comments

"They punished him by remotely downgrading his firmware" is a frankly absurd claim... and incorrect, too.

The guy's own post [1] said it was "pending" (e.g. downloaded and ready for an automatic update), not installed. Given that someone else in the same thread notes that the update when installed on their car didn't actually fix the important charging bug it was supposed to, it's much more likely that it was "somebody in engineering decided to cancel a bad update" than his paranoid claims.

[1]: http://www.teslamotorsclub.com/showthread.php/63905-Tesla-s-...

> The guy's own post [1] said it was "pending" (e.g. downloaded and ready for an automatic update), not installed.

This is correct. It was an attempted downgrade. He stopped it.

> it's much more likely that it was "somebody in engineering decided to cancel a bad update" than his paranoid claims.

This is unlikely. He wasn't being downgraded to the version prior to the current one. He was being downgraded to a much older version that did not contain any of the secret info. And nobody else was experiencing the downgrade. If there is a critical flaw that requires a rollback, you do it for everyone and it's big news. He also made it clear that he found a ton of other stuff and was not disclosing that (yet).

It looks like somebody at Tesla pushed code to production that shouldn't have been pushed since it isn't relevant to any production vehicles. Once the leak happened, there was a bit of a panic and to stop anymore leaks, they tried to downgrade ... which was silly, because if you know how to root the car, you probably know how to make a backup and stop remote access.

So a rogue corporate exec ordered an unauthorized op against a ronin hacker who Robin Hood'ed the underhood of his electric car.

We are living in cyberpunk times.

If Tesla employees can address individual cars with no oversight or audit trail or "2+ man rule" in place, that's a huge lapse of professional responsibility.

We need something like Certificate Transparency to log ACL hits for all these "connected, but proprietary" things in our lives.

Do we have any guarantees a rogue tesla employee can't just tell your car to drive off a bridge while you're inside it?

Uber got busted for allowing general access to a "god mode" that let employees tail passengers. Musk may have brought more appropriate practices from PayPal though.

http://nypost.com/2016/01/06/uber-will-no-longer-allow-creep...

Or perhaps the car detected its firmware had been tampered with, and auto-reverted to a backup image.

Not enough evidence to say for sure, so Occam's razor applies.

Wouldn't that be worse security wise? Say if there is an exploit in the wild. The customer upgrades it to the latest version. Now all the bad guy has to do is to mess around with the firmware enough to trick the system into downgrading to the exploitable version.

If you have enough access to tamper with the firmware, you don't need a more exploitable version.

A car that has an app, a car that can be remotely updated, and a car that has all communication running through the same BUS, may be susceptible to remote break in without requiring any sort of physical access. Now the firmware may require signature verification to be patched, however in this case all we need is to corrupt the existing firmware or atleast make it seem like we had access to it in order to trigger an auto-downgrade.

Regardless, even under your logic an auto-downgrade without a user's input is completely unwarranted for.

The "Occam's razor" choice is frequently whatever one guy claims it is.

I think it's often a result of Occam's razor being mixed up with Hanlon's razor, or something similar: they claim that an argument for accident, forces of nature, or incompetence is more convincing than an argument for malice or willful intent, and that the former requires less assumptions than the latter.

I think this is often true, but definitely not always.

Or it comes down to the fact that Occam's Razor is just a cheap rhetorical device that's employed to lend a luster of legitimately onto what is otherwise just a subjective intuition. Fully comprehending how many assumptions are involved in a given line of reasoning probably takes a lot more time and thought than some guy on the internet is devoting to his HN comment.

You convinced me not to try to understand this argument...

Well, don't be too convinced. I didn't spend much time on that comment.

Maybe you just came up with the simplest possible refutation of Occam's razor. Maybe not :-)

I think this is because "simpler" is so subjective. Witness the Occam's Razor argument for theism - what's more likely, all of this crazy complicated quantum physics mumbo jumbo, or 'a wizard did it'? The latter only requires one assumption... until you start asking pesky questions about what, exactly, was the 'it' that the wizard did, and how was 'it' managed.

http://www.teslamotorsclub.com/showthread.php/63905-Tesla-s-...

It's all in framing. "person in control of software pushed an update to said software" would be just as accurate without sensationalization.

So a rogue corporate exec ordered an unauthorized op against a ronin hacker who Robin Hood'ed the underhood of his electric car.

We are living in cyberpunk times.

I remember when cyberpunk was fresh and new, before it was a codified collection of cliches. Mind you, this was also a time when an author could write, "His buyer for the three megabytes of hot RAM in the Hitachi wasn't taking calls," and think this sounded futuristic and criminally lucrative.

That reminds me, I was as a public service going to make an auto-updating epub of Neuromancer which simply incremented the units as needed to sound duly impressive.

"His buyer for the three petabytes of hot RAM..." etc...

Better off making something up. Flebibytes or something. Gibson should've done that in the original.

Flebibytes? That's a crazy amount.

On the street, sure.

In academe anything under ten mangabytes is noise these days tho and I imagine the dark economy is two factors beyond anything we have access to. :/

That's why Star Trek used things like kiloquads. Unless and until you know what a "quad" is, it's just a number in mid-air.

Or being deliberately nonspecific. Three memory modules, going cheap if no questions asked. That's why Snow Crash still reads so well, it skirts the quantitative and focuses on the qualitative.

It depends on what the three megabytes consists of. It could be quite lucrative.

What defined "golden age" cyberpunk that set it apart from ordinary science fiction? My modern mind associates the term with movies and video games that boil down to knock offs of Blade Runner and Deus Ex.

There's a fairly good analogy between SF and music. 70s music (and SF) was fat and bloated and slow. High concept stuff about galactic empires and gatefold sleeves and Serious Issues - Vietnam and the nature of reality.

And suddenly this fast, spiky thing appears, that's all nihilism and dystopia and edge and noir and anti-heroes and technology abuse, and instead of looking ahead a billion years it's looking ahead about 15 minutes, and instead of sounding like an orchestra it sounds like the wind over tensioned steel cables, and it wears sunglasses at night and it thinks technology is power and its just so gosh-darned cool...

Yeah, that's pretty much what it was like. Ok, it's all cliches now, but so's a modern punk band.

People forget that there was a reason for the "punk" in cyberpunk, and it wasn't mohawks and leather jackets.

a car company founded by a guy that created a bank in cyberspace shows that he has the power to push software to your car at will, though this time he choose not to, but do not care when his underlings do the same.

Great article, and the best is Musk's response. Just one more example of how, ala Allison and Zeilkow's "Essence of Decision," so much of what happens in industrial organizations ("industrial" in the philosophical sense -- meaning organizations with hierarchies, divisions, etc.) is motivated not by the overall interest of the organization (cf. Musk's response), but rather of the more parochial needs of the individual managers. Which is to say someone within the Tesla organization, fearful of an error they made, sought to retaliate against the guy -- even though the retaliation was unauthorized, and even counterproductive. An amazing use case for how Twitter, when optimally utilized as a total free-speech zone, can really help move the world forward, as things like this enable information to percolate directly to the top without winding its way through the "mittelebureaucracy."

What would you do if your latest commits went live and they indeed did have new things in there, e.g. attribute values?

You might just roll the whole thing back a release, in the first instance to 'update' the hacker to a 'safe' version. If there were a need to roll everyone back then some type of patched new version would need to be released, or, if speed really mattered, just remote downgrade everyone to something safe. The major version numbers might not be the safe releases, the last release from a previous major version might be safer. Hence back to v.12.x.y for him. No malice be involved, just prudent reaction.

> No malice be involved, just prudent reaction.

It's not malice, it's panic and it's profoundly stupid.

Only those with root access to the car can access the secret info.

There are a handful of people out there who rooted their cars. They are individuals of extraordinary technical ability.

You think they don't know how to make a backup? Or how to stop remote access?

That's exactly what happened. He blocked them and then deleted the pending downgrade.

Let's assume there is a problem caused be the rooting and as a result the Tesla spins out of control and kills a bystander.

Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

> Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

No.

Cars have had software for a while. Tesla is the FIRST and ONLY company to do over-the-air updates. I don't think you should be criminally liable for turning a feature off.

Sorry, but once you push to production, the cat is out of the bag. I absolutely HATE this new attitude that it's OK to not thoroughly test code before pushing it to production, because hey, you can always fix it later or roll back, right?

> Would you agree that this extraordinary technical ability individual should be criminally liable in this case?

No. There's no mens rea. He should be liable for damages in civil court.

there is if he is told the downgrade is for a critical safety bug that needs to be rolled back. Then he has shown criminal negligence for not allowing his car to be fixed

That's not what the OP said. He didn't say anything about downgrades. All he said was rooting and presumably, messing with the system. Neither qualifies as criminal negligence in my mind. I'm not sure I want to live in a society where gaining access to hardware I own gets me thrown in jail? I would also like to emphasize that this has never happened and is totally hypothetical.

Gaining access to safety critical hardware is a questionable right.

What if you are the owner of a building and you decide you want to "optimize" the fire detection/fighting system because you have some ideas in this area, or maybe you want to run an "open" software on it, with better reporting, and as a result some people die in a fire.

Yes, is totally hypothetical, but you seem to be saying that one is not responsible if he does that, because he was just "hacking", with no mens rea.

>Gaining access to safety critical hardware is a questionable right.

It happens every time I put more air in my tires.

(Serious question) But there's no law requiring tires to be able to be inflated by the consumer, is there?

By this logic killing someone while driving drunk is also not criminal since there is no mens rea.

There is such a thing as criminal negligence.

https://en.wikipedia.org/wiki/Criminal_negligence

In your case, the person was being reckless. I don't think gaining access to hardware you own can be considered reckless, regardless of the unlikely circumstances that may result. Your death-by-rooting example is entirely hypothetical and has never happened.

It hasn't happen only because the number of rooted smartcars so to speak is negligible compared to the total number of cars.

Just like drone accidents, each year we get closer and closer to an actual kill as their number increases (https://youtu.be/MvF49R_ZX5E). One day heavier drones will be required to run only certified software if flown over crowds, and rooting those drones will raise the same question.

Absolutely it could be considered reckless. You think some guy messing with his car's firmware understands it well enough to be as safe as the manufacturer? If an accident caused by manufacturer firmware can make the manufacturer liable for an accident, then someone tampering with it obviously can make them liable.

It has never happened? Who cares, what does that matter?

I never said the hacker shouldn't be liable. I said he shouldn't be criminally liable.

By your logic, if the manufacturer puts out software that is responsible for a death, would you throw the software developer who wrote in jail? That's silly.

All this stuff is limited to civil liability.

It really depends on what the developer did. If the developer knowingly removed all tests that weren't passing because the car, sometimes, did not recognize old ladies crossing the street, then this person should go to jail. This is evidently negligence.

Hmm, I see your point. But I don't know how you can draw a distinction between this and other forms of reckless endangerment. A manufacturer at least in principle isn't being reckless. But some random person that fancies themselves an elite hacker messing with things they definitely do not understand seems definitely reckless. And therefore criminal.

I agree entirely.

If I root my phone and accidentally brick it, nobody here would say, "Gosh, that's terrible, it's all Motorola's fault!" They would say, correctly, that if blame is to be assigned, it'd all be mine.

I think it's even clearer with something where even small errors could be fatal. Anybody rooting a car is betting both their life and those of the people they encounter on their technical skill. It might be an excellent bet, but they should be entirely aware that they're betting.

> Anybody rooting a car is betting both their life and those of the people they encounter on their technical skill.

Yes?

So is every guy who pops his trunk and starts working on the car? And people have been doing that since the very beginning. This isn't anything new.

Since upvotes aren't visible, I'll note that I was going to post the same thing. Person puts new brake pads on his/her car, brakes fail (or don't fail), same thing.

Sure, and if you are negligent in making modifications to your car, you will be liable. Mechanics regularly receive criminal charges when their actions cause deaths.

There are three new things here.

One is that automotive mechanical systems are relatively well understood. If you are a mechanic working on a car, it was literally designed for that. The manufacturer makes manuals explaining how to do it well, and they sell extra parts that are meant to be swapped in. There is a century or so of evolved practice an understanding here. This is not true with drive-by-wire software, which is certainly novel, and is frequently terrible.

The second is that mechanical stuff is generally linear and easy to understand. If I change a lightbulb, I have a pretty good understanding of what that will affect and what it won't. Software is highly non-linear. A novice working on, e.g., a device driver, can have effects they will find extremely surprising.

The third is that in software we have a long culture of consequence-free tinkering. We're used to being able to power-cycle something, to restore from backup, to just re-install the OS. Any software person is going to have a lot of habits and biases that become extremely dangerous when working with life-critical systems.

I'm not sure the law about drunk driving has much to do with logic anyway. If I can't drive a car under influence because that makes me unable to evaluate risks and in general to act responsibly - then how can I be considered completely responsible for the choice of driving, if I made it when I was already drunk?

It kind of shows that responsibility and liability, as they exist in our current legal systems, are flawed concepts that are in some cases too weak, and in others too strong. We could really use an update that takes modern behavioral science, neuropsychology, and intended/unintended consequences into account.

But in the case of drunk driving, you were negligent at the point of taking the first drink for not making safe transportation (or lodging) arrangements while sober.

Not all crimes require mens rea. Drunk driving, in particular, is a good example of a serious crime that is strict liability. Normally, however, strict liability is only applied to lesser offenses.

This is in principle no different to people modifying their cars, which they've been doing for about as long as cars have been sold.

Actually longer, since the first cars were the result of people modifying their horse carriages to not require horses :)

Yes, absolutely there should at least be the possibility of the owner being criminally liable. It's like if you drive with shit poorly strapped into the back of antruck and it falls off on the highway and kills someone. If it can be shown that the tampering caused the harm, it's really hard to argue there is no criminal liability.

One can envision various ways of showing that. Loading the custom firmware onto a different robocar, which then exhibits previously-unseen unsafe behavior related to the mishap in question, would be convincing.

Unfortunately from this thread it seems clear that we'll just have some blowhard Tesla manager swearing that "our code is perfect and any change to our code would automatically cause vehicular Armageddon, never mind the fact that we change it on a regular basis!" They won't want me on the jury...

Or you know, you do it like Apple. You make no comment whatsoever, keep the software and let the rumor machine keep you on top of tech news sites.

I do not understand "Good Hacking is a gift" remark.

I would love if Tesla's were open for hacking but a previous article [0] pointed out that there appears to be a strong stance [1] against this.

Disclosure:I am a huge fan of the Tesla brand, just cautiously optimistic.

[0] https://news.ycombinator.com/item?id=11233898 [1] https://news.ycombinator.com/item?id=11234465

Elon Musk is someone who takes personal offense to criticism of his company/products. If a hack is to showoff an awesome new car, which this is the case, I doubt he would have any issues with it.

On the other hand, if some guy found a major security flaw and posted it all over the internet I bet the car would be remotely driven into a brick wall at 100mph+ with the driver inside.

http://www.huffingtonpost.com/2013/06/24/michael-hastings-ca...

> On the other hand, if some guy found a major security flaw and posted it all over the internet I bet the car would be remotely driven into a brick wall at 100mph+ with the driver inside.

That's a bit much. They'd probably just revoke his license to use their software, leaving him with an expensive brick.

> I bet the car would be remotely driven into a brick wall at 100mph+ with the driver inside.

That sounds like something the government would do to your car if they suspected you were a terrorist.

Or a journalist: https://en.wikipedia.org/wiki/Michael_Hastings_(journalist)#...

Or maybe if your natural gas company's stock were in free-fall?

Elon Musk != Tesla

Or at the very least, Elon Musk is the voice of Tesla, but there are many other people who work there. You judge a company by their actions, not their words. I'm a huge fan of what they're doing, but not necessarily how they're doing it.

The cars are absolutely awesome.

Unfortunately, they only provide service manuals in Massachusetts (where they are required to by law) and charge $100 per day to view them. Do not sell parts to the public. Ports in the car (obd/ethernet/etc.) are disabled by default. API is undocumented/proprietary. No access to OS. No access to diagnostic information.

Very similar to how Apple does things.

> Elon Musk != Tesla

He is the CEO, he is responsible for the actions of his employees that are work related. Under the same guise, he shouldn't receive praises for anything everyone else under him does either.

http://mashable.com/2016/02/03/tesla-refuses-rude-customer/

How did someone manage to find the image from the SHA hash?

I just made a lucky guess from the keywords in the article:

    $ echo -n 'P100D' | sha256sum 
    5fc38436ec295b0049f186651ebba5fd55e8d7b81eb61cbd00d3f1bf18dd9c81

It's so short that you can also just brute force it. Using hashcat in "try all short alphanumeric+symbols strings in increasing length" mode on this hash produces the resulting string in about ten seconds on my computer.

I imagine it was just guessed, though. Any Tesla fan thinking of "cool new upgrade to the car, currently secret" would put a P100D model near the top of the list of things to try.

Yep. There are plenty of sites that have already brute-forced hashes for arbitrary n-length alphanumeric strings.

https://md5hashing.net/hash/sha256/5fc38436ec295b0049f186651...

He clearly should have added a salt to that hash..

> P100D

> (a reverse lookup of the hash posted above yields this)

5 days ago, http://www.teslamotorsclub.com/showthread.php/58951-Let-the-...

It wasn't an image, just the text.

If you put 5fc38436ec295b0049f186651ebba5fd55e8d7b81eb61cbd00d3f1bf18dd9c81 into the google, the second link (because first is article about it) will tell you what it is[1].

[1] https://md5hashing.net/hash/sha256/5fc38436ec295b0049f186651...

Do you own your car? Just the metal, but not the software that's required to actually make it do anything useful.

"Root the car" man what an amazing time we live in.

I don't see any evidence that this guy's car was targeted specifically. From the info in the article, a much more likely scenario is that they accidentally pushed an update with private data to all cars, realized their mistake after the tweet was posted, then rolled back the update on all cars, so they could fix it before rolling it out again. That's exactly what you'd expect them to do and not sinister in the least.

I haven't read the 48-page forum discussion linked in the article, so there may be more info there, but at the very least the article writer hasn't adequately backed up his claims.

I seriously have no idea why this is getting downvoted. Anyone?

In TFA it quotes Musk himself confirming OP's claims. Why would Musk do that if they weren't true?

where does it say that? Are you inferring that from his tweet?

He doesn't say that, though. "Wasn't done at my request" means that if it was done, he didn't order it; it neither confirms nor denies that it actually happened. Musk is being cagey; it's possible that he didn't yet know exactly what had occurred or who had ordered it.

Again, maybe there's something in that 48-page thread that proves me wrong. I'm fine with that. But it makes no sense to swallow the article's simplified interpretation without first doing the homework.

Makes sense to me. Guy messes around with highly proprietary computer system, computer system does something weird. I also didn't read the forum, but nothing in the article suggests anything retaliatory. Or maybe the computer noticed something weird, and reverted to a known stable configuration.

$100 a day to view basic service manuals? I will Never buy a Tesla; even if one day I can afford one.

I can't believe you wealthy boys are putting up with this.

Do you guys really want to be sitting on the side of a road clueless over your toy? I don't expect you boys to pull out the DVOM, and Snap-on tools, but a little knowledge of why it broke down?

Isn't it kinda the American way to at least know what the underlying problem is? Or, have we been conditioned into being good obedient victims?

Personally, I feel emasculated if I need to ring a ding ding AAA? Especially, if the problem is minor. Will never know if we aren't able to read up on the toy?

If you can afford a Tesla, you can probably buy a new one before the warranty wears out. And you probably/definitely have roadside service.

Not a very good argument considering they're coming out with cheaper models. Not to mention, just because someone bought something doesn't mean they can afford it by a wide margin, it's sad but true. I think the poster you respond to has some points on why it isn't optimal for the end user, but they certainly could have phrased it a bit better.

I'm not saying I endorse this position. (I'm amazed I got downvoted as if I did.) I'm just saying this is why people have put up with it: Almost nobody realizes it, because people who buy Teslas generally aren't servicing their own cars, and most of them are still under warranty and they'll probably upgrade before that date arrives.

Same thing with Apple devices; locked down, no user-serviceable parts, replacement parts not sold to consumers...worked out pretty well for them. Most people just don't care and want the New Shiny more, and that's enough to persuade them.

It's much more feasible to replace a phone every 2 years, as soon as it starts to show significant wear, than it is to do that with a car. We know Tesla plans to sell to the full mass market (Toyota price point) eventually, so they're going to need to allow for long term maintenance and independent repair shops at some point.

Note that Tesla's already supporting body shops.

The "cost" of service information is hardly unique to Tesla

Audi: https://erwin.audiusa.com/erwin/showOrderFlatrate.do ($250/m)

Tesla: https://service.teslamotors.com/ ($350/m)

BMW: https://www.bmwtechinfo.com/ ($250/m)

This is true. It's also true that I can access service manuals for BMW and Audi.

However, I cannot access the service manuals for Tesla. Because I don't live in Massachusetts. Does that seem right to you?

They even make sure to verify your address by mail to make sure you really do live in Massachusetts before they grant access.

We detached this subthread from https://news.ycombinator.com/item?id=11255679 and marked it off-topic.

Reply on news.ycombinator.com