Self-Crashing Cars
zachaysan.com
zachaysan.com
https://www.wired.com/2017/02/malware-sends-stolen-data-dron...
Also reminds me of the stories of reverse engineers using, lights, motors, speakers, etc to exfiltrate boot images.
> "If hacking a Jeep is as straightforward as hacking a server, and servers are routinely breached, then where are all the hacked cars? It's a bit like the Fermi Paradox."
One barrier to entry is the actual cost of acquiring a vehicle. You can buy a lot of iPhones to hack on for the price of a Tesla.
> "We think of Teslas as cars just like we think of an iPhone as a phone, but a more accurate account of reality is that they're both just computers."
It's actually kind of worse than that, cars are actually rolling datacenters with multiple computers and multiple networks. (CAN bus[es], Ethernet, proprietary networks)
//Disclaimer: I work for GM, but not on any of this.
Hacking a car to kill people is a lot different than kicking someone out of a video game. Even people who SWAT don't anticipate their rival/victim being killed.
They want the person to be miserable, not dead. It's cruel and vindictive, but it's not homicidal. Or at least not most of the time.
The most recent high profile one being Michael Hastings(https://en.wikipedia.org/wiki/Michael_Hastings_(journalist)#...)
Could be paranoid nonsense. But white hat researchers have demonstrated a disturbing number of attack vectors that could be used for this sort of thing.
I was simply trying to answer the question of why this isn't happening all the time.
If it is happening all the time (a point I won't dispute) then the question is wrong.
This may be happening, and just being written off as user error like all the rest of the massive number auto accidents we live with everyday.
Sounds like you've never been to 4chan.
That hasn't been true in a while. Most malware these days is about money or political goals.
Assuming you're just a random person and not a serious organization (which would have money to buy cars), can't you just get a job at an auto repair shop to fool around with their computer systems in the evening?
source: I work on this, but not for GM.
http://www.alldata.com/alldata-repair
Protip for home mechanics, get one of these: https://www.amazon.com/Reader-Diagnostic-Check-Engine-Light/...
You can dive into polylith forums and find guys that thinker with the code in the OBC to 'improve' some of the aspects of the car. Think Dieselgate, but more hacker-y/backfire-y.
Example for Scoobys: https://www.carid.com/2017-subaru-wrx-performance-parts/?fil...
(1) Admit we have a big problem.
(2) Admit it will be difficult to solve and there aren't easy solutions.
(3) Start working on solutions.
Unfortunately, these steps never seem politically popular. But it will be tough. Except in very rare scenarios (NASA), software security and correctness aren't treated as important in the scheme of things, relative to functionality and features. Nobody wants to invest in actually secure, carefully-audited code. (They might claim they do, but their standards for "secure" and "careful" would be litigation-worthy in any field but software.)
For example, I believe in freedom of software and ownership rights, which seems to mean that I believe people should be allowed to run open-source code on their own cars' computers. But this impacts public safety. How do we develop reasonable regulations similar to those for physical "street-lega" modifications?
Based on years of working with electronic engineers' code. (Apologies to those of you who are competent in both)
My broader point is that this (among others) is a discussion we should already be having and eventually must have -- if not amongst "ourselves", then with legislators and laypeople. Yet the state of software security, awareness, and incentives is such that it could be years or a decade until we do...
The big problem is allowing firmware updates.
Many years ago, I knew some of the people involved with the Ford Electronic Engine Control IV, the "EEC IV", used in 1980s Fords. This used a custom CPU, an Intel 8061, which was an Intel 8051 with some extra timer features. The program was etched onto the CPU silicon. There is no way to change that program without replacing the whole unit. There's an external ROM with some tables, different for each engine model, but it's a true ROM, not something that can be rewritten. If you want to replace it, it takes a wrench.
The level of paranoia and testing that went into that program was very high. Any bug meant bringing back hundreds of thousands of Fords in a recall. That didn't happen, and there are still many EEC IV vehicles running today, 30 years later. That's what it was like in the days of hardcore embedded programming.
I'd argue that safety critical vehicle software should not be downloadable. If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system.
Autonomous vehicles should not communicate with each other. Waymo's cars don't. Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving. Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple. (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)
You seem to be out of touch what happens today. Car2Car or Car2Infrastructure is about as aircraft to aircraft communication is used in TCAS. It is about what car (object) is at which position, so that collision may be detected as early as possible to prevent them.
Also next generation cars will have their maintenance modes locked down as far as possible. OBDII will be closed down as far as possible. All those dongles, which are hip right now, will be rendered useless. But of course the tuning scene has still a lot of incentives like in the past to thwart all this and to reverse engineer. It will be a cat-and-rat game like it always was.
http://www.templetons.com/brad/robocars/v2vdata.html
Pithily, "It can never be that useful because if it's being really useful, something is very wrong."
ehhh, cars shouldn't be communicating with cars they can't see. but i'd like to them to communicate over free space optical, aimed at the level of other vehicles. brake lights could be modulated to contain current braking force, and reason for braking. turn signals could contain the destination lane. head/tail lights the current speed, and target speed.
Seriously, "let's have everyone jump on the bandwagon all at once, and we shall be redeemed by $TECHNOLOGY_DU_JOUR" is a very brave proposal.
Given that it seems like a foregone conclusion that these things are going to end up with some WAN interface [0], pure ROM is out of the question. The best we could probably do would be to assert that the flash could only be updated through a local debug port, with protections against eg full RCE allowing remote re-flashing.
Given physical access, it's trivially easy to sabotage a car - the new problem here is having it done remotely. More important would be some kind of auditing so that a local debug device could be assured of reading out the exact code image, knowing how many times it had been flashed and ideally checksums, and being sure that a reflash reset the system completely.
(btw, loved "shit early, shit often")
[0] I mean, I'd personally rather they not. But it seems like a foregone conclusion of trends/marketing/sloppy design/implementation shortcuts/can't stop the signal.
As for surveillance related to V2V communications, I fail to see how it meaningfully adds any surveillance over just monitoring the other electronic emissions from the car like those that emanate from the media centre or even the phones of those within the car.
This doesn’t follow.
> including improvements related to security.
This should not, under any circumstances, be a reasonable justification. Cars should not have any external attack surface, period. The ECU and any other critical electronics should not be networked at all. Unfortunately, that ship has long since sailed.
It should be possible for cars to be updated, but it should require physical access, and any mandatory update (e.g. due to a safety failure) should result in the manufacturer incurring heavy costs, e.g. you have to give every owner $1000. We have to internalize the externalities of shitty software, especially in stupendously dangerous machines like cars.
> This should not, under any circumstances, be a reasonable justification.
That presupposes that humans cannot make mistakes, which they do. Whether it is hackable software defined radio, or just plain oversights.
The levels of abstraction that programmers deal with do not allow for the necessary security you propose.
> Cars should not have any external attack surface, period.
What you advocate for is a car with no ports, no radios, and a perfectly secure component manufacturing chain. It isn't doable. There is a reason the ship sailed.
They're expensive, annoying, etc. But they're safe (or safer, at least), and if you screw up badly enough that a safety-critical system needs an update, they're necessary.
I don't think anyone is arguing that the infotainment system should be disconnected from the world, but the core driving systems (steering, brakes, transmission, etc) should be air-gapped and inaccessible without physical access to the interior of the car. Also, that access could (and maybe should) be disabled when the car's engine is running, preventing an attacker from plugging in a wireless device and then subverting the system with the car in motion.
In any case, I agree that it's unlikely unless there's a severe disaster related to OTA updates, but it's still the best solution from a security and safety perspective.
I don't think so, just that the radio and other non-critical components mustn't be connected to the critical components. Essentially the radio becomes like any other portable device, that just happens to be physically installed in the dashboard.
If it is discovered that any model of car is vulnerable to this, it needs to be recalled and scrapped.
> The levels of abstraction that programmers deal with do not allow for the necessary security you propose
This is true of the kind of shit pumped out by low-quality commodity hardware and consumer-focused software companies. Anything controlling millions of multi-ton missiles should ideally be formally verified top to bottom. Exceeding NASA vehicle code standards are the sort of thing that would be appropriate here. However, just the plain old standards we apply to the embedded code in ECUs plus full hardware isolation (i.e. no “start from your phone” bullshit) would be sufficient.
> What you advocate for is a car with no ports, no radios,
No radio attached in any way to the critical hardware is sufficient. The goal here is to defend against bulk remote attacks (the kind that are actually dangerous in a noticeable economic sense) and bugs, not evil maids.
I'm not sure I'd go that far. OTA "automatic update" probably not a good idea. Downloadable/flashable at the dealer OK, no need to go as far as swapping out ROM modules.
Following avionics software engineering standards should be absolutely required. Entertainment/nav systems that require network access should be 100% air-gapped from the vehicle control systems.
How cars were designed 30 years ago is completely irrelevant, because that's not how all or most new self-driving cars will be designed. They will be designed so that the software can be changed on the go.
For example, cars being able to broadcast their immediate path and do some dynamic route mapping based on which lanes and areas will be in use by the time it arrives; being able to agree with nearby vehicles about routes that are partially colocated so they could join together bumper to bumper for that leg and realize fuel efficiencies (I have no idea if that would actually be useful but I’m brainstorming possibilities); being able to negotiate a car parking space in advance of arrival...
Is it that there actually aren’t many applications where inter-car communication is useful, or is it more that the benefits of communications like these are outweighed by risks to security, or am I missing important pieces of the puzzle?
In my case, for example, I have a 2014 and a 2015 car which are both more than a year overdue for a software updates. One is related to the braking system; the other to a problem where if you stomp on the gas in an emergency (like to avoid a crash), the car stalls.
The problem is that the dealers in my area don't have the capacity or the will to do software updates. One dealership only has maintenance hours between 9am and 4pm Monday through Thursday, and the ONE guy who is allowed to do software updates is only in on Saturdays from 9-noon occasionally.
The other dealership doesn't take appointments at all. It's first-come, first-served, 8am-5pm Monday through Saturday. I stopped going there because even at 7am, there's a huge line of people waiting for service. The final straw was when I sat there from 7am until 3pm for a very simple non-engine part swap that was covered under warranty.
The next nearest dealer is 138 miles away.
tl;dr version: OTA updates are good because there are places where you can't get a software update done at the dealer in a reasonable time.
Unfortunately there’s no quick fix about the current cars having the “guided” self driving principle.
(randomly, as in "where do you want to go today? meh, nevermind.")
Better yet, the software should be uploaded to an intermediate agency, that is responsible for the testing of cars (could be part of the US Department of Transportation for example). Only this agency can (after sufficient testing) upload the software to the actual cars.
In my opinion the demand for a government controlled kill switch in every piece of hardware that is somehow able to harm people is much more threatening in so many ways than the insecurity the author is trying to reduce. Just a few:
1. Based on the asumption, that ones current government/state is for the good of all people, what gives him the confidence that this will stay that way? What if your beloved government goes rogue? That's a lot power for an autocratic regime.
2. Why even trust the government in the first place with that kill switch? They are the same people which are careless about infrastructure critical ITSec since decades.
3. An univervsal security module which is highly standardized is a very profitable target. While the author is aware that finding an attack vector of one particular vehicle can mean that all vehicles of that type can be compromised, he doesn't come to the conclusion that the same logic applies to his security module.
Here is where I still sit, however: the government had predator drones and will soon have killbots able to take out individuals based on facial recognition software.
If we can’t trust them to turn off our cars we’re fucked anyway.
Let's say a security vulnerability has been discovered. An attacker has wormed their way into as many cars as they can. They send a 'go' signal from their command and control servers. Across the world, cars start looking for opportunities to kill their passengers and bystanders.
We send the shutdown signal. The safety modules wake up and take over from the compromised computers. What do they do?
They could brake. However, the car might be in a turn, on a rainy day. Braking could send the car into a skid and kill its passengers.
Maybe they brake, but slowly. However, the car might be behind someone who just suddenly changed lanes and slammed on the brakes. Braking hard might be the right move in that circumstance.
Maybe they do something conditional on the sensor input they get. However, if the control computer can do something to 'blind' the safety computer, that doesn't help. For example, can the control computer issue firmware updates to the camera sensors? Can the control computer fill a sensor's bus line until it can't respond?
I can't think of any solution to this that doesn't involve duplicating a substantial part of the control computer.
And a few thousdand died on September 11th, nearly 20 years ago. The collective fear from that made the world a far worse place than the actual death toll. If there's a terrorist attack that targets internet connected cars, what do you think the collective reaction might be?
I contend that 40 years from now our grandchildren will be astounded that we got into such dangerous vehicles before self-driving cars.
[1] - I'm being facetious, of course.
It's fortunate that terrorists are both very incompetent and very low in number. How else do you explain the fact that there's been exactly 1 very serious and successful foreign terror attack on US soil (the highest value target in the world) in the past 50-ish years?
Currently there's the very possible opportunity of a power grid/infrastructure attack that could kill tens of thousands. But nobody should be truly worried about it.
Terrorists just aren't that good at what they do. Why would they somehow be better at hacking cars than they are at anything else?
The Oklahoma bomber killed 168, for example. The first one that came to my mind.
https://en.wikipedia.org/wiki/Terrorism_in_the_United_States
Most news organizations report on foreign and domestic terror differently. When the average American thinks of a terrorist it isn't a white guy with a U-Haul.
If that happens, car crashes aren't the problem - it's the widespread famine that follows in a week's time.
This trend of doing everything over the Internet for no good reason other than business model is growing from just ridiculous and user-hostile into something that's actually dangerous to people's lives.
Thus it is going to be hard to avoid a network connected car and it seems likely that network will be the Internet.
Well, I agree that maps as a second source of information can be important for autonomous vehicles. However, I don't understand why "map" would imply "network connected". Offline navigation systems with detailed maps have existed and still exist for more than twenty years now. I fear that the "offline autonomous vehicle" will solely fail to manifest because of business decisions (online being "more convenient" for both the end-user and the company), not because of technical limitations.
Edit: which is to say, maps aren't really secondary parts of current systems but more like "co-primary" parts. The cars aren't planning identify traffic lights where they don't expect them.
"Almost all of the fully autonomous vehicles currently allowed on public roads are still under the direct supervision of human pilots, and they’re only driving on roads that have been heavily studied and mapped in three dimensions."
See:
https://www.consumerreports.org/autonomous-driving/self-driv...
or "Cars will only be able to drive themselves if they have access to high-precision maps. The digital material contained in today’s navigation systems is not enough. To be able to drive itself safely, a car needs to know its position on the road down to the centimetre. "
https://www.mercedes-benz.com/en/mercedes-benz/next/connecti...
Comma.ai, on the other hand, feeds their AI the camera feed and the sensor signals from the car, and it responds, as far as I know, almost entirely based on that stimulus. Of course, Comma.ai's car is presumably less predictable, it relies on a black box to "think", but you could feed it the general concept of what path to take from A to B, even a set of waypoint GPS coordinates of where to turn, and hypothetically, such a car could navigate to that destination otherwise offline, or with the grade of maps reasonably available offline. It's intended to drive like a human drives: Based on the information it perceives in the world around it.
When a dashboard GPS is out of date, it's irritating. When your car's autonav system is out of date, your car is useless.
Specifically, look in the article:
>First allow me to address what I think won't work:
>Trying to air gap autonomous devices.
For the purposes of outside knowledge queries you might not be able to come up with in advance, there's good cause to outsource those rare requests out to the Internet: Just do it intelligently. Require a prefix instruction for an outside request.
For instance, I went ahead and implement Wolfram's API for knowledge queries. They have a great "spoken answer" endpoint, which replies with a string meant to be piped straight to speech output. So I "ask wolfram how tall abraham lincoln was", my program hands everything AFTER "ask wolfram" to the Wolfram API, and Wolfram's API gives me a string back with exactly what I asked.
Now sure, I'm not entirely offline at that point, but everything regarding my personal data, home automation devices, etc. is under my control, and any time I reach out, it's specifically using a command authorizing it to do so.
Of course, caveats before you think my project sounds impressive: A. It's written in Visual Basic. B. Speech recognition isn't working (yet).
Visual Basic is a fine programming language with very weird syntax. Hell, Microsoft has put out .NET bindings for CNTK so you could even use that.
https://www.forbes.com/sites/thomasbrewster/2015/01/15/resea...
The dealers don't know about the cars at this level of detail. I had to use these questions to pry the data out of them:
1. Is this a connected car? Can I unlock it or start the engines from my smartphone?
2. Is that feature an option or is the feature part of the base model?
3. Can I get the connected feature later? Would I have to bring the car in to get something installed or can you enable them from your computer?
#3 is a really important question. Subaru (and perhaps others) ship all their cars with the hardware for connectivity, but the actual feature requires a subscription. The car is always connected to the internet, because the dealer can start your subscription remotely, but the salesmen don't understand that implication. They're thinking solely in terms of features you're getting, not what hardware the car has.
[1] It's almost impossible now to buy a new car without passive keyless entry that isn't a bottom-tier economy model, despite the well documented security problems with many of those protocols.
The Toyotas Camrys also seemed good, except I couldn't find a V6 to test drive and they didn't have Android Auto.
Just a note: I'm not super-paranoid about my car getting hacked. I didn't want a cellular modem because I know I won't use whatever features it enables, and I didn't want my car to be ransomed for a bitcoin. I didn't want passive keyless entry because it didn't seem like much of a convenience, and it weakens security against petty theft.
My criteria, in priority order:
Must haves:
* V6 engine
* No cellular modem
* No passive keyless entry (not really available anymore, so I had to drop it as a dealbreaker)
Nice to haves:
* Android Auto/Carplay
* Adaptive cruise control, etc.
Given the above, convince me I should care about how hard it is to break into passive keyless entry cars?
Once an attacker can remotely hack a single car, they can hack all cars that have an identical configuration, with little additional cost.
What happens then? Even if insurance companies could replace all affected cars simultaneously (very unlikely), they’d have to replace them with a model that isn’t affected.
Anyway, even if it isn’t autonomous, suppose a car has a smartphone app that allows you to turn on the heating before getting in. And then someone exploits that and gains control over the heating. They could then proceed to drain the batteries or the fuel tank by leaving it on over night, let’s say.
Not exactly a threat to national security, but still a major inconvenience.
I did not confuse anything. I only mentioned passive keyless entry in a footnote, as an example of an insecure technology that you can't really avoid anymore. You still have a chance to avoid "connected car" features, but in my estimation the days are numbered for that.
And do what with them? It's decently difficult to fence a single car, it's impossible to make millions of cars disappear.
Btw, the truck attack in the German market would've been a much larger scale if the truck's safety mechanism didn't trigger the emergency break.
Regardless, that Jeep hack sure proved a clusterf*ck of system design, and should be a cautionary tale.
You wouldnt buy a car that is safe- because bad security does not smell.
> They never had their car stolen and if they did, their insurance would have replaced it.
You'll be a fun thicket convincing your insurance your car was hacked. Especially if it turns out to be true. Nobody has rulebooks for that one yet.
If you have really good insurance, perhaps you'd end up driving a loaner for a few months.
I don't really care if you care or not, it's your car, but I care.
Also, the issue with passive keyless entry isn't just theft of the car itself, its more often theft of its contents. It makes break-ins much easier to do undetected.
Are you comfortable with sharing your reasons? I feel similarly but do not have concrete plans quite yet.
From my personal experience, the service departments will happily download data from your car and sell it to the highest bidder. My case was simple: the dealer updated the mileage record in Carfax using the odometer reading from my warranty-provided oil change. The car is leased, so I'm 99% sure I had no way to opt-out.
Sounds innocent, but my insurance company was watching. They extrapolated the mileage and decided I would cross the 7,500 mi/yr threshold - which triggered a premium hike. Funny thing - I didn't exceed 7,500 miles that year, but they already have my money now.
What else could a dealer read off your non-internet connected car when you bring it in?
This is just my own unorthodox methodology. Your mileage may vary.
(The best transmission shop I even saw.)
I thought that the article taught me nothing new at all, but then realized it did: it taught me how little the issue is understood by the regulators.
I'd say that an attack like that is just a matter of time, if I didn't think that a mass-destruction scenario that leads to a legislative change wouldn't happen sooner due to a bug.
- Live in the country and get really good at repairing standard transmission cars and dodging emissions inspections, or
- Live in the city and get really good at dodging in general (it's a city, you'll learn that skill anyhow)
Blackhats, in stark contrast, are sexy.
Bounty hunters are sexy. Why doesn't the government start paying bounties? Paying bounty hunters to run honeypots would convert many black hats into hunters of black hats. This could well create an ecosystem where the more knowledgeable hackers directly prey on the script kiddies for fun and profit. Taking useful idiots out of the ecosystem strikes me as desirable. Such a program would also be useful for recruitment.
In a way, this is analogous to such bounties in the transition of the wild west into a more normal society.
I thought of that too. This also happened with bounty hunters. There will need to be safeguards.
They employ NSA workers to find and exploit them instead.
The only evidence I could find was this study: http://journals.sagepub.com/doi/abs/10.1177/1087054706288103
Although I can't access the article, the abstract says that teens with ADHD had higher attention to driving when using a manual transmission vs an automatic.
That's quite different from your laptop or a smartphone, which can run pretty much any code.
It would be very very very hard to hack something like that. As in, steal the signing keys from Tesla hard without being noticed. I wouldn't be surprised if their signing server is air-gapped.
And even if there's a firmware update mechanism available, the manufacturers' abilities to maintain older software will also degrade, like legacy systems do everywhere.
This interested me more than the idea of car-bombs, are we moving towards a post-law society? Thinking about it, I haven't seen a single piece of legislation regarding self driving cars, cryptocurrencies, or shared computing in my country; and are transport, currency, and communication not the underpinnings of civilisation?
Then I noticed he'd recently published an article on Zero Width characters being used for fingerprinting which got some attention on HN in the last 30 days: https://news.ycombinator.com/item?id=16046329
(The crash wasn't caused by remote control hacking -- that's an absurdly complicated and high-profile way for well-funded professional murderers to murder a single relatively minor person. If we was murdered, it was by some kind of mechanical sabotage or by impairing his mental state with some drug.)
I knew what you meant only because I had read the article.
(This is just a cultural aside. I know there is a world outside of the United States.)
Many imagined terrorist threats are threats only in a vacuum. A rifle and a tall building resulted in 50+ deaths and 500+ wounded. Any technologically complex attack has to beat those numbers.
The only notable attack with an actual WMD was the sarin gas attack on the Tokyo subway. It killed about one fourth the number of people as a man with a rifle in Las Vegas. It took a secretive cult organization to implement. That's why nobody uses WMDs for terror attacks.
Trucks and rifles do enough damage without even having to acquire bomb-making skills. Autonomous vehicles that are programmed to avoid hitting pedestrians are likely to make it harder to use vehicles as a weapon, not more deadly.
You did not read the article
You figure out how to hack it over the internet and load a program that uses its cameras to identify a pedestrian and drive straight towards it at full speed. You turn on this program at some busy time of day. There are times of days in which there are tens of thousands of late model Camry's on the road in the US, and this is the very first example I could think of when considering the problem raised in the article.
Assume you have a fleet of self driving cars on the road deployed throughout the nation. A vulnerability is discovered that allows an attacker total control. They deploy that vulnerability, and cause every single car in the fleet to accelerate and aim for pedestrians.
So the next bit we have to ask is:
How many self driving cars would be on the road at the time this attack gets deployed.
Multiply this with:
How likely the driver/occupant will die in this event, how likely is this event going to kill or harm pedestrians, and how much structural damage would be done?
Play with those figures for a bit. Personally, what I'd consider reasonable guesses at these numbers results in some very large bodycounts.
(That said, there are issues around actually reaching these machines, because a lot of them don't have routable IPs and may not load your attack web page. Similar issues may arise for cars, which sure would be nice as a mitigation.)
Even so, 0.5% of 1 million is about 5000 people. Not quite WMD territory, but also way out of typical "car bomb" territory... Maybe the mitigating factors would mean the actual fraction would be even less. But it would be nice if we didn't have to hope so.
To my knowledge, there's no evidence that Clinton's email was breached. The DNC, sure, but the Clinton email controversy wasn't based on any actual known breach. It's sort of alarming to see this weird retconning of history.
This is extremely dangerous thinking. Passwords and tokens are not in the 'security by obscurity' category because you can observe how the entire system works, review its source code, read its deployment configuration, and do packet captures of the encrypted valid traffic and still not have a way to gain access to the system.
Security by obscurity refers to hiding how the system actually works, and that's a lot harder to keep a secret because you are one compromised device away from revealing all of that and it can't be easily changed.
Do not call passwords and tokens "security by obscurity" or claim security by obscurity practices (e.g. Running on non-standard port numbers) is on the same level as passwords/tokens/keys.
There is a reason the strongest crypto is using public algorithms and only private keys. Obscuring a system just means that your silly bugs don't get found and exposed early.
Was that meant to be a joke? Sure, they can establish a new standard that will apply to all the new car models coming out four years later. But who actually expects hundreds of millions of cars that are already on the market, to receive a software overall with a new architecture?
This is why it has always bothered me that almost no one seems to bring this problem to the forefront - certainly not carmakers. They're all too focused on how awesome self-driving technology will be and how it will save us from drunk drivers. Thus, disregarding the fact that once we have 100 million to 2 billion self-driving cars on the road, that will be a huge market for cyber criminals, from ransomware and cryptojacking (hello powerful GPU computer + free solar power charging!) to assassinations.
And before anyone says "how much harder it is to hack a car than a PC", consider the fact that most cars today aren't actually connected to the internet. And most of those that are, only have their entertainment systems connected to the internet. Self-driving cars will be able to receive OTA updates that will improve their engine, steering, and brake performance = the OTA software has access to everything.
Combine this level of access to the high level of recklessness in the name of profits carmakers seem to be showing today, when they advertise features such as "unlocking your doors through an app".
EFF's former chair and someone who worked on Google's Waymo, has some decent ideas about how to protect self-driving cars, if only carmakers would listen:
http://ideas.4brad.com/disconnected-car-right-security-plan-...
We need ways of disabling autonomous devices and detecting when they get hacked, not trying to win an impossibly hard game.
> 1. Sounding like a crank.
> 2. Giving ideas to terrorists or hostile foreign governments.
#1 is this article's biggest problem. These problems are real, but I think the author sounds like a crank. (Based on the way this article is written, I suspect that the author actually is a crank--obsessed with security, but fundamentally lacking in the relevant skills to do anything about it.)
If the author is serious about this, here's what you do.
1. Become/join a non-profit organization. You want to be frequently quoted in the press, but not as "well-known software security expert Zach Aysan" but as "Zach Aysan, president of the Organization for Global Security."
2. Build your reputation by finding and earning credit for security problems. Do ethical reporting, but when the issues get fixed, exhibit them in a flashy way.
3. White papers, not blog posts. This "article" starts with a subtitle "with apologies to Elon Musk", followed by a personal dedication to Zach's father. This is not the tone of a white paper from a think tank.
The subtitle of the post should be the thesis statement of the article. "Self-driving cars lack adequate security protections." The first paragraph should be an executive summary of the argument of the post. Each paragraph should support the thesis statement.
Have someone read your articles; update your work based on their feedback. Thank them in the footer.
4. Separate "how to" articles from arguments. This article is long because it is both attempting to persuade the reader that we haven't invested enough effort into securing critical systems and also to give a list of proposals. These should be separate articles, with one article arguing why security is important, and another article giving a list of proposals.
I don't know, but I had the same idea years ago, and I later read about an author using the idea in a 2006 book (Daemon, by Daniel Suarez). I'd say this shouldn't be an issue, and it also solves number two: if we can think of it, so can hostile governments* and mad men.
That said, the article does (after he gets to the point of "weapons of mass destruction") seem to get a little obsessive and, yeah, crank-y. But that has little to do with the concern itself.
* Not "hostile foreign governments" because every government is foreign (and perhaps even hostile) to someone.
I think you have to be paranoid, really paranoid, to be able to think about this kind of thing. "Normal" humans seem to have psychological stability mechanisms that operate pathologically when confronted with events that are too far outside of their world-view. It's fine to disbelieve Bigfoot. It's dangerous to disbelieve e.g. Stuxnet.
Even the author:
> It wasn't until I'd read through the code [of Stuxnet] myself that I finally believed that it had actually happened.
Is it a failure of imagination? Head-in-the-sand-ism? Just not paranoid enough?
To everyone reading this, if this article sounds like a crank, please re-evaluate your world-view. This is the clearest and most well-written description of the problem I've yet read.
James Bamford has made a long and distinguished career of writing about the NSA with his first book coming out in 1982.
Society needs to stop replacing any attempt at understanding with deference to "someone who knows computers", etc.
The solutions are also very very wrong, he proposes a state enforced kill switch for every device capable of autonomous operation. Abusing that system seems like a much greater source of havoc than individual compromises of certain brands.
His main point is pretty insightful and well argued, giving life and death powers to very fragile software on a massive scale will end in disaster. The computer industry is woefully unprepared to deliver hardware, software and systems with the provable integrity many of these applications require.
I dunno, we already use software to run trains and it works OK. Gone are the days of track circuits and physical protuberances on the track to open the brake tube when a train tries to go where it shouldn't. Modern positive train control uses 802.11-based wireless networks communicating between trains, and as scary as hacked-up WiFi routers sound for life-and-death applications, it has a better safety record than the easier-to-understand previous-generation technologies.
https://en.wikipedia.org/wiki/Communications-based_train_con...
Complex software systems _can_ be successfully implemented. I might even be inclined to trust them more than cars driven by people that got an hour of sleep last night, or people whose financial incentives line up in such a way as to make driving 60mph on residential streets a smart business decision.
I guess, a few years ago anyone who would assume that Facebook can be used to propagate fake news and influence presidential elections would've been called a crank. I wish we have more "cranks" like the author. And of course when it comes to a chance of massive coordinated terrorist attack it's better to have many false positives (no matter how annoying they are) than overlook a real problem.
I think that is also a big reason why many potential black hats do not become black hats. The only 'sustainable' black hats are ones associated with a criminal organization or nation states.
Also, for computer security programmers need to overcome convenience practices and start behaving responsible, but also people in management/politics should not make technical decisions if they can't understand the implications.
During the late 90's and early 00's I often wondered why with all of the Microsoft hate why someone didn't create a simple virus that did something destructive like just format windows hard drives. There must have been other incentives at play that caused this to never occur. Perhaps in the same logic if you can infect every Tesla it is more valuable to not crash them, but instead scrub the data and sell that back to [insert company] or something.
If there was ever something that would cause a formal programming guild to sprout I would be willing to bet that it would form its roots around security.
Sometimes viruses are written to by self-limiting. MyDoom, which caused about 10% of all email traffic for a time, was programmed to deactivate on a certain date. Also once viruses get to a certain level of infamy they get a lot of attention. Blaster was mitigated in just days, so by the time it started its DDoS it was already mostly wiped out.
Personally I think the robot uprising concern is at least 50 years premature. But the "what if someone could take the controls" concern is a concern we should have been already been considering yesterday.
From personal experience I can tell you that people with STEM backgrounds can also be too impatient to read an article this long.
edit:
> (Bio) After years of building startups and advising organizations large and small on data science and cyber security, I'm turning my focus to improving public literacy on technology.
Starting your article like this is a bad way to do it.
edit: Fooled me, looks like it's been corrected.
I imagine (some of) the downvotes are from people with a STEM background that didn't even open the link