Let the worm spread for a while through the fleet and activate a malicious piece of code at a set moment that accelerates and steers the cars into an object. There's not enough medical personnel to tend to all these accidents. Total chaos.
Am I wrong in thinking that with the passing of time the probability of such an event tends to 1?
You're forgetting to factor in the human element. Technology doesn't progress independently. There are dampening effects when the "real world" decides technological possibilities don't fit the world they want (for example, copyright applies artificial limits to the infinite copying potential of digital assets, or the recent EU politics around AI).
So I suspect relatively isolated cases like this will eventually lead to a push for legislation on automotive digital security. Cynically, I suspect in a way that raises barriers to entry to the market after the incumbents have secured their market share, but that's still probably better than the alternative.
I wonder if, as we do at the moment with human piloted cars, we'll just shrug it off and offer a passing "poor human" (in the case of injury/death) and continue with our day.
Death is around us 24/7 and I am not convinced even if cars were hacked and told to drive into objects we'd care very much, we'd probably fix the bug and move on.
Tesla is a software company and probably has a lower chance of getting hit due to expertise and funds being poured into security (even tho not infallible as this post shows), but there’s a race to the bottom and soon enough Car companies that couldn’t pull of decent navigation will have some form of computer-controlled-steering as stockholders are looking at Tesla stock price and breathing down their necks.
I have been worrying about this for some time, even tho I am also a tech lover and Tesla driver. was thinking of writing a blog post about this, but seeing this comment, maybe it isn’t a new thought and everybody is already aware of this risk.
I think you are right about medical personel. But also: if you make everyone crash around rush hour, you take out a significant share of the working population. And how do you clean up the infrastructure to let trucks and ambulances through again. Not to mention the catastrophe of cars crashing into stores and pedestrians in city centers.
Fixed that for you.
No good for self-driving cars though.
What's the bigger issue?
Adversarial manipulation of the sensor inputs. This can be equated to verbal or visual manipulation in humans, something that becomes much harder to detect. While most of these attacks would be against a local target, I could also see a widespread deployment that goes unnoticed and slowly degrades many neural networks, and those being erroneously propagated.
The latter is a much bigger problem than "worms" because it's effectively invisible. We can audit and identify malicious code, there's an entire industry built around that. But, neural networks are for the most part still a black box solution. How does one detect and solve manipulation in a black box solution?
Well, maybe my Tesla will meet your Tesla in therapy and they can talk about it.
What are those reasons?
Edit: downvote for what?
The truth is the moment it's technically possible to hack a car remotely, cars will be hacked remotely. We've had computers of all kinds for decades and couldn't manage to make them "hack-proof". Consoles are as close as it gets and I'm sure if they were as critical as a car they would have been thoroughly hacked by now.
Having any kind of "self driving" feature means safety critical systems (acceleration, braking, steering) can be controlled entirely by the car's computer. And having OTA updates means there is some link between that critical computer and the outside world. And in that outside world people managed to hack airgapped computers in a military nuclear facility. If only that facility was "a software company"... they could have CI/CDed the malware in their infrastructure.
Contingency plan: go back to the technology of 50+ years ago. Remember the old unwired unhackable Battlestar Galactica vs the newer ships hacked by Cylons.
Something like https://www.wired.com/story/how-30-lines-of-code-blew-up-27-... applied on wide scale to a developed nation's power infrastructure has the potential for enormous numbers of deaths without the "well obviously the rest of the world will hate us" consequences of nuking someone.
I would imagine that's a certainty.
"sufficiently capable nation state..."
Which I doubt is as many as you're implying given the use of the term "certainty."
"... with the intent to directly harm average citizens."
Which yes, I'd argue is a negligible amount.
But, private corporations can be wound up. Nobody is ultimately obligated to maintain this kind of work.
If shit hits the fan, then it's not obvious that "Tesla, Inc." will stick around to deal with the consequences. (If it becomes medium-term unprofitable, then it seems to me obvious that it won't.)
I'll add that while I think it's unlikely ANY incorporated publicly traded business would stick it out to deal with the consequences... TESLA seems to have treated medium-term unprofitablity as a consequence of failing to meet quality and production goals without heavy divergence from long-term profitability plans.
No code required.
Yeah, if you have visions of the Joker hacking control of the Batmobile turning into a large RC car then we are safe from that. But that’s far from saying that remote control isn’t an issue.