Tesla Tests Self-Driving Functions with Secret Updates to Its Customers’ Cars | Hacker News Reader

Tesla Tests Self-Driving Functions with Secret Updates to Its Customers’ Cars

technologyreview.com

43 pointsprostoalex50 comments

For those reacting to the headline and first few paragraphs:

>Tesla’s engineers initially test new self-driving software against those records. Any that perform well can also be tested by secretly installing them onto customer vehicles and watching how they respond to conditions on the road, although the software doesn't actually control the car.

>“We will often install an ‘inert’ feature on all our vehicles worldwide,” said Anderson. “That allows us to watch over tens of millions of miles how a feature performs.”

The article is sparse on details, but this little bit buried in the middle certainly seems to allay the fear I (and others here) immediately felt. It looks like Tesla tests software, then pushes it out without giving up any control of the car; the software simply logs what it would have done.

This is huge. No matter your in-house testing, there will always be bugs, but if Tesla can test software across 2 million miles a day, and in varied conditions, without any risks, I'd imagine a faster, less error-prone pipeline, leaps and bounds over the other teams.

As Sterling Anderson mentions at the beginning of his talk, all updates are opt-in. There isn't anything "Secret" about that part. Video: http://events.technologyreview.com/emtech/digital/16/video/w...

It's secret because the car doesn't tell you what's in an update until after the update is complete, and even then the release notes generally don't get updated unless there are new user-facing features or UI changes.

Yep, whenever someone say that GM/Ford/VW is going to destroy Tesla with their better mfg experience this is one of the things I drive home as Tesla's real advantage(along with the supercharger network).

Seeing the changes in the ~1 year we've owned a Model S has been incredible. For each new feature they add/improve they increase the value across the whole fleet(assuming AP hardware/etc).

Heck, they've said they already have a huge mapping network, you could start giving some of the major mapping companies a run for their money just with that data alone.

The networked transportation sensor suite is insanely valuable and it seems to completely be overlooked by both competitors and investors. Between this, the energy storage business, and obviously the vehicles, I am excited to see what they can do.

Doesn't matter if you can't get cars on the road.

Thank you for the substantive summary. These click-bait headlines make me want to tune out entirely.

Yes, that's great. I do hope that Tesla nor any other car maker ever pushes a silent OTA update while the car is in motion, though. In fact, there should be a hard rule embedded into the code that prevents any such in-motion update from happening.

As Anderson mentions in the talk, these features are rolled out in regular opt-in updates. Updates are not happening behind the scenes at some random time, although the article seems to imply that might be the case.

I wonder how it works though. Who checks the logs to make sure what it would've done is the correct thing. And if it's done by software. Why isn't that software "driving" the car?

Quite interesting, I've always wondered about pre-migration through on the field duplicate-logging. Any other company doing this ?

Sure, this is a pretty typical way to field-test algorithms that you need more data about but are worried about handing over control to.

Google does it, for example.

Sorry I wasn't precise enough, I meant in critical businesses (medical appliances, procedures, transportation, ...).

Why does it have to run on the customers vehicle? Why not just let customers opt in to sending telemetry, and test with that data. It seems like a pointless risk to run the tests live.

I imagine it's a pragmatic consideration, if they're installing it on that many vehicles- that's a shitload of telemetry to stream over whatever data connection these cars have, and then have tesla devote however much computer power to post-processing;

vs. current approach of distribute the processing among all those scads of nodes, and have them only upload the interesting parts.

Because the amount of data on the CAN Bus is many orders of magnitude larger than that which is sent as telemetry. Presumably the new features need to run on the raw data stream. I fail to see the pointless risk of logging what the computer would have done.

They still should let you opt out though. Once I drive my car off the lot I want to control what software goes into it.

Sterling Anderson (Tesla Director of Autopilot Programs) was a researcher at MIT for a long time before joining Tesla. I'm surprised the article (MIT Technology Review) doesn't mention this. I'm also surprised they didn't link to his video presentation, since they are the ones that put on the event. Here it is:

http://events.technologyreview.com/emtech/digital/16/video/w...

> he showed a chart illustrating how self-driving Teslas using the Autopilot feature hold themselves much more tightly to the center of the lane than humans do when steering the car.

There are a couple slides form Anderson's talk in this other article, including the one that this article mentions:

http://electrek.co/2016/05/24/tesla-autopilot-miles-data/

This is a huge competitive advantage for Tesla. Google is running a few test cars around mountain view, while Tesla is gathering training data from 50k cars already on the road.

Yeah, I mentioned this a month ago. Google only has 1.5 million miles of testing. Tesla will soon have a couple hundred thousand cars on the road. They should be able to get 10x the testing.

https://news.ycombinator.com/item?id=11560807

Actually, it looks like they might be getting 100x-1000x the amount of useful data:

"Every 10 hours Tesla gets another million miles worth of data, he said."

How useful is this data really though? Say their software makes the decision to go straight and maintain the speed limit. I, as the driving choose to speed up. Was their software right or wrong? Did I speed up because I was avoiding some hazard or because I'm driving a Tesla? Is this an important data point, or some fluke that should be thrown out.

I think one has to be very careful when inferring any conclusions from these logs.

I've seen the Google self-driving cars here in Austin for at least a few years. edit Not that it makes much of a difference to your main point.

I would hope its not truly being done without notification and agreement of the customer. That is just rude and should be illegal.

The car should tell me each and every time the manufacture did something and what it was. I should have to actively agree to be part of any test regardless if my direct involvement is required or not.

Many modern cars have an "event data recorder" that holds information about pretty much everything when you're driving. Admittedly they only hold about 30 seconds worth of information (in case you crash) but the point here is that tracking what you do isn't a new thing and expanding the period the data is held for would be trivial.

http://www.npr.org/sections/alltechconsidered/2013/03/20/174...

I'm on the fence, but think I mostly agree with you. I expect some new regulations to be created due to Tesla playing fast and loose with this tech.

I suggest not buying a Tesla if you'd prefer not to agree to the software update ToS that permits this (its part of the purchase agreement, and opt out is available).

Tesla provides constant updates to the vehicle's functionality (which no other manufacturer offers). In return, they perform machine learning using acquired data (at no cost to you).

Fair trade. You are reading the ToS of everything you agree to, no?

Yup, they can also do remote CAN diagnostics which can save you a trip to the shop to debug(plus they have a log of exactly what happened).

There's downsides for sure, but they are definitely leveraging all of the advantages as well.

Plus they can scold you if they catch you tinkering with your car.

On the plus side: they're watching you. On the negative side: they're watching you.

You own the hardware, they own the software.

This article takes a lot of liberty in promoting the idea that "secret updates" are actually a thing. Sterling Anderson simply says “We will often install an ‘inert’ feature on all our vehicles worldwide,” And from that, the author makes it out as some nefarious endeavor.

If anyone at Tesla is reading this, can we see some information of what the car does when it senses that there is a motorcycle lane splitting. Most drivers will adjust to the far side of their lane, what do the autobots do? (unless someone has a Tesla and can tell me)

As a motorcyclist, if the Tesla just maintains its course that would be perfect. And when they get to switching lanes, if they signal before they move, that would be perfect.

I don't need cars to move over (what if there's a motorcycle on the other side?). I just need stable, predictable behavior.

If you are a driving a car and you move over when I come, I don't mind. Because that tells me you probably see me and makes you a little more predictable.

> Because that tells me you probably see me

Thats the main reason I do it, as well as to be polite and leave you a bit more space.

I commute regularly on the US101 from San Francisco to Mountain View. I second brianpan that the car doesn't need to move over, and predictable stable behavior is more valuable.

However, I do notice that Teslas drive more in the middle of the lane. I have no idea if that's automatic driving, or just a coincidence.

Motorcyclists generally try to lane-split between the two lanes furthest to the left - because cars on the leftmost lane tend to stay closer to the left side of the lane (because there are no cars there), which creates a larger space.

So if anything, it'd be nice if Tesla automatically stayed a little on the outside of the lane when it has either of the end lanes and there is a good shoulder.

Yeah, that was the behavior i was talking about, primarily for cars in the carpool lanes.

"...secretly installing them onto customer vehicles..."

Do Tesla owners explicitly consent to this? As in terms and condition of sale state that they give Tesla permission to update the software on their car without informing them? If so, is that condition transferred to subsequent owners if the car is resold?

Where I am from, secretly installing software on a customer's hardware is really bad form, and would get you sued if discovered.

I believe owners sign a document at time of purchase permitting a wide variety of remote access and logging, but they can opt out if they wish.

Whether the owners are reading the privacy policy before agreeing to it is a different issue.

Source: https://www.teslamotors.com/about/legal

Not that this addresses the whole issue but if it is a lease the dealer technically still owns it.

I believe that these 'inert' features are installed with regular software updates and simply not activated while they are tested.

That's what I was wondering. This article goes on and on about the data and technology, but doesn't address the blaring privacy issues.

> “Since introducing this hardware 18 months ago we’ve accrued 780 million miles,” said Anderson. “We can use all of that data on our servers to look for how people are using our cars and how we can improve things.” Every 10 hours Tesla gets another million miles worth of data, he said.

For those following the space, this is an amusing quote, because a RAND study recently came out pointing that given known human accident rates, to have a statistically significant improvement in safety would require data from - gasp - hundreds of millions of miles of self-driving car travel, and with an expensive testing fleet of as many as 100 cars, why, that would take decades!

Not that the calculations were wrong, but I think everyone kinda laughed at it, and this quote emphasizes why.

Anyway, this is the sort of method you would expect. Everyone who can records car telemetry and does training/evaluation on simulations using that data. The only surprising thing here, I guess, is that Tesla is doing evaluation live rather than recording everything and pushing to Tesla servers and running the simulations there.

It's not always straightforward to throw more data at an algorithm. Having so many miles might not benefit them so much.

Google is doing a lot of special testing in simulators - those corner cases that happen once in a million miles, that you'd have to drive quite a lot to collect hundreds of examples of. Also, Google is solving the much harder problem of full city and highway driving.

The point is that a whole fleet can rack up daunting numbers in a very short time.

This suggests Tesla stores a recording of every driver's traffic violations. Have they received any subpoenas for that data?

There is no suggestion that the human's driving behaviour is recorded.

How the telemetry works is an unknown. We might speculate about it, but I would not be surprised to see that Tesla has taken measures to prevent their tech being used to punish their drivers for driving.

Testing "inert" features like this is not necessarily a bad idea. The worst that could happen is it isn't isolated correctly -- maybe you introduce a bug that transgresses its sandbox. Certainly Tesla's software architects spend a LOT of time working to ensure this doesn't happen, but it is a risk.

But there is a recent case where Tesla pushed a software update that resulted in an accident, and it wasn't an inert feature. It was a user interface change.

In 7.1 they added a shortcut for the Autopark feature that worked like this:[1]

1. Double tap the park button

2. Exit the car

3. The car starts moving forward

So... can you guess what could go wrong with this???

Yup, a user accidentally double tapped instead of single tapped and exited the car. The only safeguard against this was an alert on the center console.. but it wasn't a confirmation dialog, it was merely an option to cancel. Terrible UX. The user didn't notice it on the way out the door. And no other safeguards like having to hold onto the key fob to ensure you're paying attention.

So the car moved forward and hit something in front of it that its sensors didn't detect.[2]

What's even worse, Tesla blamed the user.[3] Not the kind of attitude you want in a company pushing updates like this.

[1] https://www.youtube.com/watch?v=t-JoZL9edlA

[2] http://www.theverge.com/2016/5/11/11656496/tesla-model-s-aut...

[3] http://www.theverge.com/2016/5/11/11658226/tesla-model-s-sum...

With cyber warfare becoming a real thing, imagine a third party hacking into Tesla's update mechanisms and pushing a silent update to overheat the engine or crash into as many casualties as possible.

Imagine if a fiberglass-bodied junker from the 80s overheated it's engine...

Actually, now that I think about it, that probably happens every other week on the local stretch of the interstate.

When did adaptive cruise control and lane keeping become "self-driving"?

Reply on news.ycombinator.com