Tesla's Autopilot Chief Keller Steps Down After Two Years
bloomberg.com
bloomberg.com
As far as the tech goes, I am in agreement with Elon. I think LiDAR is a short term advantage, but the companies/stacks depending on LiDAR will reach a local maximum from which they won't recover without totally starting again from scratch. LiDAR is short range, low resolution, and extremely expensive. There is no reason that visible light (+ IR) CMOS cameras can't do better by an order of magnitude, at a lower cost.
That's the only thing I know about it, and my source is a review of a C7 Corvette ZR1 (which Chevy cannot sell in the EU to my knowledge - due to the fact that the engine forms part of the hood)
”Our vision is that by 2020 no one should be killed or seriously injured in a new Volvo car.”
That's what other companies with driving assistance features do. The system tries to keep you safe, but it will complain rather loudly if you release the steering wheel for longer than a few seconds, it's neither intended nor sold as a self-driving system.
Collect data and see where the two systems (one human, one automated) differ and where they react the same, and from there extrapolate how well the system works (vs a baseline of human drivers under all conditions)
Do this for 10 years and you'll have a pretty good idea of how to improve autopilot.
I'm surprised such a study hasn't been done inside telsa itself.
The space shuttle if I remember correctly, does something similar to this with the redundant processors - all computers does all calculations simultaneously, and if there are discrepancies then either use consensus (2 out of 3) or sound the emergency alarm.
Also, I don't think a "restart from scratch" is required when moving away from Lidar - I don't have any hard evidence to make that claim and am happy to change my mind, but I haven't found any argument for doing so yet.
I’m not convinced that the camera-based approach is just a learning problem. As far as I know, cameras still have awful dynamic range compared to humans. Remember the Uber crash video where it looked like the victim was only visible for a couple seconds before impact? Cameras need to work a lot better to be suitable for autonomous driving.
So, in a direct comparison where cameras are allowed to use a similar set of tricks tricks including a fair amount of post processing they actually win. Aka, Retna vs CCD the CCD wins, iris vs iris camera wins etc.
As for dynamic range: HDR and the like can easily take care of that. The proliferation of phone cameras has thankfully taken care of this. I don’t think cameras are even remotely a bottleneck here.
My guess is that LIDAR is superior in that scenario, but I've never tried a Sony A7M3 either.
I always had the feeling they softened the image. Or at the least it was camera with low quality/settings giving a false impression of what the view was. If you pause the video right before impact the detail is still much poorer than you would expect.
https://www.reddit.com/r/SelfDrivingCars/comments/869olw/hdr...
It's more likely that the sensors in the Uber setup are set for fast processing, i.e., less light per frame, rather than for visual fidelity, i.e., more light per frame. After all, faster processing is more important to a vehicle traveling 60 mph than sharp borders. If the shutter time is set low enough, you would get something similar to what Uber released even on a DSLR.
Also, as others have pointed out the Uber video was from a dashcam. The point of the dashcam was to record events in the event of an accident, not be a high-fidelity visual record.
A single image is always going to be a poor way of judging the light levels, because the exposure (shutter speed, aperture, ISO) can make it look drastically different.
It is extremely cheap compared to a chauffeur.
Can you quantify this statement? All of the statistics suggest that it is in the same ballpark of safety as fully manual driving.
> I think it is cognitively harder to use autopilot and pay attention, than to simply drive safely.
Everyone is entitled to their opinion, but don't you think we should look at what the actual data says about autopilot safety (including for the various flavours of autopilot tech)?
It was expensive and low resolution. It is already cheap and very soon going to be - super cheap.
Nobody debugs without working with a limited group of people let alone something that kills people.
The Waymo approach seems to be the much more responsible.
SDC - Self Driving Cars
Why would you need to start from scratch? AFAIK all the sensor output, whether results from vision or LIDAR get thrown into a mix together with other sensors anyway, and the aggregated results from potentially conflicting sensor readings is then used in the later stages of the process.
Thus, it doesn't sound like an impossible task to switch out that part of the system. There seem to be a lot of startups out there working on commoditizing each of the sensor families too, so even if a company makes a wrong choice now, they could catch up pretty quickly.
Waymo may have less data, but it's data appears to be far more useful.
Worst of all, is putting in danger other people's lives who happen to share the same road with this type of Tesla drivers.
Instead of just speculating, the NHTSA did an actual study on this, and they reached the opposite conclusion. They investigated crash rates in Tesla vehicles that had the original autopilot hardware installed, both before and after autosteer was enabled via a software update, and found that autosteer reduced accident rates by 40%.
When Tesla talks about how much lower their accident rate per mile is compared to other cars, that's kind of statistical bullshit, since there are plenty of other uncontrolled variables that are in play that could affect this besides the cars themselves. But in the case of the NHTSA study, they were looking at the same drivers in the same cars, and the only difference was that one day a software update came out and enabled autosteer, and suddenly accident rates went way down. That's pretty damn convincing if you ask me.
https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.pdf
But exactly how NHTSA reached that conclusion has been a mystery. It’s also the subject of an ongoing lawsuit between the agency and a group of researchers aiming to obtain NHTSA’s data so it can try to replicate the findings.
That may be an impossible task.
On Friday, in response to a Freedom of Information Act request filed in March by Jalopnik, NHTSA said it was unable to provide most of the materials and data used in the investigation, citing a “confidentiality determination” it granted Tesla in July. (It’s not uncommon for companies to submit requests to NHTSA for certain information to be treated as confidential. From there, NHTSA then chooses whether or not to grant the request.)
In particular, the agency said, NHTSA is withholding in full an “excel spreadsheet Tesla submitted that contains production data from disclosure because the spreadsheet contains information related to trade secrets and commercial or financial information.”
[...]
Autosteer, however, is relatively unique to Tesla. That’s what makes singling out Autosteer as the source of a 40 percent drop so curious. Forward collision warning and automatic emergency braking were introduced just months before the introduction of Autosteer in October 2015. A previous IIHS study shows that both the collision warning and auto emergency braking can deliver a similar reduction in crashes.
https://jalopnik.com/feds-cant-say-why-they-claim-teslas-aut...
EDIT: So let me add that yes, this report probably isn't as pro-autopilot as some people are claiming, but I would still consider it positive for Tesla considering it is the only publicly available report on the subject and it shows that autopilot is at least no worse than a human driver.
I disagree that your interpretation is the most conservative interpretation. The most conservative interpretation is that we know basically nothing about the safety of Tesla's autopilot. There are so many unaccounted variables here - the presence of AEB and collision detection, total utilization of TACC vs TACC + Autosteer, utilization of Autosteer over time (for example: are people conservative with it at first and then gradually increase their usage into riskier driving situations?), non-airbag accidents, the constantly updating nature of Autopilot systems (for example what if an update actually makes the vehicle less safe?), whether people have more accidents initially driving any EV due to being unfamiliar with the instant torque of EVs and so on and so on. Mix in proprietary data which no one is willing to share and I think the only thing we can say is that we don't know.
Also:
>it shows that autopilot is at least no worse than a human driver.
It does not show that!
There are human driven miles in both buckets, for all we know the second bucket could have more human driven miles than the first bucket which is why the accident rate is lower. Tesla has never proven that it's Autopilot system is safer than or as safe as a human driver.
That doesn't follow logically at all. Collision avoidance and autosteer were introduced within months of one another. One could be reducing accidents while the other increases them. The most conservative interpretation is that the study may not have properly isolated these variables and is misattributing the safety gain of collision avoidance to autosteering.
> The only way for autopilot to be less safe than a human (taking the numbers from that study as true) is for those safety features to be substantially more effective on a Tesla.
You lumped the two types of features together. There's your logical flaw. The most conservative interpretation is that, yes, Tesla collision avoidance could be great while autosteering is "not nearly as effective as they claim."
For example the most recent fatality was directly caused by an autosteering error directing the car towards a median. However the accident could have been prevented if the automatic emergency breaking triggered before the collision. Considering it didn't, I have a hard time imagining that the automatic breaking has surpassed the rest of the industry to a large enough extent that could make up for the autosteering being as dangerous as you are suggesting.
It is certainly possible, it just seems unlikely that Tesla threads the needle to be extremely competent at one feature while also being incompetent enough to bungle a very related feature. That scenario relies on a lot more assumptions than the scenario I proposed which is why I described my original scenario as being the most conservative.
> I have a hard time imagining that the automatic breaking has surpassed the rest of the industry to a large enough extent that could make up for the autosteering being as dangerous as you are suggesting.
This is another comprehension error. You claim was "autosteering at worst had no effect on the accident rate." Thus any increase in the accident rate caused by autosteering, not merely a "large" one, would invalidate your claim.
If one has "moments" of inattention significant enough to drift out of a lane or "drive into another vehicle", one should not be driving.
There is a current (Toyota ?) commercial where a young driver with her father is saved from mowing down a crosswalk full of pedestrians by some form of "auto stop" and the father proudly declares "it stopped for you". If you need that to avoid killing people, you should not be driving.
The driver has no reliable way of estimating x or y and therefore can’t judge the risk of their lack of attention causing a crash. Maybe they “shouldn’t” be driving according to this formula, but they don’t know that, and autosteer could save them from an accident.
Most likely, the presence of autopilot increases x and decreases y. We can’t tell which affect dominates by arguing on the internet. We need data.
Full automatic steer isn’t needed for this. Many many cars, for at least a decade, have had lane keep assist.
Now active lane keep assist is even more common, which steers you back into your lane, but doesn’t actively steer until you drift, preventing the main abuses of the feature.
From anecdotal experience I would explain it as normal driving requires 80% of your mental attention, regular cruise control requires 60%, adaptive cruise control requires 40%, and Tesla's autopilot requires 20%. If you use it in that manner it is almost certainly safer than manual driving. The problem is that people get complacent and spend large stretches of time without dedicating enough attention to the task. This is supported by the fact that neither of the drivers involved in the two confirmed autopilot fatal accidents made any evasive actions. Yes, autopilot is partially to blame for that, but so are the drivers who weren't spending enough mental capacity on driving. That isn't a problem limited to autopilot. It is the same problem when people manually driving get distracted by their smartphone.
You might be surprised to learn that autoland has been around since the 60s... the UK were early pioneers due to notoriously poor weather in the British Isles, autoland systems allowed them a highly safe all-weather landing capability in zero-visibility conditions. The US was much slower and our aircraft really only caught up to some of the UK aircraft's capabilities in the 80s and 90s, and tended to rely much more heavily on ground guidance.
Certainly it's not a perfect mode of operations, but entirely human-driven cars run into things through inattention, too. Is it worse? Better? I don't think we know for certain yet, other than it's not an obvious major problem. There's a lot of Autopilot use out there and only a handful of wrecks.
That's like complaining about cruise control:
I think the brunt of the "cruise control" problem is not the tech, but the marketing department (which has to include American Motors and Plymoth of course). This "kind of autonomous driving" is absolutely deadly, by definition. It's never going to work well. "The car will accelerate by itself, until it can't. So don't pay any attention, until you need to save your own life. We won't give you any warning".
Just out of curiosity, have you driven long stretches using Autopilot? I just did a road trip and on Tuesday during which I drove 11 hours, most of which using Autopilot. I emphatically disagree with you. I'm able to stay more aware of traffic and situations around me because I'm not having to constantly do the tedious tasks like staying in my lane and monitoring my speed (they aren't hard tasks, but take some amount of effort, and are mentally fatiguing after several hours).
It's rather important to note that Tesla does not even promise that. It's clearly the impression that a great many people have including technically minded people here on HN. But that impression can kill you.
I'm all in favor of the long term goal here, but just focusing on the short term, what exactly is the user benefit of autopilot if you have to promise to stay 100% as focused and attentive as if you didn't have autopilot? As far as I understand, automatic collision avoidance is on by default at all times. And we've had cruise control for ages to physically rest legs and feet. So what's the point of autosteering?
I believe Tesla cites data that accident rates are lower with autopilot on. So, if they are being straightforward, that would imply it avoids collisions even more than regular always-on collision avoidance. (This is controversial as others have noted.) But... even if Tesla's implication is correct, logically this would only be true if the driver wasn't paying 100% attention to the road. Which is a violation of autopilot TOS. Therefore, it is plausible that most every accident avoided by autopilot is at best a case of misusing autopilot.
And that's a stretch. As others have noted, quite possibly it is the collision avoidance features that are responsible for the reduction in harm, not autosteering.
If instead the main point of autopilot is to train the fleet for our eventual Level 4-5 autonomous future... that seems like a dangerous way to go about it. Cheap, even. It leverages imperfect users to gather training data instead of employing trained professionals to do it in company vehicles.
I disagree. A visible-light based system like Tesla proposes is, quite frankly, a joke. Camera-based systems are too easily confused and require too much processing power to work in high-speed situations. (For example, at least one of the Tesla deaths was the result of the camera system failing to identify that a stopped object wasn't part of the road. A radar-based system would have easily recognized the obstacle.) Camera systems are also too dependent on maintaining the cleanliness of the sensors, and their performance degrades based on lighting conditions. While they certainly will be part of future AV systems (i.e., for street signs), they won't be the primary sensor.
Some combination of radar will form the basis of AV systems going forward. LiDAR is the most likely foundation, given its accuracy and speed, especially as LiDAR is expected to drop in price by nearly 99% over the next 5 years. (Recent developments in the underlying technology, plus economies of scale, have reduced the price of a full LIDAR suite from nearly $100,000/vehicle to just over $1000/vehicle. Technologies in the pipeline that are close to commercial production should reduce that cost to $100/vehicle or less.)
https://www.tesla.com/blog/upgrading-autopilot-seeing-world-...
Basic radar can provide positioning and ranging, but this requires additional processing to coordinate the input from multiple radar sensors to triangulate the positions of reflections. LIDAR gives you ranging with a single sensor, and positioning if you rotate the sensor.
That's why I don't like this guy.
that's nonsense
>>and extremely expensive.
and that'll change soon
(which has to include Elon Musk of course)
As these accidents demonstrate drivers can’t be relied upon to ‘monitor’ the autopilot and react if something goes wrong. It’s a deeply flawed design concept from an otherwise very innovative company.
Here's a quick improvement on testing for the learning agent: watch the driver while he is driving, watch the road, AI predicts what do in the same situation, diff that with the driver; diff it offline if necessary. Bang! massive improvement in learning data, time and pinpointing blunders. If they are not already doing this, they should.
A lot of people want to treat Tesla like they're just a tech company, while ignoring that the "A/B test" conducted here could literally get people killed (and has).
For example, in the most recent death some owners are claiming that an over-the-air software update caused the issue (lane centering), it made their cars drive differently. In tech regular updates are common, but normally the biggest consequence for getting one wrong is angry customers, not dead customers.
And is it fair to put the blame solely on a cherrypicked action by Tesla engineering? A tragic coincidence is a compilation of events.
Conducting A/B tests with live human subjects and in a context where failures will kill people (including those not even associated with the experiment!) is the level of scientific rigor I expect from Josef Mengele.
If true, having that data would give Tesla an enormous amount of data on situations in which Autopilot might have failed if it was in control of the car, such as if Autopilot (when not engaged) didn't see an obstruction but the human driver swerved suddenly.
This is also an advantage of Tesla installing full Autopilot hardware on every car regardless of whether the Autopilot upgrade has been purchased.
Plus who does not work with a limited group of users while you are testing. That is true for tech that does not kill you and more so when it can.
What Tesla is doing is honestly close to criminal. Waymo approach makes far more sense.
Google has indicated this for years. There really should NOT be safety drivers behind the wheel.
What Tesla is doing is almost criminal. Having a autopilot button when the car really can not do auto pilot is about as wrong as it gets.
I do not believe you can do SDC directly to the consumer while it is really in beta.
Waiting until the technology is perfect will take a long time.
We don't know how many other accidents have been prevented by autopilot. A safety system should not be judged by whether it's perfect, but by whether having it is better than not having it.
The NHTSA did a study on exactly this issue by looking at accident rates in cars with autopilot hardware before and after autopilot was actually enabled via a software update. They found that accident rates went down by about 40% after autosteer was enabled.
This seems to suggest that either people are fine with monitoring and taking over when needed, or else autosteer was already so good in 2016 that it made fewer mistakes than humans.
https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.pdf
No no no no no! The NHTSA report says nothing about whether Tesla's system is safer than humans. In that study there are two buckets, one which is total Tesla miles in TACC-enabled cars and then after the update total Tesla miles in cars with TACC + Autosteer and they calculated on airbag deployments. Human driven miles are going to dominate both of those buckets and there's a reason the NHTSA report makes zero claims about Tesla's safety relative to human drivers. It's totally outside the scope of the study. Then add in that some researchers who are skeptical of the methodology and have been asking for the raw data from NHTSA/Tesla have yet to receive it.
[1] It can do this, see: https://electrek.co/2017/06/07/tesla-software-update-airbag-...
Wow, I really hope that doesn't worsen the current imbalance of the x86 and GPU markets. Given that Keller seemed to be to Ryzen at AMD what Raja was to Vega/Polaris, it doesn't look good.
[0] https://www.anandtech.com/show/6907/the-king-is-back-raja-ko...
It'll be interesting to see Keller work in that environment.
Currently MobilEye seems to be the only vendor who doesn’t oversells their capabilities like crazy and has a solid evolutionary roadmap on all 3 landscapes - technology, regulation, and business model.
This one about their safety model is also quite interesting: https://www.youtube.com/watch?v=L92hR5CSw1E
Tesla was than later coming into the game as customer of MobilEye. The difference is, that Elon enabled much and sold it differently than the very same technology was used and still is used at BMW and Volvo. Then with first big accident (the death in Florida) Elon wanted to accuse MobilEye. But actually it was Elon who sold that technology in away it was still not ready for. MobilEye was very aware of that. But taking all the blame was the reason to split up.
Btw. BMW and Volvo both have radar on board, which the first Tesla lacked of, because Elon doesn't wanted to spent that money.
I did find it odd when he took a hybrid hardware/software role at Tesla. Probably was mainly working on clockless low power neural net processors.
Considering what he did at AMD with meagre resources, I can't wait to see what impact he makes at Intel with an unlimited budget.
A bit of both I'd say. From what I've been following, Tesla's autopilot efforts have been a particularly rocky road, comparable with the Model 3 ramp-up troubles.
I do not know if that is what is happening here but might be.
IMO, you can NOT be kind of driving. SDC technology should NOT go directly to consumers at this point.
Should be done with a commercial offering as we work through testing.
SDC - Self Driving Cars