Tesla AI Day [video]
livestream.tesla.com
livestream.tesla.com
Seems like the biggest problem was each pixel from a camera does not tell you how far away it is, so even if you know the camera is X feet off the grounding pointing at Y degrees, you don't know if there's a wall in front of the camera or not. So you can easily reconstruct a 2D space from all the cameras just by knowing their positions, but you can't simply translate that back to 3D.
I could barely follow the solution to this problem, but it seems to approximate the distance somewhat okay... but it required yet another universe of features in the neural net pipeline just to achieve it.
The funniest part is... isn't this exactly what LIDAR is amazing at?? Or what about having each "camera" actually be two cameras that can achieve depth via parallax?
I really don't know why this isn't leveraged more (maybe it is and I am unaware?).
It makes the software problem harder but it certainly is possible.
Animals also have (non)-artificial general intelligence. That doesn't mean Tesla's making a reasonable design choice if they build something in a way that relies on having that kind of capability.
But betting your FSD on doing object reconition and physical intuition seems like a bad bet.
edit: Since this got such a negative reaction I'll amend it with more of an argument.
If an animal can do X with eyes, this in no way implies we can do X with binocular cameras, as it completely discounts
1) our eyes are preprocessors for our brain in a way that cameras are not. That both sensors capture light doesn't mean they are equivalent.
2) robot brains are not there, in any way, shape, or form.
edit: I don't understand how this is a controversial statement...
Do you have any scenarios in mind? Driving is not only a navigation challenge, but a social challenge as well. Have you ever tried to navigate a 4 way stop when multiple cars get there at the same time?
But depth perception isn't really involved there so those theory of mind issues don't really have any bearing on the need for lidar for self driving cars. Clearly creating world models of sufficient accuracy for driving scale navigation challenges is possible (but potentially pretty hard) with just vision data.
Do you know how animals do it?
And it would be great if it something that almost never broke as things that move a lot tend to. As far as I know static lidar are nowhere near as good.
Then if the difference in quality is not actually that great, doing the development effort once will scale to millions of cars the next few years. So we are literally talking many, many years of millions of lidars produced and integrated (and not replacing other sensors) to maybe have a slightly better outcome.
However that depends on your sensor fusion and how the lidar handles weather conditions and other complexities.
But I am sure its all ego.
Tesla has cars with Lidar on the road to check the models as well.
And who says anything about entitled, they have spent a huge amount of effort and therefore money into a solution that they think will be better.
Nobody with Lidar has delivered either so what is better is still an open question. Tesla has picked their strategy and working on it. Others do the same.
My point is simply, not that Tesla has the perfect solution, rather the idea that Tesla is operating based on 'ego' is idiotic. Elon Musk and his companies have repeatably shown that they are very willing, comically so, to throw away work and go back to the beginning if they have encountered a problem. E
lon is well known for not falling into the sunk cost fallacy. Consider the manufacturing of Model 3. Consider Starship being planned to be built from carbon fiber, and switching to stainless steel.
Do you really believe they know lidar would be better and they not changing because of what Musk said 4 years ago? That is literally the opposite of how his company have operated.
As a matter of fact, yes: self-driving Teslas have been crashing into emergency vehicles so often, they are being investigated[1]
The investigation you reference stems from previous versions of the softawre.
The broader context can't be ignored when discussing the new software, but it's reasonable to point to progress as demonstration that the chosen technology path is a viable one.
The sunk cost fallacy is stronger when it's not just R&D but thousands of people who paid 10k$ on the promise that their car had all the hardware required... Cars that shipped without a lidar.
And I believe waymo uses lidar and has, as far as I know, killed nobody. They also take a more cautious approach where bystanders are not turned into beta testers, of course :-)
Sorry, what’s your point? You need to prove that the autopilot software has killed more people than if the cars were driven by humans.
> And I believe waymo uses lidar and has, as far as I know, killed nobody.
With a footprint probably orders of magnitude smaller than Tesla. What do you want to bet is the difference in terms of miles driven by Waymo vs. miles driven by Tesla autopilot?
Keep in mind you're saying this about something that attaches to a car, which already has dozens or hundreds of parts that rotate at hundreds or thousands of RPM.
> Then if the difference in quality is not actually that great, doing the development effort once will scale to millions of cars the next few years
The adage "do things that don't scale" comes to mind. If your options are "have tens of thousands of autonomous vehicles" or "have none", I'm not sure how the second is better. And we know that hundreds of thousands or even millions of lidar units are being produced (cruise and waymo and co have multiple units per vehicle and collectively hundreds of thousands of vehicles).
In some cases it can simply not be avoided of course but you better make sure that you really need it if you are gone add it to millions of cars.
> cruise and waymo and co have multiple units per vehicle and collectively hundreds of thousands of vehicles
Waymo has vehicle count of a few 1000s as far as I know. I don't know about cruise but I don't think the have even 100k vehicles. Please show me a source.
And even if that were the case, the build those fleets of quite a long time and the cost per vehicle they currently have is nowhere near close to what would be possible for a car like the Model 3 or future cheaper models.
Margins in car industry are incredibly tight, car companies invest gigantic amount of capital to remove a few $ per car. These vehicles need to be able to be produced at a run rate of millions per year. If you are proposing to add something that cost 100-1000$ that is a absolute killer.
In addition its an additional part that can hold up your production. This current chip shortage is a perfect example where having a Lidar would just introduce a whole host of new chips that might be in a shortage.
I don't get this. You have to pay ~8000$ as an extra to get the FDA in a Tesla. Can't they charge the cost of the lidar in there? Now you have to pay 9000 dollars to get it.
Waymo and cruise especially, but even a number of others, have demonstrated better autonomy than Tesla. Musk has been claiming full self driving is 6 months away for 5 years now, and he hasn't gotten that much closer.
Have they? They have a different approach where they focus huge effort on individual Geo-fenced locations and they both are losing 100M of $ every year. Neither has given any timeline for general availability in all locations.
The race, as far as I am concerned is still open. It is not at all clear to me that Waymo is ahead at what actually matters.
> Musk has been claiming full self driving is 6 months away for 5 years now, and he hasn't gotten that much closer.
The claim they made no progress is objectively false.
Where they actually have driverless vehicles. That's the key difference. Waymo and cruise have demonstrated driverless vehicles. People get driverless taxi rides today from waymo, and I think cruise as well. Tesla doesn't. It's still sitting in driver assist land and isn't particularly better than other luxury vehicle driver assist.
> The claim they made no progress is objectively false.
Thankfully that's not what I said.
If their engineering is driven by ego (most of us think - it is) then - are they really engineers? We all know it’s coming from Top - Elon, in this case.
It was around the time he made the comment that anyone using lidar is doomed, which is much easier to dig up as it was a headline everywhere:
https://arstechnica.com/cars/2019/08/elon-musk-says-driverle...
Hoping someone else has the link to the discussion of humans using nothing but vision, my google-fu is lacking this evening.
That doesn't make much sense to me. Tesla also needs chips for computer vision.
But Elon says it is because they don't need it.
This is not really accurate. Our brains do the distance estimation, and they use all kinds of tricks and contextual clues to do it, not just parallax (~2.5 inch parallax for objects more than ~100ft away isn't all that helpful). And this is the whole problem with vision-only autonomous driving - ML capabilities are nowhere near the capabilities of a human brain.
[0] https://roxanneluo.github.io/Consistent-Video-Depth-Estimati...
So, first, this presentation we're talking about is literally about problems like that.
Second: your brain isn't nearly as good as you think it is, it's just constructing a coherent story to fool you into thinking it is. Try this on the highway sometime as a passenger: close your eyes and recite the distances to the vehicle in front of you and the one to either side. I bet you anything a Tesla is going to do that better.
Third: they pretty much cracked this already. They stopped shipping radar on US Model 3's and Y's in the spring, have shipped hundreds of thousands of them now, and there's not a hint of signal that something is off with distance measurements with the cars. My car doesn't get this perfectly (you can actually watch the animations on screen bounce around a bit as the estimates change) but I think it objectively does better than I do.
Distance/Lidar framing is old news, basically. Vision works fine. The worst bugs remaining with all the FSD Beta footage on Youtube are almost entirely pathing and planning issues. The car sees its environment just fine.
Telsa should pivot and innovate to make lidar cheaper.
But people in California probably don't get this.
Also interesting to see how focused they are still on pure vision input to construct the vector space instead of just using LIDAR, but I wonder how limited they are by existing models already being out there with set hardware.
Boston Dynamics has shown it’s viable. Cross their work with FSD neutral nets and see what happens.
Along these lines, I read about the ditching of the radar, and I understand the technical limitations with the phantom stops under overpasses. But could someone help me understand why Teslas don’t just have the run of the mill Camry collision braking as a secondary measure independently?
The flip side of this is false positives: you don’t want a car slamming on the brakes because of a plastic shopping bag or other soft piece of trash lying in the road. This is a hard problem, regardless of technology employed.
On the other hand, you need the car to distinguish a mattress lying in the road from road markings. This is an easy problem if you have depth sensors, which brings me to my next point:
> But could someone help me understand why Teslas don’t just have the run of the mill Camry collision braking as a secondary measure independently?
Because for whatever reason (hubris?) Tesla insists on eschewing LIDAR/RADAR for 100% camera based systems. Not being able to explicitly sense depth is really kneecapping their system’s abilities.
Is this really a problem with current vehicles? I've had ford, gm, and dodge/ram vehicles over the last ~8 years with forward collision sensors and automatic braking and never once had any of them slam on the brakes due to a plastic shopping bag or other piece of soft trash lying in the road. I honestly haven't even had them give warning from one of those items being in the road (or any item that wasn't a car to be honest).
It's a retrofit problem. The moment you admit that you need LIDAR/RADAR, every Tesla on the road loses value unless you're willing to retrofit them all.
Comma.ai has sidetracked this issue by basically using the built-in sensors on the vehicles, and there's rumors of a camera-only version for those without radar coming in the future. This would allow any car that has fly by wire with steering control to use it for "level 2 autonomy"
Comma/OpenPilot doesn't work with Lidar anywhere I'm aware of...
Driving is hard.
I totalled* a car trying to avoid what ended up being a black trash bag full of foam rubber on an on ramp at night.
* By insurance standards, it was perfectly drivable.
Edit: Wow, Ashok's section was amazing. It does seems as though their ability to simulate (on top of their fleet's data collection advantage) could allow them to rapidly accelerate toward true FSD. I really like that they capture real world failures and simulate many realizations of them.
This honestly sounds pretty similar to the current state of affairs.
Currently, there are 0 people that fully understand the human brain. If you're driving, and someone turns in front of you suddenly, you can ask "why did they do that", but neither you nor anyone else can truly understand the set of inputs and thoughts that went into that decision at the time. Even if the other driver says "I guess I didn't see you", they also don't understand their own brain. It's probably a post-hoc rationalization.
The argument you have applies just as well to letting humans drive as letting opaque neural networks drive... better in fact since we have a better understanding of neural nets than we do of brains.
I think this is a fundamental problem with driving cars, and it's exactly this reason that I would rather take trains and subways exclusively than drive. Even if the operator's brain has a weird malfunction on a train, we have well understood systems (tracks) that prevent them from turning into traffic.
Contrast that with a human: you know the subset of possible actions a human would likely take. Even when you see a car swerving and acting like the driver is drunk, you have a mental model of the set of actions to expect from such a car (you might add more distance between you and that driver for example).
[1]: Not relevant to the above comment, but I once encountered such a bridge. It was while driving in the dark, at about 1am and with no traffic, when the navigation in my, iirc, Android 2.1 phone, suggested I cross an unfinished bridge. The only warning sign ahead of the precipice were a couple of traffic cones. The road was familiar, though, and I drove very carefully, curious about the new bridge.
By external measurements and statistics. You measure an ML engine by performance. Literally no one anywhere knows how to debug these things by looking inside the model.
If it's a "fragile system" then it will be shown to be so in data. At the end of the data, either cars crash or they don't. It doesn't matter what kind neurons made the decisions.
I agree with you that those techniques won't eliminate errors, but I don't think that's the requirement for L5 - human drivers' errors cause lots of deaths today.
You don't know. You measure and you decide, based on numbers and science and moral reasoning about risk. You don't get absolutes from anything else, why are you demanding it from cars?
It's a perfectly fair question and the general expectation of things we have as a society is that things generally shouldn't fail catastrophically at random. In a plane, that's achieved to socially acceptable levels by an incredible amount of systems engineering, redundancy, aerodynamics, and ultimately well trained pilots & ATC. With food it's achieved by government oversight, standards, and the general principle that most things won't harm you. With ships, we simply have them avoid icebergs.
What isn't a good answer is just handwaving about measuring model performance. That's the smallest of the many prerequisites to safety here.
That's why I think Tesla's approach is wrong. It tries to replicate a human driver. The argument against lidar is "we don't need a lidar to drive, our eyes and brain are enough". Yes, enough to kill thousands of people. You can easily find pictures of involuntarily camouflaged obstacles that take way too long for us to notice, and as expected, vision based system fail just like us, but lidar have no problem.
It is hard for computers to be as good as humans when it comes to vision. That's something our brain is exceptionally good at, and a large part of it is dedicated to it. But here, for true self driving cars to be a reality, they need to be way better than we are. It means that we should throw every advantage machines can have at the problem, and extra sensors are part of it, don't be limited to visible light cameras. lidar, radar, sonar, IR, inter-car communication, satellite maps, use everything.
And BTW, focusing on seeing roads is nice, but don't forget that there is more to driving that seeing roads. People communicate with their cars, sometimes in subtle ways, current self driving tech doesn't, and that's a big reason why they feel so alien and other drivers hate them. Self driving cars should communicate too, and better than humans.
Actually, you're being asked to "Always watch the road in front of you and be prepared to take corrective action at all times. Failure to do so can result in serious injury or death.", until FSD gets released :)
Do you trust cab drivers to "be trained on every edge case under the sun"? Self driving only needs to beat humans, who have some serious flaws, not achieve perfection.
In particular, though, it needs to beat good human drivers, not average ones.
Suppose you had 1.4 trillion examples in the following test set (using a model with 175 billion parameters):
(1,2)->3
(2,3)->4
(3,4)->5
...
Do you think it is possible to overfit and score perfect on the test set, while failing to generalize?
If you're trying to fit to some more complex space where a and b are unknown and you're given 3 numbers in the sequence, then what you're trying to fit is `f(a, b) = a + 2(b - a)` (or 2b - a, however you want to represent it), which is a swell function, but if you only give data that can be equally represented by `f(a, b) = b + 1`, you're mis-training your model.
But you could once again do that with a model with a dozen parameters. In both cases, the issue isn't overfitting, but misrepresentative data.
"Overfitting" is memorizing the training data instead of generalizing. The example you're providing isn't overfitting, it's just generalizing to the wrong function. Overfitting would be if the validation set was, say, 30 random values that you got right, but didn't get other values along the same lines correct.
> I didn't specify the training set, just the test set
Then unless you constructed the training set with the intent of mistraining the model, I think a training set that got good accuracy on that validation set would generalize.
> The point is that it doesn't require trillions of parameters to overfit to a trillion-sized test set.
You can't "overfit" a validation set, unless you've done something wrong. Overfitting is, by definition, learning the training set too well such that you fail to generalize to a validation set.
No, that's just mis-modelling. Overfitting is specifically doing so in a way that learns the training data too well, at the cost of generalizing. If you try and have a single layer perception network classify a nonlinear function, it will fail to generalize. But it certainly isn't "overfitting".
Overfitting is not the only form of mistake when training a model. You've presented a different one, which is just like trying to train on misrepresentative data. But that isn't "overfitting", it's just having bad data. Your model isn't "failing to generalize", it has nothing to generalize over.
The classic demonstration of this is that overfitting usually results in a accuracy curve that "frowns" on validation data. Your accuracy peaks, but then decreases as you learn the structure of the test data instead of the general structure. In your example that won't happen.
Training a model in the wrong problem isn't overfitting. In fact, your example is more like underfitting than overfitting. The model in your example would fail to see the full complexity of the structure, instead of as in overfitting, make it more complicated than reality.
All this shows is that you don't need parameters anywhere close to the number of test examples to overfit.
Your example training set is not filled with noise that the network is picking up on to its detriment. Your example training set is simply not representative of the function you are trying to teach.
My exact point is that if your test set isn't representative of the underlying distribution, then accuracy on the test set doesn't mean that your model isn't overfit.
But you can say the same things about a human driver.
Say, on average, human drivers crash into barricades once out of every 500,000 miles driven. And say Tesla's FSD can beat that number by crashing into barricades once out of every 1,000,000 miles driven, it's still possible (probable?) that many people would trust their own control of the vehicle more than Tesla's FSD AI. And they might be justified in doing so!
Why? Because human-caused accidents are not uniformly distributed among the driving population. Insurance companies have employed actuarial analysts for decades to split the driving population up into buckets of greater and lesser risk. Thus, if you're a 17-year-old male, or an 85-year-old woman, or have several DUIs, you're likely to pay higher premiums than most other drivers.
If you're not in one of those high-risk categories though, it's possible your driving performance exceeds the average, and maybe you might only crash into barricades once out of every 2,000,000 miles driven. In that case, you'd be right to trust your skills ahead of the AI.
My example is contrived. It's possible that the FSD AI leapfrogs even the most skilled human drivers. But until it does, there may be rational reasons for many folks to continue distrusting it.
This is such a great point. Controlling for driving circumstances (e.g. weather, location, etc.), human caused accidents are not uniformly distributed across the population, but machine caused accidents are.
This, in my mind, is the fundamental reason people are distrustful of self-driving cars. Everyone thinks they’re far better than the median driver, even if this is mathematically impossible.
Only for one of the drivers! There are at least two cars involved in almost every fatal accident.
I made this point elsewhere but it bears repeating. Even if you think your driving is so perfect you can't benefit from a self-driving car, don't you want everyone else in one?
Pretty much. I think if you're going to make an argument like "accidents by teenagers, senior citizens and drunks don't matter" you need to put some numbers behind that.
I mean, if you had a teenager or a family member with a substance problem, would you feel safer with them in a Tesla? If so, then I really don't think I understand your point.
Basically you're just saying that you, personally, as a young single man, feel like you're much better at driving than the median driver. So you want a car that is significantly better than even the median before you will "feel safe".
Alternate framing: it doesn't matter how safe you think your driving is, you're still sharing the road with grandparents. And you (if you're being rational about it) want them in a Tesla.
Right. Rationally, we all want folks who are worse than the state-of-the-art FSD AI to be using FSD. But thinking that the "very low bar" that has to be cleared is drunk drivers misses the point that most drivers are not drunk. So it's much less impressive to a "safe" human driver to clear the low bar.
I said median, not drunk driver. Existing shipping autopilot is already saving DUI fatalities. This feels like a strawman to me...
Right, and I originally said:
> I think one of the reasons there's such an inherent distrust of Tesla's FSD is that when it fails, it fails in scenarios in which people can easily see themselves succeeding.
The above-the-median driver is rationally able to see themselves beating Tesla's FSD. But importantly, even below-the-median drivers are likely to think they can beat Tesla's FSD for as long as the mistakes it makes seem naive and trivially-avoided by them. Human trust will require a higher bar than "beats the median human on 5 KPIs, but still on rare occasions decapitates someone because it doesn't know what a semi truck looks like from the side".
FSD advocates seem to want to focus on the "beats the median human" bit. I'm saying it's unlikely they'll gain much public trust until the "can't recognize a sideways semi" stories die down.
Exclude the worst 1% of drivers and the average goes up considerably.
The consensus seems to be that it's legal in many places, however a dangerous lane change is always illegal and changing lanes in an intersection is almost always dangerous.
[0] https://www.reddit.com/r/NoStupidQuestions/comments/25kbg3/i...
I've of course heard of people on the ground floor of Tesla (factory workers) not being treated well, I've heard of SpaceX employees having high churn as they get "burnt through", and of Elon being kind of an ass.
But having said that I have a Tesla and am one of those types that like to work on things that bring me value or that I can have some sort of tangible connection to.
It was high pressure in that there were hard deadlines, and these deadlines needed to be met. If they weren't met, you needed to have a damn good explanation for as to why they weren't met.
Somewhat disorganized in that they did have a complete dev-ops lifecycle, but it felt kind of patched together. The team I was on did not do any formal standups or sprint planning. There was a very robust QA, validation, and homologation process though. They expected developers to do most of the QA, though. The point of the actual QA team was to ensure that there were 'no surprises'. I happened to be paired with a dedicated QA rep, though, which was nice.
Results-driven in that you were expected to do what you needed to do to get your job done. "Nobody told me what to do" was not an excuse. If you needed something done by someone else, you were expected to take initiative and ask and bug people until it got done. Most employees knew this, however, and were happy to help.
Engineers somewhat idolized Elon but also legitimately did not want to be the one to tell him bad news (such as they would prefer a different company rep do it). There are a lot of really smart people at Tesla, and you really do get to work on things that have never been done before (I got plenty of news articles written about what I worked on), but you really need to be dedicated to the company.
When I was there, there was a lot of crunch towards the end of my contract, and they said it wasn't usual, but I'd say expect crunch at least once or twice a year.
Do sensible people really take this seriously?
He's slowly building out the roadmap for the labor pool, transportation, communications system and so on for the colonization of Mars.
https://www.digitaltrends.com/cars/tesla-coast-to-coast-auto...
Hard to take them seriously given how much hype they generate each year.
To me, the question is: is the technology getting better over time? And from what I've seen I would answer that question with a resounding yes.
Then why is it sold as a $10,000 full self driving package?
Same thing come 2026: Tesla will almost surely have new hardware for that time, and I'm willing to bet that FSD still will be a vaporware product and incomplete. If it happened over the past 5 years, history tends to repeat and I can make the same bet will happen in 5 years into the future.
“ As a kid Elon Musk had so much trouble being on time, his younger brother developed a trick to make sure Musk didn’t miss the bus to school. Kimbal Musk — who currently sits on Tesla’s board — would lie to his older brother about the time, telling the future billionaire that the bus would be arriving, several minutes ahead of schedule. The slight manipulation ensured Musk wouldn’t be left behind.” https://www.washingtonpost.com/news/innovations/wp/2018/06/0...