Tesla Live Stream – Autonomy Day [video] | Hacker News Reader

Tesla Live Stream – Autonomy Day [video]

livestream.tesla.com

360 pointskiddz524 comments

Karpathy's presentation is really good. Watch it later if you get some time. The key points are

- Telsa is using fleet for learning (Fleet learning). This has 3 components, a. Trigger infrasture that collects the kind of data telsa is looking for training. The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly. b. Data Engine which learns from these examples. c. Shadow mode where they deploy the learned model and test how its doing in real world and iterate this process.

- Drivers are themselves acting as labelers and tesla is in a unique position to take advantage of this.

- People drive vision only. Visual recognition is essential for autonomy. Lidar has much less information than vision, for example to identify the thing on the road is a plastic bag, lidar provides few points. But vision gives more data for telling this.

Karpathy is great at presenting executive-101. However self-driving is much much more than image classification. You have a time component, you need to integrate information as it comes in to meaningful state (aka SLAM) and make sequential decisions that has potential to kill people. This is still unsolved research problem unlike punny ImageNet classification. Karpathy could have easily spent entire day talking about current state of the art and how they have improved on it but instead we got Deep Learning 101. I hope he didn't intentionally used this tactic to avoid talking about the elephant in the room. It seems they are hell bent on vision-based end-to-end learning which vast majority of experts would agree that is far away in the future. So this whole thing is quite bold and if they really make this happen in 1yr3mo then I'd say you should dump your entire 401K in buying Tesla stock. Collecting data is actually more trivial part of this problem.

Good overview, I share your opinion.

Tesla is betting BIG on NNs.

In the presentation they say that we have NNs in our brain. Which is why we can look at one photo of a dog, and distill enough identifying information in order to spot the breed from then on. Training sample of 1.

Curiously though, they than say that NNs don't work like our brains do, and require A LOT of training data from all angles, etc. Training sample of N.

The presentation doesn't actually show, how they plan to train a single NN to encompass all the knowledge for self driving. In particular, the looooooong tail. But maybe they'll have independent models, say based on "observed" conditions, which swap in and out ... based on more NN logic.

But at the end of the day, there is no known (to me) examples, of NNs even approaching human ability. Tesla is bascially hoping we will believe, they will be the first to show this capability of NNs.

Let's wait and see, 2020 is not that far away.

> require A LOT of training data from all angles

Also, we learn things outside of our cars. I ride a bike, this gives me insight into what bike riders will do on the road.

I have kids, so I know a kid on the pavement will be more risky / unpredictable than an adult.

I drove down a single track road today and met another car coming towards me. We stopped, then he started to reverse, there was a passing place behind him. Another car pulled up behind him, so I then reversed back into a field to let the cars past. This sort of thing happened maybe six times today as I drove around. Not at all surprising, not even noteworthy enough for any of the drivers to acknowledge even a flick of the finger in thanks. Just part of driving here.

Going to guess that you're new to this.

So deep learning via neural networks is pretty much the only way to implement self driving cars these days. It's what everyone is using and is used also in FaceID and other recognition systems.

So Tesla isn't betting big on it any different to all of the other self driving companies. We wouldn't say Facebook is betting big on PHP just because they use that technology. And Tesla definitely isn't in the top tier compared to say Waymo.

Plus talking about neural networks approaching human ability is just marketing buzzwords. It doesn't mean anything tangible since we can't define our abilities in terms of models with hyperparameters.

This is still unsolved research problem

Luckily, Karpathy is pretty good at solving research problems.

Karpathy could have easily spent entire day talking about current state of the art and how they have improved on it

Why would he do that? To help Uber?

Collecting data is actually more trivial part of this problem.

For Tesla, yes. For all others that’s the hardest part.

When they say that collecting data is the trivial part, they mean that (in comparison to the other parts of the problem) it is easy for everyone to solve. In other words, they are saying Tesla has an advantage in area that doesn't matter.

> Luckily, Karpathy is pretty good at solving research problems.

Oh come on - there are a lot of people good at solving research problems, but hard problems remain unsolved. To pretend driving autonomously in the real world is anything but very very hard is naive.

Maybe when Tesla cars stop driving into medians at full speed...

>Karpathy's presentation is really good. Watch it later if you get some time.

I would second this recommendation. He did a really good job not only explaining Tesla's approach, but also breaking down the concepts in an easy to understand way for laypeople. That is a rare skill to have especially among someone who is obviously so technically accomplished.

Based on what I know about how high-level executive meetings and presentations (which is the role Karpathy is playing here) go, there's a pretty good chance the presentation was not actually developed by Karpathy but a Tesla team of significant size.

It was significant that the brief bits of video showed the system identifying "driveable" areas, shown in blue. That's important. The first step is to conservatively mark driveable areas, where you can physically drive. Then you get to plan where to go.

Amusingly, if you wanted to train a vision system to recognize driveable areas, the straightforward approach would be to use a LIDAR system to measure road flatness. Watching the road go by on video as a human driver drives is not very useful for this. That just tells you about the part of the road a human driver chose to use, not about the road geometry itself. The driver probably won't use the road shoulders, but the drivable area system needs to evaluate them.

Recognizing drivable areas is what keeps you from hitting solid objects. You don't have to classify them, just note that they're not flat road.

> The rational here is we don't need massive amounts of same kind of data but needs right kind of data for training neural networks correctly.

This is really important. At a certain point additional miles on similar roads in nice conditions with fair traffic won’t help anymore. The data variety is really important.

They also probably want to learn from the best human drivers and not repeat the mistakes of the average.

What I found interesting as well is that they (him and Elon) hated the idea of using high definition maps, saying that if you need them you don't do self-driving cars really. For example, when driving condition change (roadblock or anything) you're done.

If this is true, why do Waymo (or any of the top 10 self-driving companies besides Tesla) use Lidar?

Two facts:

1. Tesla can't use Lidar because of cost and aesthetics.

2. Elon has a strong incentive to oversell their approach to self-driving.

As far as I understood HD maps are static data obtained by many various ways (not only lidar)[1]. AFAIR they claimed that Lidar itself provides some value (teaching neural networks how to measure the distance to various objects etc) but the gist is that image recognition is an absolute must-have for self-driving cars (which seems right to me).

1 - https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-r...

From the presentation they mention that Lidar is good at developing tech demos quickly, but they've found it is insufficient in reality. Which of those 10 companies you reference have moved far beyond tech demos?

From what I can see, overselling is Elon’s standard strategy for whatever technical approach he’s decided is the right one. (And I can’t really say that doesn’t work for him.)

In most legal jurisdictions, lidar in consumer devices requires special permits from health and safety agencies. It's such a new tech in consumer devices that regulatory approval timelines can be highly unpredictable. And if there's one thing that Elon Musk loathes, it's unpredictable regulatory approval timelines. Plain old videocameras don't require any sort of special licensing in the vast majority of legal jurisdictions.

A better question is, how come any of the companies using Lidar haven't shipped anything as good as Autopilot given that it's so much better?

That is very interesting. High definition maps perhaps not, but I do hope maps of some kind are an integral part of the logic.

For example on a country road near my house, the road veers left while a side street continues perfectly straight. The current auto-pilot has a very hard time staying on the main road and almost always disengages before putting enough left-steer to stay in lane.

A human who didn’t know about the road (no knowledge of the left bend ahead), and who missed the big yellow “left curved arrow” sign, would likewise probably end up going straight onto the side street.

This is a hierarchy of stateful awareness which I believe is essential for successful self-driving — the only way to increase NN weighting to the point where the car will convincingly execute the left bend to stay on the main road is some awareness that a left bend is upcoming. This is a case where you can’t drive the road by staring at the lines. And there are many, many others.

Does anyone know how Lidar would work in the future when many other vehicles are also using lidar?

From working with some different types of photo sensors (CCD, avalanche photodiodes, photomultiplier tubes), you might imagine that incident light from another car's emitter would be much brighter than the reflected light the lidar sensors are reading and cause saturation.

I've only got experience with aerial survey lidar, but "blinding" the photomultiplier with your own transmit pulse is pretty common, e.g. hitting something retroreflective like a cats eye marker or a street sign. Even the sun off a puddle can do it too. It is a pretty obvious artifact to recognize and remove. I guess it would depend on the mechanism of the lidar (i.e. if it is scanning then what is the probability that the different cars are scanning at each other at the same time, probably low). Might be a problem if they use flash or geiger mode lidar. But then the pulse rate would be lower. Interesting to think about. I'm sure the people working on it would have an answer.

Phase lock or introducing jitter to laser timings. Check out Animats' comments on this topic, it's where I've learned from regarding this.

Thanks!

Probably the same way GPS satellites avoid jamming each other: modulate the lidar transmitter with a unique coded sequence that can be detected by correlation.

Does anyone doing lidar not also do visual recognition?

I don't think any one is doing this. This is the point Karpathy is making. Vision is essential and needs to be perfect and also has all the information to recreate the 3d environment. That's why lidar is not required and is expensive.

  lidar is not required and is expensive.

Given that the only the company claiming that is the one whose cars sometimes hit (or even swerve into) clearly visible stationary concrete barriers in broad daylight [1, 2, 3, 4], I consider that claim far from proven.

[1] 2019 https://arstechnica.com/cars/2019/03/dashcam-video-shows-tes... [2] 2018 https://news.ycombinator.com/item?id=16772748 [3] 2017 https://bgr.com/2017/03/02/tesla-crash-video-texas/ [4] 2016 https://www.thesun.co.uk/news/1402992/tesla-self-driving-car...

Vision is probablistic. You can improve your probabilities with more data and deeper nets, but the probability is never 100%. Inferring depth using triangulation of extracted features using vision is also probabilistic.

Lidar gives you very rudimentary depth and positional data, but it is precise, and reliable, and the false positives are low. Lidar is not seeing the world as a series of probabilistic phantasms that may or may not be a pedestrian, or a jersey barrier, or whatever.

Elon said: "Lidar is lame! Lidar is Lame! lame lame lame lame."

But then he later said: "We've gone over this multiple times. Are we sure we have the right sensor suite? Are we sure we have everything we need?.... No."

Lidar is a crutch and we'll see how hard Tesla is limping in a couple years when Elon is claiming They're going to have Robotaxis.

To Tesla's credit, it is theoretically possible to develop a fully autonomous vehicle without needing Lidar, but before you can deploy, you need to integrate Lidar, and it can be retrofitted as a largely independent system, necessary for it's enhanced redundancy, but not needed for informing the planner, except when the Lidar is seeing things the vision system isn't.

The presentation was a pretty good run down on various techniques and methods that are being utilized by different autonomy developers. The downside is that everything is framed as a rationalization as to why Tesla has some kind of secret weapon, when they're mostly doing conventional things, and not all the conventional things you need to build a complete autonomous vehicle.

I was waiting for Elon or Karpathy to trot out the magic pixie dust that will have Tesla's driving empty in a couple years, but that was nowhere to be found.

Waymo is almost certainly using a ton of computer vision in its cars. Vision is needed to classify objects on the road, predict the behaviors of pedestrians, bikers and other drivers, read road signs and traffic lights, etc.

As far as I can tell, LIDAR makes a lot of prediction tasks much easier (since you have both images and LIDAR point clouds), and is also a backup for obstacle detection to ensure that your self-driving car doesn't crash.

I agree with the essential bit, but “has to be perfect”? Do you know how bad human vision is at estimating distances, speeds, angles, etc? I don’t think you need perfect vision to implement the first law of autonomous driving (“Avoid hitting stuff”)

It also doesn’t have all the information to recreate the 3d environment. There’s all kinds of inferences that human drivers make that require a good model of the world, to start with. For example, a car shouldn’t brake for a small object that moves with the wind, even if it looks like it’s made of concrete, or looks like a child.

Vision is not perfect though. It is impaired during night time and difficult conditions and can be obscured if dust/dirt gets on the camera.

And not sure who you are but I trust all of the other companies who are doing this a lot more. And they are all using LIDAR.

Yeah I don't see how it is either/or. Autonomous driving is so hard surely you need all the sensors you can get! Maybe when it is working really well you can optimise the cost and only use cameras, but we are decades away from that.

I suppose if you want to use customers as guinea pigs you do have to have a low-cost solution first, which is why Telsa are advocating it. I'm sure if LIDAR were cheaper I'm sure they're be using it.

The livestream is gone though https://youtu.be/Ucp0TTmvqOE

Other channels have recorded it https://www.youtube.com/watch?v=tbgtGQIygZQ

>"People drive vision only."

Can you elaborate on what you mean here? Do you mean that Tesla will only be using Computer Vision for autonomy? Doesn't or wouldn't Lidar compliment CV? Or is there a practical design consideration that makes them mutual exclusive?

That people drive vision is so idiotic.

Humans have far higher resolution sensors and the most advanced computer ever seen behind them. And even that fails at a far higher rate than we want self driving cars to be.

> Humans have far higher resolution sensors

This is really not true. The phone in your pocket is just plain better. More total pixels per unit area, much greater dynamic range, able to sample over time periods an order of magnitude faster.

The reason your vision seems better is because your brain is amazingly good at synthesizing a picture of the world around you. But all that data (the sphere around you is something like 150 MP at eye resolution) is an illusion. You're only looking at a million pixel(-equivalents) at once, thereabouts.

[1] Your eyes come close only in the middle half a degree of your fovea. Everywhere else your brain gets a blurry mess and has to extrapolate.

Conversely computers don't get tired, or "distracted", or drunk. They're also faster - human cognition is running about 150-200ms behind what you actually see. Some reflexes are running faster then that, but overall the human platform is not being used with that super-computer in full-control while driving at high speed - our hardware isn't capable of it.

The thing we most bring to driving is navigating complex, low speed environment changes - but we're not great at that either (see the number of toddlers run over in their drive ways, for example).

The resolution outside of the fovea is pretty poor. It's a little unclear what portion of accidents we could cut out if we had human level performance, but never let the attention waver or disallowed reckless driving. Judging from NMVCCS it'd cut out somewhere between 34 - 75% of accidents.

https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...

>"And even that fails at a far higher rate than we want self driving cars to be."

Thanks, but did you mean to say that people fail at a far lower rate that we want self driving cars to have? Maybe I misunderstood your point?

How many technologies do you see people working on where they say, "well we have one approach we think is gonna work, but we're also going to spend a ton of resources and time developing this other solution that would be useless if the first approach does work. Oh and it would make our cars 5k more expensive atleast."

Tesla has take a firm stance against using LiDAR and is doubling down on CV. I'm not sure if they have radars for parking assistance, but they are using cameras as primary sensor for perception and localization.

It was great including his explanation of how Tesla drives in snow and rain which is hard to understand how a car can do it.

> People drive vision only.

No we don't. We have vision which we then interpret with our _minds_. We have the ability to _understand_ what we are seeing, reason from it, and make decisions accordingly. "Deep Learning", no matter how big your dataset, fundamentally cannot possess this capability. Which goes a long way into explaining why Tesla's "self driving" tech keeps on killing people in situations that are trivially simple for humans. There's no way to rigorously test for edge cases in opaque deep learning algorithms, nor is there any way to implement redundancy as there is in ordinary deterministic systems.

This is a mad attempt to replace human cognition with "AI".

> "Deep Learning", no matter how big your dataset, fundamentally cannot possess this capability.

Why not? I’d agree that current models can’t, but what’s the fundamental shortcoming of deep learning?

The fundamental shortcoming is that there can be no understanding, and therefore no judgement, because there is no mind. See Searle's [in]famous Chinese Room Argument (https://plato.stanford.edu/entries/chinese-room/). Or for a more concrete example from today: https://twitter.com/mit_csail/status/1121447605412741120?s=2...

I question how much of our driving is logic and analytics based as opposed to instinct and reaction based.

I'm sorry but I'm finding it really difficult to watch this and match up what the engineer is saying to what Elon Musk is saying.

For example, the engineer says the custom ASIC does 144 TOps for 2 chips vs the NVidia drive Xavier - does 21 TOps. Okay, well yeah I expect your custom ASIC does have a nice performance advantage over the equivalent GPU. at 3.5x advantage probably seems reasonable. Cue Elon Musk:

"At first seems improbable, how could it be that Tesla, who has never designed a chip before would design the best chip in the world but that is objectively what has occured. Not the best by a small margin, the best by a huge margin".

Mate, it's a dot product with some memory attached, and not a single detail your half hour deep dive has gone into suggests anything other than a bog standard ASIC.

"All the cars being produced right now have all the hardware necessary for full self-driving"

And this is where I'm totally lost. I want to believe! But he's lied so many times now. This man is sucking the credibility out of every engineer in the room. Don't repeat the same lie twice.

This doesn't seem wildly inconsistent with my understanding of Elon musk, in that he has a higher level of confidence of his understanding of the world than others would, and in a "you don't know what you don't know " way it seems he didn't know about custom how custom ASICs are much more efficient at specific tasks than a general purpose computer. Doesn't make him a bad person, and in fact something like that is exactly who you want leading a company trying to change the rules of the industry they're in, it's the job of the boots on the ground to exercise due diligence here.

I don't understand this response to the above comment. You write,

> it seems he didn't know about custom how custom ASICs are much more efficient at specific tasks than a general purpose computer

Yet, Musk is clearly stating his new chip is state-of-the-art. He is not underselling it.

> something like that is exactly who you want leading a company trying to change the rules of the industry they're in

You want someone in charge who does not understand the hardware he's touting?

> it's the job of the boots on the ground to exercise due diligence here

Who are you referring to? Employees and investors?

What, then, do you consider are the responsibilities of the CEO, if due diligence is not part of the job?

I think the part that Elon, and the engineer, are trying to express that you are looking past is "A dot product with some memory attached" is what they need to do low latency image inference on the camera data.

They don't need a multipurpose CPU/GPU, they have a single well defined task that dominates their compute needs. They built a chip to do this that is cost and power effective, and redundant for safety.

Right, but Nokia's 5G backhual infrastructure is far more capable of doing Mobile backhaul than a GPU, you don't get the head of Nokia standing up in a press conference telling everyone his team has designed the best chip in the world.

> For example, the engineer says the custom ASIC does 144 TOps for 2 chips vs the NVidia drive Xavier - does 21 TOps. Okay, well yeah I expect your custom ASIC does have a nice performance advantage over the equivalent GPU. at 3.5x advantage probably seems reasonable. Cue Elon Musk:

Here's my issue with that. The on-chip SRAM is only 32MB, and the RAM is LPDDR4 rated at only 68GB/s.

Assuming a dot-product (multiply + add) over INT8 data, that's a limitation of 2-operations per 68GB/s (that the RAM moves at). Or 136 GIOPS (Giga-integer8 operations per second). You're limited by RAM, based on what I've seen in the presentation.

Unless their neural net is 32MB and fits entirely in on-chip SRAM. That seems unlikely to me...

Nvidia already responded:

""Tesla was inaccurate in comparing its Full Self Driving computer at 144 TOPS of processing with Nvidia Drive Xavier at 21 TOPS," a spokesman said in an email. "The correct comparison would have been against Nvidia's full self-driving computer, Nvidia Drive AGX Pegasus, which delivers 320 TOPS for AI perception, localization and path planning." The statement also contends that "while Xavier delivers 30 TOPS of processing, Tesla erroneously stated that it delivers 21 TOPS. "

From what I understand, Pegasus consumes about 500Watts, compared to under 100 Watts for Tesla's FSD computer. Elon in particular emphasized the performance per watt (as it's always possible to cram more chips to increase performance if you ignore cost and power consumption).

The comparison made in the video: 500Watts for an hour consumes about 2-3 miles of range. In a city in slow traffic, going 12mph, that's a significant range reduction. So you might have a 10% improvement in range for the Tesla ASIC in low speed conditions.

Convolutions have a much higher arithmetic intensity than dot products.

But in terms of CNNs, it's basically dot product is it not?Can you give an example of a CNN where it isn't?

I think the key thing is that you assume a single dot-product per RAM load. In general you get higher compute performance by doing more than this. You load weights for a layer in the SRAM, then stream from RAM tiles of data that you can compute multiple operations on.

In the same way, to reach peak FLOPs on a GPU you better use the local/shared memory as much as possible.

>> how could it be that Tesla

> Mate, it's a dot product with some memory attached, and not a single detail your half hour deep dive has gone into suggests anything other than a bog standard ASIC.

There you go, you answered the question Musk put forward. Grats.

Apparently it’s actually less than half the performance of Nvidia solution: https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...

https://www.nvidia.com/en-gb/self-driving-cars/drive-platfor...

I'm a fan of Tesla, but not Musk. You really can't believe much that comes out of his mouth. If you look at his past claims it becomes clear that he should just be ignored.

Being the fan of a company while completely ignoring the ceo of said company is questionable to me.

Tesla has great products, created and supported by thousands of people who appear to go above and beyond. Kudos to them. And kudos to Musk for the vision and the early execution, but that doesn't mean I have to like him or believe that he is the right person for this phase of the companies growth.

> If you look at his past claims it becomes clear that he should just be ignored.

What's an example of this? His timelines are often much too aggressive for reality but he repeatedly delivers (eventually).

>he repeatedly delivers (eventually)

This is essentially unfalsifiable. In the case claims about FSD or a person on Mars are wrong then this statement can only be proven wrong when Elon Musk chooses to admit defeat - suffering a significant personal financial loss as a result - or when he retires.

The main example, of course, is in self-driving where he has most certainly not delivered. And he wasn't just a little off.

But also his claims about being able to automate the manufacture of Model 3 turned out to be very wrong and to do terrific damage to the company - they are/were way ahead with their technology and should have taken a less risky approach to getting that tech onto the market and taking advantage of that lead. And how did he not learn from the model X experience?

hasn't delivered full self driving (yet)

Is it really an ASIC? I thought they said over and over it is full custom.

Definition from wikipedia:

> An application-specific integrated circuit (ASIC /ˈeɪsɪk/) is an integrated circuit (IC) customized for a particular use, rather than intended for general-purpose use.

Is there anything more "custom" than an ASIC?

I suppose both are still called an ASIC. What I'm wondering is if it is gate-array/semi custom

https://en.wikipedia.org/wiki/Application-specific_integrate...

or full custom:

https://en.wikipedia.org/wiki/Application-specific_integrate...

I watch Tesla fairly closely from both the bull and bear sides. In short, I don't believe Tesla is anywhere close to Waymo. They won't achieve FSD in 2020.

It's important to see this event in the context of their significant demand and cash problems. Even enthusiastic Tesla investors like Galileo Russell suggested that Tesla is in a cash crunch and should raise money. Which hints at the main mystery about Tesla, why haven't they raised money yet?

Take a moment and think about why are they doing this event now. Elon is setting a stage for a capital raise, he's pitching the autonomy narrative after the Model 3 cash cow narrative failed. They're trying to convince investors (and customers) to give them money because money-printing autonomous taxi service is coming next year.

Also, think about why basically everyone except Tesla uses Lidars. Is it because they're stupid or because Tesla cannot use Lidars even if they wanted to?

P.S.: Nvidia issued a statement saying that Tesla's claims about their chip are incorrect: https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...

Edit: In 2012, Waymo reached the milestone of handling 8 100-mile routes, specifically chosen to capture the full complexity of driving. I doubt Tesla is currently at that level. Source: https://events.technologyreview.com/video/watch/dmitri-dolgo...

If I were a billionaire and I wanted to change the world, I would put pressure on existing companies by creating a competing company and pretending extraordinary achievements.

Truth or not, failure or not, Musk moves forward the hopes and expectations, and I'm thankful just for that.

I don't see how we can blame him for taking position

What you're describing is securities fraud.

Saying what your future goals are, while clearly and repeatedly reminding people of your past history of being late on those goals, is the opposite of fraud.

Wow. This sums up the incredible attitude of some Musk followers. And I say follower because it is literally like a cult. To this individual, Musk can do no wrong.

https://www.latimes.com/business/la-fi-hy-tesla-musk-panason...

the fact that suppliers don't like them has been a red flag for a while now

bulls and dreamers don't want to pay any attention to the very imminent problems with tesla's financials and furthermore tesla's executive record

there's a very good reason bears refer to him as fraudboy

but watching him scam taxpayers/unfortunately gullible people/govt (my fav ex. battery swap zev credits) while people fan over him and his company is pretty amazing

also the list of lawsuits...

Tesla needs to use Lidars too and frequently crashes into things Lidars would save them from (white trucks that loook like the sky, etc.). Unfortunately, Tesla promised that the cars they were shipping had all the equipment needed for self driving, so they can't say that publicly. Lidars would also bump up their cost by thousands, resulting in fewer sales.

You clearly did not watch the video stream

Aside from the timing of this event, why else are you convinced that Tesla is nowhere close to Waymo?

2020? Try 2040.

...out of all the things I expected to see from a Tesla stream, I definitely did not expect a 25 minute discussion on the cost of 32bit additions, dot products, sram bandwidth, and chip design, at the level of a third year college hardware course.

If you’ve listened to their earnings calls this is par for the course. Say what you will about his tweets, but I find these discussions fantastically informative and transparent.

absolutely glorious, they're taking questions. first question: can you use anything else besides ReLU activations in your neural net engine?

this is wonderful

I'm not a NN expert, but based on what I have found, the point of using RELU instead of other activation functions is what is called the problem of "vanishing gradients".

Basically (IIRC), during backprop the error difference gets ever smaller the further back in layers you go, ultimately getting "lost in the noise", making learning in the earlier layers more difficult to impossible.

I'm not saying RELU is the only option to make this work, or that it's the only activation function that provides a "fix" for the issue; I'm sure there are other ways to deal with vanishing gradients that I don't know about.

I also lack the mathematical knowledge as to why RELU helps in this manner, but I suspect something having to do with the lack of "asymptotic structure" approaching the extremes (I don't know what the proper term would be). Or maybe it allows for some form of "forgetting", in the prevention of multiplying very small numbers (such values just go to zero ultimately)?

Maybe someone else here with the knowledge can explain it better, and we can both learn...?

what a funny question.

"you say you can put bananas in your smoothie. just a thought, but perhaps, might I ask, do you have the capability to put strawberries as well, or are we not there yet?"

...is it?

I don't know the context here but it feels like they're trying to ML-splain Karpathy?

I keep saying this he knows what he doing. You can tell he knows what's happening. Which is rare for a CEO to know this level of detail.

Which might explain why he's a crummy CEO. If your job is to strategize for the entire company and you're busy learning the minutiae that you spend a lot of money for other people to worry about, maybe you need to consider using your time a little better. If you want to build neural nets, keep your stock, fire yourself and go build neural nets.

> If your job is to strategize for the entire company and you're busy learning the minutiae that you spend a lot of money for other people to worry about, maybe you need to consider using your time a little better

This is what everyone that is not Tesla or SpaceX are doing. And have been doing for a long time. If the CEO is not an engineer at heart, what are they?

I seriously doubt Elon Musk has more engineering knowledge than the people he hires on their specific fields. However, he can make pretty well informed strategic decisions if he knows WTF the engineers are talking about without taking their word – not even that, as explanations have to be dumbed down.

This is not a new thing. Bill Gates was like that (1). Steve Jobs was no dummy and had an engineering background, but not at the same level – he did parter with a genius engineer, however.

I think Musk is doing the right thing.

> Which might explain why he's a crummy CEO

That's quite debatable, I'd say. Isn't he getting results?

(1) https://www.joelonsoftware.com/2006/06/16/my-first-billg-rev...

But then you have the major car company problem and the phone companies before steve jobs. You have people who don't know what's possible, don't know who are the best people to hire, and can guide a vision. if you know capacitive touch screens are possible no one has ever tried it makes it easier to implement. Same with google and how their CEO are all engineers. So pitching the CEO on AI project becomes easy

Actually being aware of the technical details of your core business is a very useful skill to have as a CEO. But it doesn't seem to actually result in any benefits with Tesla. He probably knows the limitations of neural networks but that doesn't stop him from using the tried and true startup strategy of promising the world and delivering very late. Building a car that can stop on red traffic lights is something they should have had by 2016 already. But it's 2019 and their self driving technology they have demonstrated in their demo blatantly ignores the speed limits on the highway by roughly 10mph.

Funny, I get the opposite impression; the more I hear Musk talk about technical specifications, the less I think he actually understands.

Can you elaborate on this and give us some specific examples?

Holy cow Elon said a lot of things that will drastically change the world in the next 2-3-5 years. I think it's very, very clear they've been doing a lot of stuff behind that scenes that nobody has given them credit for over these last 5 years. Everyone thought they were just throwing everything at the Model 3 ramp and doing nothing else, but it turns out the opposite is true.

At the end Elon said autonomy is basically their entire expense sheet!

Whether or not Tesla can pull it off is obviously going to be an enormous topic of debate with haters and lovers on each side.

This is super, super exciting. I'm going to grab the popcorn and enjoy watching Tesla try. Whether they succeed or fail I admire them aiming so high, and planning so far ahead.

None of what Elon said will change the world in the next 5 years...

It's also pretty clear based on the selective presentation of data that they're even further behind Waymo than everyone thought. This was the time to actually prove that their ahead, and all they could give was more of Elon's hot air.

At the end Elon said autonomy is basically their entire expense sheet!

Must have missed that part, but if he actually said that it would have been a material misrepresentation of Tesla's financials, since they supposedly made a chunk of cars in Q1...OTOH, if the statement is true and Tesla basically made no cars in Q1, it would explain by Panasonic is refusing to make additional investments in the Gigafactory.

Whether they succeed or fail I admire them aiming so high, and planning so far ahead.

Other companies are both capable of aiming high, planning, and succeeding. Tesla has yet to demonstrate the capacity to do items 2 and 3 on this list. Case in point: pretty much everything about the Model 3 launch, from building to distribution, the solar roof panel (and really, the entire SolarCity acquisition). I'm even going to throw BoringCo in there since it's based out of the Tesla lot and uses Tesla vehicles and engineers.

> None of what Elon said will change the world in the next 5 years...

Oh come on! If even half of what he says comes true in the next five years the world will not be the same.

Entire car rated to 1 million miles (including the battery pack) with minimal servicing

Level five autonomy at the end of 2020. Remove steering wheels and pedals from cars a year after that.

After robo-fleet goes live Tesla will be "massively profitable"

Cars that today cost $38k will make $300k + in the next 11 years for their owners in the fleet

etc.

> Other companies are both capable of aiming high, planning, and succeeding

He mentioned that all their cars since 2016 have had redundant power steering, redundant wiring and even redundant power. Even if it loses the main power pack it can still steer and brake safely.

Give me ONE example of another auto manufacturer that has thought far enough ahead to design that kind of redundancy into their vehicles planning for self driving? It simply does not exist in the industry.

The man really is seeing the future and trying his best to make it happen.

Like I said there will be a huge debate about if he can actually do it.... I'm just saying that if he can it's going to be a very exciting time to be alive

He mentioned that all their cars since 2016 have had redundant power steering, redundant wiring and even redundant power. Even if it loses the main power pack it can still steer and brake safely.

Given that so many Tesla owners have complained about the power steering and wiring going back (and the delays in getting these issues fixed) the most likely explanation is that Tesla is attempting to forestall maintenance issues by simply adding built-in backups rather than by improving quality control and making the primaries work better.

IOW: having built-in redundant parts for some of the most important parts of the car--which aren't known for breaking down in cars from other manufacturers--is actually a sign of failure on Tesla's part, not a sign of forward-thinking.

Entire car rated to 1 million miles (including the battery pack) with minimal servicing

That's nice. Tesla can't manage to get car panels aligned properly, and a number of cars are still awaiting parts a year or more after they went in for service, and Tesla hasn't managed to fix its own billion-dollar factory line to work properly...but in 5 years they're promising better-made cars than any other car maker? Toyoto has earned the right to make that claim with their decades of consistency. Tesla hasn't, especially not after the Model 3 fiascos.

Level five autonomy at the end of 2020. Remove steering wheels and pedals from cars a year after that.

Call me when they manage to actually stop their cars from veering toward freeway dividers. It was still happening as of November 2018... Also, there's the pesky little problem of Tesla not yet having demonstrated a door-to-door trip yet in uncontrolled conditions despite promising that back in 2016.

After robo-fleet goes live Tesla will be "massively profitable"

See above. Need to hit Level 5 before then. But based on all the lots filled with unpurchased Model 3s, they'll probably have the fleet for it if they ever achieve Level 5...

>Oh come on! If even half of what he says comes true in the next five years the world will not be the same.

I could say 20 things that if even 1 comes true in the next five years the world will not be the same!

I took it as weasel words. The robotaxi business will be the vertically integrated capstone of everything Tesla does, therefore 100% of their expenditures are building towards the robotaxi.

And I was told for years by HN that actually, Tesla is a battery company.

Now it's a robotaxi company? What is it going to be next year, a debt restructuring company?

It's so frustrating to see that people eat Elon's BS. Even after:

- the Unsworth lie

- the "zero concern and I mean zero about 10k Model 3s per week in Dec 2018" lie

- the "funding secured" and "only uncertainty is shareholder approval" lies

- the "short burn of the century" lies

I mean, they were even wrong about Nvidia's chip today, Nvidia had to issue a statement: https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...

The tech specs of AGX Pegasus say they're 130 TOPS[1]. Or am I misunderstanding something?

[1] https://developer.nvidia.com/drive/drive-agx

Where are you getting 130 from that page? I don't see it. On this other page [1], it clearly says 320. "NVIDIA DRIVE AGX Pegasus™ achieves an unprecedented 320 TOPS"

[1] https://www.nvidia.com/en-us/self-driving-cars/drive-platfor...

There was no "Unsworth lie". There was Elon saying some very unkind things after Unsworth personally attacked him. Neither thing is appropriate, but let's not pretend this wasn't a tit-for-tat.

There's huuuge difference between telling someone to stick something up their butt VS repeating, multiple times, to millions of people, that someone is a pedophile and you have proof of that.

He purposefully framed it as a seriously meant allegation, not as an insult. And then even doubled down on it.

When plotted on a logarithmic scale.

And I think they are far ahead of the game compared to anyone else w.r.t autonomous driving, they have a fleet sending them data back real time.

This is definitely not true. The engineer explicitly stated that Tesla only gets data when trigger conditions are met (and that this was in fact always the case), so most of the miles that Tesla drivers have been driving was actually never even sent back to the mothership for use in ML training.

IOW, despite Elon claiming for years that they're ahead because of the "millions of miles" of free data collection by Tesla drivers, it turns out once the engineers on the ground actually speak...they're actually pretty far behind on the data front, and most of the data they are getting is the same safe routes over and over again since they're not trying to put Autopilot into novel situations to acquire new data or test the ML algorithm's capabilities to handle unique/novel situations (like trucks turning in the highway, or freeway dividers).

They cherry-pick rare cases and use their fleet to get more examples of these situations. This seems like the right approach given more miles following the same car in a straight line is pretty useless.

My takeaway from the presentation is that Tesla will perform better than other companies in this space (although I don't know enough about Waymo to comment) due to the following:

-You want a large dataset (Tesla and many companies have this and can simulate) -You want a varied/diverse dataset (Tesla and many companies have this and can simulate)--the point here is simulations for simple cases work (you can only simulate when you know), but for complex ones are close to the difficulty of actual FSD -You want a real dataset (Tesla is the only company who can say this and can say they have data on how X00Ks of drivers will handle these situations)

Isn't the "millions of miles of free data" the edge cases? Yes, they're not literally streaming back everything but they have millions of miles with people driving auto-pilot and sending back a significant amount of edge case scenarios, which is the truly valuable information. Sending back data of "successful auto-pilot" can be sending back 10 hours of driving on a straight two-lane road; which seems quite useless.

They also have cars all over the planet while Waymo seems to only be tested in a few cities. I can't find the podcast, but Elon mentioned how one of the main difficulties is the varying types of intersections. There are only so many intersections in Mountain View. But Tesla is getting data for all kinds of intersections.

I have wondered about this myself... and I am working on an CV ML project myself... but data that is correctly predicted is not not as useful.. what you want are edge cases, images with low confidence, and errors. So when users take over and disengage, that is useful data. when the system beeps and forces a disengagement.. this is the data you need. That has been my experience and I'm glad to hear them validate that. Your training data needs diversity.

If the current state of autopilot is even tangentially related to their progress on full self-driving, then it's a lot more than a few years away.

How recent is your experience? I’ve owned a Model 3 for over a year and the pace at which autopilot has been improving is staggering.

Which car available for purchase would you recommend looking at for something closer to full self-driving?

Quick look at their earnings shows it's not true. With amount of cars they have, even sending tiny % of it back home would require Tesla to spend billions on datacetners.

or he's setting overly optimistic timelines at usual. But I would love to get excited this easily

So far, half an hour from the head of IC design. Nice special purpose IC and board. Dual everything for redundancy. Wide special purpose neural net evaluation. 100 watts for the compute system. Code signing. Shipping in new Model 3 cars since last 10 days.

"All we need to do is improve the software" - Musk [12:07 PDT]

LIDAR "unnecessary" - Musk [12:13 PDT]

Computer vision guy is now speaking.

Recognizes "driveable space", not just obstacles. Video shown, but just for a freeway. This is crucial to safety. Need to see this is a cluttered environment.

> "All we need to do is improve the software" - Musk [12:07 PDT]

That's an old claim, see this 2016 press release [1]:

> as of today, all Tesla vehicles produced in our factory – including Model 3 – will have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver.

At this point I believe the claim once I see the self-driving functionality having been rolled out to the public and the accident number been reduced.

[1]: https://www.tesla.com/blog/all-tesla-cars-being-produced-now...

"LIDAR is a fool's errand. Everyone relying on LIDAR is doomed." edit: beat me to it

it snowed a few weekends ago in Chicago, my autopilot turned off because snow covered up the cameras. So I am not buying all this "self driving with no lidar" brouhaha

And would you drive if snow covered your windscreen? Or would you clean it? Which is obviously possible for cameras too.

OK but snow blocks lidar too.

Snow completely disables lidar as well if I recall correctly. Only radar will work in a snowstorm.

Doesn't LIDAR has similar problems with precipitation? Regardless of what solution proves to be the best, there is going to be a decent amount of time between when a self driving car can handle most scenarios and when it can handle all possible scenarios.

Lidar is useless when snowing

Can snow cover lidars?

Not cleaning snow off of your vehicle before driving is crazy.

This talk of a "cut-in" detector is scaring me. It's like they lack any sort of higher level planning and decision making (which notoriously is not a neural network of any sort).

Musk says there will be a driving demo later today. Up next, the neural nets person.

how did the demo go? Is there separate video?

the narrative in this conference seems, minus the exaggeration, to follow exactly the flow for the presentations given by the MobilEye CTO for the last six years.

Having two GPUs on one board is not redundancy.

The thing I don't get about Tesla is why they keep heaping more on their plate rather than just (for large values of just) re-doing the motive power bit. That alone is a massive challenge and it is not something they've licked at economies of scale that allow a $10K vehicle to be brought to market, which is what it will take to really make this a success.

Every extra they tack on to the base product is one they will be expected to deliver on as well diverting attention from the main problem they are dealing with, which may in the longer term leave open enough room for the competition to wiggle through.

Sure, autonomy is a big deal, but it is also something that will once cracked be an instant commodity and there is a lot of money and talent focused on that particular problem, which is surprisingly hard to do well.

If Tesla ends up going under because of one of the side shows (Solar City, autonomy, Power Wall etc) that would be a serious loss.

I think it's related to who Musk is, and the kinds of things he wants to achieve.

SpaceX isn't trying to just do what has always been done and just make it 5 or 10% cheaper. That's boring. SpaceX is completely turning the launch industry on it's head.

Tesla is the same. Elon isn't interested in making "just" an electric car. He wants to change the world, and electric cars that drive themselves will do that.

Can he do it soon? I'm not an expert, I don't know. But damn it's exciting to watch.

The purpose of Tesla is to save the world from global warming, though. I'm not sure self-driving is that relevant compared to the electric propulsion part.

If anything it'd be detrimental since it would allow more people to drive longer and more often.

First of all, a lot of these things are orthogonal. With that I mean, that they are independant from each other. The self-driving effort interacted only minimally with getting the Model 3 production up and running. As long as the money does not run out, it makes sense to do things in parallel, which are parallizeable. On the other side, Musk has a big picture in mind, which stretches quite a bit into the future. So these "side shows" actually fit together. You use solar cells to produce electricity, which you store in your power wall which can power your electric car.

If Musks ideas about self-driving actually become true, he holds the holy grail of the car industry. On one side, self-driving abilities become crucial in future car sales. On the other side, if he can make the Tesla car sharing network happen, the revenue of that could be a magnitude or more larger than the one of car selling. This would justify some of the crazier valuations of Tesla stock.

> Sure, autonomy is a big deal, but it is also something that will once cracked be an instant commodity

Not if the company that cracks it eats its competitors or, assuming it's like waymo, commoditizes car manufacturers into suppliers for autonomous taxi fleets.

How would power wall lead to them going under? They would just be selling excess batteries, which seems to be one of their advantages over competitors. Autonomy, I'm ambivalent on it, they can charge customers for it right now and many will pay.

If anything, the solar city/roof shingles side-show has seemed the worst to me personally, alongside the falcon-wing doors and the self-infliction of twitter harm by the CEO.

> once cracked be an instant commodity

Can't they patent their breakthroughs? How is this different than pharma breakthroughs?

“diverting attention from the main problem they are dealing with, which may in the longer term leave open enough room for the competition to wiggle through.”

Perhaps this is the intent rather than a side-effect.

Indeed, Musk has encouraged competitors from the start. I believe they open sourced a lot of their patents in an effort to encourage competition in the electric vehicle space.

I was actually referring to the first part of the quote.

Intentional misdirection to distract investors.

>but it is also something that will once cracked be an instant commodity

Definitely not and I'm not sure why you think this.

The data pipeline advantage is a big part of this talk.

I wouldn't have bought my Model S if it wasn't for Auto Pilot and the promise of Full Self Drive

> The thing I don't get about Tesla is why they keep heaping more on their plate...

Musk's ego plays a lot into this for better or worse.

Two words: Vertical Integration.

Apple's work on developing their own SOC (A13) have paid dividends for them. The same legendary chip designer (Jim Keller) designed the new Tesla chip after working for Apple.

For vertical integration to work you need to have funds to support it long term. Apple has it. Tesla - not that much (at least not in the current state)

Interesting pieces:

- First principles hardware design of focused self driving computer (many times better than any competing existing hardware). Already shipping in all newly produced cars. Currently working on next gen that will be 3x better to ship in a couple years.

- Lidar is an unnecessary mistake that competitors are making that won't succeed (too expensive, need too many, unnecessary).

- Real world fleet testing is critical to success, simulations are not good enough since there are too many unknown unknowns in the real world. Tesla uses simulations too, but nobody else comes close on real world fleet testing.

> Lidar is an unnecessary mistake that competitors are making

This is very controversial and rest of the industry thinks exact opposite, many even claiming that Tesla is being irresponsible and even delusional in trying to do autonomy without lidar. The main points I have heard in favor of lidar are that computer vision is very flaky not only in suboptimal weather but even in good weather. Cameras are simply no where close to in performance in dynamic range, rapid adaptation, focusing etc as human eye. Imagine car going under the shaded road with rapid changes between bright light and shade. The likelihood that your depth estimation will get messed up is very high. Of course, night driving becomes highly questionable as well. In additional the long range depth estimation is very flaky with stereo vision right now and a topic of research for mono-vision. If you want to retreat to level-4 only and that too with conservative speed, weather etc then may be vision+radar more doable?

They pair it with a forward facing radar which is inexpensive and good at depth perception.

Elon predicts all competitors will eventually drop lidar. He mentions it's expensive, but also not as good in a lot of cases (and all roads/signs are designed for vision).

He argues that getting vision to work is a prerequisite for getting self-driving to work and once you have it working, lidar is worthless (and unnecessary).

Elon Musk is noted for his premature optimization.

Case in point: both the Gigafactory (vastly overbuilt for the quantity of batteries actually produced) and the Alien Dreadnought (vastly overbuilt for the number of cars Tesla current produces...assuming that Tesla is ever able to get the fancy automation working). Boring Co digging a two-mile tunnel in West LA without bothering to learn how to pour concrete smoothly, or to make the "rails" the proper width, or learning about ventilation, or access points....

“In my view, it’s a crutch that will drive companies to a local maximum that they will find very hard to get out of,” Musk said. He added, “Perhaps I am wrong, and I will look like a fool. But I am quite certain that I am not.”

https://www.theverge.com/2018/2/7/16988628/elon-musk-lidar-s...

Their forward facing radar has failed to avoid crashes into big, stationary objects (at least 2 semi trailers, a fire truck, and more). The explanation I got was that radar is rather noisy, so it's handy to filter-out stationary objects (like road signs, broken down vehicles on the shoulder, and fatally, any any semi trailer crossing your lane)

I think it also fundamentally depends on the business model being pursued:

- If you are trying to sell the Model 3 directly to the end consumer with autonomous mode, the extra $10K for Lidar and bulk (which will greatly effect the exterior design) are definitely non-starters.

- If you are going to robo-taxi route (such as Waymo and Uber), the extra one time $10K cost to add lidar for the 5 year life of the car is probably a blip on the income statement of the operator as compared to a full time human driver which probably costs $10K PER MONTH for the lifetime of the service. For the robo-taxi business model - its a bit of a no brainer - they could stick every sensor known to man on the car and still make out like a warlord by getting rid of the human driver but maintained a 100% safety record. Plus making the car stand-out with a unique Lidar inclusive shape is a great marketing differentiator. Also reduces your liability if a taxi rider ever sues since you can claim you have redundancy in the system.

I don't think anyone wants to use Lidar - the claim is that they need it to achieve full, safe, autonomy. If it turns out that it isn't needed than Tesla will be in a great position.

But only time will tell.

- Only a matter of time (evolved AGI research) before vision suffices. We know it can work because of bio evolution.

- Also only a matter of time before lidar cheap enough that even Tesla will add for redundancy and edge cases.

>many times better than any competing existing hardware

Except that it isn't. NVIDA's Drive AGX Pegasus delivers 320 TOPS:

https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...

Of course, nobody inside the Musk Reality Distortion Field actually cares about this.

Not that I'm confident about either side of this, but the TOPS isn't the only important factor here. It's also the efficiency. If Tesla's CPU can do more ops/watt, then it's more ideal for the conditions that Tesla needs the CPU to fulfill. On top of that, I have no idea what "320 TOPS" means in the context of the specific workload Tesla has. Will they _actually_ get 320 TOPS?

This software talk is fascinating and a great introduction to neural nets.

Next iteration of FSD is going to be 3 times as full. Can't wait.

You missed the most glorious part of

>Currently working on next gen that will be 3x better to ship in a couple years.

Someone asked what the primary design objective of the next generation chip would be and the engineer muttered 'safety' before Musk made the 3x claim.

Eh, I'm pretty fine with that. Imagine the current gen slightly beats human drivers - also known as "revolutionary, but not perfect" - of course the next gen would focus on improving safety further.

> Currently working on next gen that will be 3x better to ship in a couple years.

That's ... not great. If that was CPU type of unit, it'd be great. But TPU-type accelerators are growing at massive speed (as it's still pretty new and simple tech), where you're more looking for 10x type of gains.

"many times better than competitors"

"next gen that will be 3x better to ship in a couple years"

applying psudomathematical varnish to marketing-speak

I was summarizing - they gave explicit technical details in the talk.

Nvidia says they were comparing against an inferior chip https://www.marketwatch.com/story/nvidia-says-tesla-inaccura...

The talk was extremely technical

Tesla: "We have a global network of cameras that can be queried to find anything on or near roads and send back photos of it."

Every law enforcement entity on the planet: drool

If this could only be used for good it would be amazing. It could look out for lost pets, missing people, stolen bikes, suspects, etc.

It was pointed out that All data is anonymized.

That's fine. I don't care who's driving the car carrying the camera. I care about what the camera can see. If you've ever been sent an Amber alert for a local child abduction it often comes with a description of the car that the suspect is driving. If I'm in law enforcement I'd much rather get an emergency warrant to ask Tesla to query the network to find that kind of car in a given geographic area.

It's trivial to identify the car. You can see the reflections of the plates at times. If that was removed, you'd still have the ability to cross-check it with traffic cameras (or business cameras).

Bull.

You can't say "You can query for anything!" and have it be anonymized.

In fact, if you're taking a camera shot, that's the most broadband sensor imaginable, and no thanks, I don't trust any company with profit motive not to crumble and pop out a software update to selectively disable picture blurrers, or to not have a Boolean flag in your data scrubber.

As a company, making that statement is just trying to slip one past and hope nobody asks inconvenient questions about it.

anonymizing data is non-trival.

At the tail end of Karpathy's presentation, he said something that reminded me of why Peter Norvig decided to join Google: because that's where the data is. In Tesla's case, they have a unique and likely accelerating advantage in having the best source of data up on which to train their models. I think this forms the basis of an enduring moat relative to their other competitors, none of which are collecting data at the same scale.

Is Tesla collecting that data? I've looked at their earnings and didn't see any big expenses related to datacenters. Amount of data they would need to store and process is massive, that would for sure show up in earnings, as one of the biggest costs.

I wonder what the ratio of fleet-driving bytes collected is for Tesla vs Waymo vs the other big players...

While you're waiting for the main event to start, here are some recent interviews with Elon about self-driving cars. He's very confident.

"To me right now, this seems 'game, set, and match,'" Musk said. "I could be wrong, but it appears to be the case that Tesla is vastly ahead of everyone."

I am eager to see what they unveil today.

https://www.youtube.com/watch?v=dEv99vxKjVI

https://ark-invest.com/research/podcast/elon-musk-podcast

Hmmm so then why is Tesla ranked last for autonomous driving by third party researchers?

https://www.google.com/url?sa=i&source=web&cd=&ved=2ahUKEwiR...

And Elon has a long history of making false claims about Tesla’s progress. For example in 2015 and 2016 he claimed that Teslas would be fully self-driving by 2018.

So why shouldn’t we be skeptical?

https://arstechnica.com/cars/2019/03/teslas-self-driving-str...

But Navigant curriculum was very unscientific. There no actual quantitive reason Tesla is worse. Is was based mainly on business factors like go-to market strategy and vision.

The first article you link to sources another article as its source, which itself calls bullshit on the ranking.

Your link:

> According to Electrek, Tesla trails behind other companies in terms of autonomous driving tech based on a list created by Navigant Research, an independent research firm.

Electrek’s article:

> Electrek’s Take

> I think Navigant’s autonomous leaderboard is ridiculous. There are way too many brands that keep most of their development under wraps, which makes it hard to evaluate them and therefore, it gives very little value to a leaderboard like this in my opinion.

What other major manufacturer has anything close to Tesla's autopilot in a car I can buy today? As far as I know, no one.

Some big time source laundering going on in here, https://electrek.co/2019/04/19/tesla-falls-autonomous-drivin...

Your "third party research" is obviously bullshit, they go as far as including Apple in their ranking.

This right here is just typical worthless marketing press release spam from a management consultancy firm.

My guess is he means "on the highway". The scary bits of self-driving is person detection, crossing detection, roadwork detection, cyclist detection (e.g. coming up on the right when you are trying to make a right turn).

The Waymo end-game that I heard was "able to go through a drive-thru". I highly doubt Tesla is anywhere near that point.

There have been news reports about the model 3 autopilot getting its speed limits from maps, lacking any sort of sign recognition or manual override to adjust to local conditions. The maps seem to be outdated for germany (1). That’s an essential feature even on the autobahn. Given that test result I’d even be skeptical about any claims of being ahead of the game on the highway.

(1) https://m.heise.de/autos/artikel/Test-Tesla-Model-3-4400919....

> through a drive-thru

The kind of drive-thru that Tesla is currently associated with involves semis rather than fast food and it would be really nice to hear that they've at least licked that particular bug (and for good, this time).

> The scary bits of self-driving is person detection, crossing detection, roadwork detection...

Your point it very astute.

Among a few other ML/AI MOOCs, I completed Udacity's "Self-Driving Car Engineer" nanodegree - so when I'm out driving, I often come upon situations where I wonder "how would a self-driving car navigate this?"

Today, driving in to work (note: USA), I noticed one intersection I've been through many times before, and that question came to mind. The intersection is interesting, because on approaching it, the road curves to the right, and you can actually see one of the traffic lights on the left before you even see the intersection. By the time you see the intersection, you're already on top of it.

So as you round the curve, you see the lone traffic signal (red/yellow/green); if it is red, do you start to brake, or do you wait until you can "see" more traffic signals? If you wait - will you have time to slow down and/or stop? ...and so forth.

This and others are all kind of "edge cases" that will need to be trained on, and/or perhaps other cues for self-driving vehicles installed or set up so the vehicles can navigate such areas successfully. I know when I first went through the intersection it was a bit of a surprise; it's not a very safe intersection (going home in the opposite direction is not any better; in that direction, you're headed downhill, have to cross the intersection, and immediately start turning to the left after going through - the curve is really abrupt, and you have protected/unprotected left-hand turns both directions, etc).

Well they’re vastly ahead in one area: data collection. No other company is even close. You could argue about the quality of data but the platform is there and ever growing, and they can upgrade their hardware in the future and augment existing data.

Don't kid yourself, the car has no bandwidth storage or performance to send back anything other than a few raw frames from disengage events or other rare triggers.

Maybe too much confident if you ask me..

I hope Elon has tested the autopilot in Finland during the winter then.

I really really don't want Tesla to die. I think it's an important company for a sustainable future. Failures in autonomous driving could easily turn into the straw that breaks the camels back.

If you call something "autopilot" and promote it as if it will drive for you and then it ends up killing dozens of people ... that's where successful class actions come from.

(I obviously don't want people to die either.)

And I don't want them to succeed because they are actively hostile towards 3rd party repairs and car modifications, plus their current model 3 microtransaction bullshit scares me as a customer, I'm dreading the moment other companies catch on with that. I'm specifically talking about the fact that the base model 3 ships with heated seats out of the factory but you can pay to have them unlocked with a software update. I suspect that having them enabled with a simple software mod would get you accused of piracy since you're using something you haven't paid a licence for(even though you've obviously paid for the hardware)

Personally speaking I would not want to share a road with an autonomous car that had been tinkered on by some random Joe. Autonomous farm vehicles or something, sure, but public roads are way too risky. This is not the desktop environment of your Linux laptop, it’s monstrously complex software controlling thousands of pounds of metal and batteries.

> I'm specifically talking about the fact that the base model 3 ships with heated seats out of the factory but you can pay to have them unlocked with a software update.

If you think this is a new thing, you haven't followed the car industry much.

Just as an example, my 2002 el cheapo Peugot was bought without the option to show instantaneous MPG/average MPG/range/etc. Spend two hours soldering/gluing on the missing $2 toggle switch, ask a friend with the Peugeot Planet update tool to enable it in software, and Bob's your auntie.

It goes much further than that though, especially when you get into chip tuning. Software upgrades that add 50 horsepower to your engine output are commonplace.

But as for the "actively hostile to 3rd party repairs/mods", this part scares me to.

dozens?

How about 2 thus far and Tesla was not at fault.

This focus on the hardware is silly. Assume for a second that their new hardware is 50x faster than their last hardware.

That does not mean that their cars can self drive today.

That does not mean that their cars can self drive three years from now.

It's 100% not proven or obvious how car self driving skill and car self driving error rates scale with compute -- but it's surely not linear.

What if they proved it by improving hardware from self-driving hardware 1 to 2. If they proved that and proved it made their NN better then it makes sense. Of course, it wouldn't to you cause you have no inside knowledge but it may make perfect sense to them based on their data.

I’m wondering what new spinoff comes from the hardware... does it have benefits / application to anything in addition to AD?

There's another one of these coming about the software specifically.

Musk: "You're only going real fast in the forward direction."

Dozens of times in my life, I have been driving down the freeway at 70 MPH in a 60 MPH zone, and I've noticed someone weaving through traffic behind me, going closer to 140 MPH.

I need to know whether to change lanes, stay in my lane, stop changing lanes, pump my breaks to indicate there is slowdown ahead of me that car might not see, etc.

Just food for thought.

EDIT: I'd also like my vehicle to be good at avoiding a car that's about to T-Bone me, at night, with no headlights on. I may not be very good at avoiding that kind of accident today, but if LIDAR is necessary to protect me from that kind of accident, then I might think it's a wonderful idea.

You're not responsible for dodging the person going 140 MPH behind you to the same extent as you are for dodging the tire doing 0mph in front of you. Humans generally don't because they aren't capable of paying enough attention to what is behind and in front of them simultaneously. The car - with 360 degree vision - is more capable in fact.

Even if I'm not responsible, it's Defensive Driving. It can save my life to be aware of the dangerous situations that other drivers are causing.

I’ve been driving for nearly 30 years and have never even seen one person going 140mph, yet alone dozens

The person who wrote that doesn’t realize how fast 140 would seem. That kind of speed differential is not something you can attempt to avoid. Even 100 vs. 70 is rare, and the safest thing to do given that differential is nothing. This is not an area any car company should worry about.

Those are some very bold claims. Level 5 by the end of this year? I find the software approach intriguing and Karpathy's segment was enlightening and did a lot to convince me of Tesla's advantages.

On the other hand, I've been sensing that Tesla is finding it harder and harder to raise cash and has been getting increasingly desperate. Are we on the cusp of a new transformative technology or the peak of the mother of all bubbles? Time will tell.

Option 1: They have solved a problem so hard that no one else, even Waymo, is claiming to be close even with $100k+ worth of sensors and a lot more compute and they have also quietly solved multiple hard unsolved research problems.

Option 2: He's lying to get money.

Here's the new self-driving demo video from today.[1] From Tesla's HQ in Palo Alto, out to I-280, down one exit, use interchange at Sand Hill to turn around, come back. No visible conflicting traffic on non-freeway streets.

Compare the 2016 demo video.[2] That's a tougher route. That's the one where we now know it took a lot of tries to get a clean video.

Waymo and Cruise have put up videos of their cars in city traffic. They get criticized for things like getting stuck behind double-parked cars, and being a bit shy of parked cars that project into a traffic lane. But they get where they are going. Tesla is not showing anything near that level.

Supposedly the analysts at the meeting got to ride in a self-driving car. Anyone seen reports from them?

[1] https://www.youtube.com/watch?v=tlThdr3O5Qo [2] https://www.youtube.com/watch?v=eAal0juXXzU

https://twitter.com/hamids/status/1120466861970399232

These guys also went on the ride-along: https://youtu.be/2BZHXh1nbWc?t=360

Feels like they're just throwing a bunch of hardware specs at investors to distract from any discussions about the current state of the software beyond "just need to improve it now".

You cannot access the stream from Tesla anymore, I found it here - https://www.youtube.com/watch?v=tbgtGQIygZQ Karpathy did a great job explaining this, I took Neural Networks a long time back and his back to basics approach refreshed a lot of concepts I'd forgotten!

It's no longer in that link. I found it here: https://www.youtube.com/watch?v=tbgtGQIygZQ

Currently just showing some generic car footage. Apparently they will demo their self-drivin features: https://www.teslarati.com/tesla-self-driving-autonomy-day-wh...

Yes, I was feeling dumb watching the same thing in a loop, hopefully they’ll start soon with the real meat...

“Anyone using Lidar is doomed”

Strong words from Musk about sticking with video only.

Later on in the software talk:

“Lidar is really a shortcut which sidesteps the fundamental problems...and gives us a false sense of progress”

I wonder if he's going to explain this position at all? Or if it's just posturing against Waymo et al. Seems like more data is always better than less data for this application?

Seems to me that relying too much on Machine Learning for this is actually the real risk, not LIDAR. Tesla's competitors are using ML, but combining it with sensor fused data from LIDAR, Radar, accurate maps, and visual data. In other words, they're starting from a view of the world with super-human sensing.

Starting from a purely visual domain likely dooms your system to making the same types of mistakes that humans make visually in judgement. At least with accurate maps and LIDAR as backup, you can sanity check output of your visual processing against a map and LIDAR. If your claim is that the map might be out of date, or the LIDAR too low rez, it still helps to err on the size of caution.

In the worst case, the map tells you you can't go somewhere that you're allowed to go. In the case where it says you are allowed to go somewhere were you shouldn't, well your visual system should be telling you not to go there. If it isn't, you're in a lot worse trouble than having a map with flaws.

Likewise, the claims that "LIDAR is expensive" is like claiming EVs are expensive because "batteries are expensive!" If AVs become common, then they'll be a huge demand for LIDAR and the costs will decline. LIDAR costs are already declining. Maybe if Elon took some of Tesla's miracle engineers and had them make a LIDAR, they could not only defeat Nvidia at chip making, but defeat all LIDAR makers as well. (sarcasm) My guess is the real problem with LIDAR is drag and vehicle trim/styling.

At this point, Tesla is basically stuck. They bet big early on a shitty sensor suite before AV technology had been worked out, and wave their hands about how using consumers as guinea pigs to feed them fleet data will magically fix deficiencies in sensors. Well, what if this is a false hope and it doesn't? It would mean they'd get sued by everyone who bought the AV suite as an option and have to issue refunds or recalls to upgrade.

I've said it before and I'll say it again, you don't ship AV as an MVP on a $100k vehicle and promise magic upgrades and fixes later before the technology is even close. It's risking people's lives and it's already killed people.

Indeed, Tesla is on the wrong side of history in sticking to their guns on pure vision autonomy.

And they're going to kill people.

It's hard enough already to communicate to the public in any meaningful sense ML results without #SKYNET brigading, Tesla is going to make it 100x harder dragging us all through their mud.

I think at this point (of having been a matrix boi long enough) I am allowed to publicly state my belief that we are very very very far away from human-level visual perception. Just staggeringly, incomprehensibly far. We've just barely started to be able to do the most basic things, sometimes.

Is it an incredible amount of collective dunning-kruger that is blinding tesla? Or perhaps willful ignorance?

I agree with all your points. However, Musk did mention that his team at SpaceX built their own LIDAR for docking with ISS. I think Tesla can custom make their own LIDAR too.

And I would secretly look forward to. I was part of a self-driving team at university and we had to raise funds especially to be able to afford a Velodyne HDL-32.

Q: "What's the primary design objective of the next-gen chip?"

Mumbled Answer: "Safety."

...doesn't that mean the current-gen chip... isn't as safe as you want?

No, it just means the next-gen design had safety as a primary design objective. Secondary objectives could be additional modes of self driving like traffic light detection and managing intersections.

Volvo leadership states the exact same thing about their car designs. The primary motivation “in everything they do” is safety.

The statement can be made entirely distinct from the current safety level.

I don't see this line of reasoning. To me "safer than humans" or "in the 90th percentile among drivers" is a fair go to market line.

Optimally safe should still be the goal. That is, no human could have provided alternative input to the computer that would have created a safer outcome.

Are any cars ever as safe as you want?

Man -- his engineers must cringe quite a bit when he talks about a future self-driving architecture built on DOJO with 'video-in, actuator controls out' ... just a tiny bit of speculation goes into saying something like that ... To my mind, it undermines the presentation quite a bit which otherwise appears to mainly argue that they will reach full self-driving by iterating on their neural network sensor perception model.

Andrej Karpathy (who was one of the presenters today) actually made similar statements last year, about how the part of the software stack that is not neural networks is gradually shrinking: https://youtu.be/y57wwucbXR8?t=352

Nope his engineers have talked about this long before he did. More than likely they convinced him not the other way around.

Some video of demo rides from someone who was at the day:

Lane change on crowded freeway: https://twitter.com/hamids/status/1120472369762590722 - It manages it if not quite as smooth as a human.

Summon in parking lot: https://twitter.com/hamids/status/1120446405410205701 likewise.

There's also an official Tesla sped up demo vid https://www.youtube.com/watch?v=tlThdr3O5Qo - hard to tell really.

Same video slowed with the display magnified and some Reddit discussion https://www.reddit.com/r/teslamotors/comments/bgb9or/fsd_dem...

I think it’s a safe assumption that driver alertness positively correlates with lives saved. But it’s hard to market this product with the sort of rhetoric that would optimize for that, so they’ve found a few ways to console themselves as they name the feature “autopilot” and endeavor to make the tech appear magical.

If they think they’ve cleared the bar just by beating the crash statistics of all cars, then they are still in the business of trading lives.

Here is the link to the recording: https://www.youtube.com/watch?v=Ucp0TTmvqOE

That video link shows as unavailable.

What kind of event is this and who is it targeted to? Right now there’s a comparison of energy consumption for different instructions on processors, I’m very confused.

Surprisingly technical versus what I'd expected, but it is targeted specifically at investors.

To summarize, it appears that their new chips can do 96x96 dot product in a single cycle (multiplying neural network weights by values), and have hardware for activations (ReLU, softmax, tanh) and pooling (as in convolutional neural networks). This results in a crazy 144 trillions ops/second.

How does this compare to TPUs and the Neural Engine in iPhone CPUs?

Can they actually do a 96x96 dot product in a single cycle? My understanding was that they can do 1 96x96 dot product per cycle - that is to say that their throughput is a 96x96 dot product but it'll be heavily pipelined.

something I don't understand, is this chip brand new? is it essential to the autonomous drive?

Is there a link to a replay of the live stream? The link on the Tesla IR site just takes me to their main page.

It was on Tesla's youtube channel up to about 2:30pm, then it just went 'unavailable' as I was watching it, and is now unavailable.

https://www.youtube.com/watch?v=tbgtGQIygZQ

It's interesting to contrast Musk's stated aversion to LIDAR with his previous statements about new technologies always starting at the high end and working their way down [1]. Tesla appears to be trying to start at the low end (HW is included with the purchase of a car even if you decide not to opt for the software to make it function. You might imagine that you would start with all the bells and whistles and pare them down as the technology matures and you invent ways to do without.

[1]https://twitter.com/elonmusk/status/1075126514851602432

How can Musk make the prediction of a million autonomous taxis in less than two years? That's just fantasy. There are just too many unknowns to solve. One of the big ones is permission from the city governments. Once a car hits a pedestrian or multiple taxies start having accidents, governments will bring the project to a halt. Tesla might have a small test fleet somewhere but not 1 million taxis.

My guess is that Tesla will need another round of financing along the way and they have to make sure that the stock's price stays up. Musk is a great promoter but at some point he needs to come back to reality.

I wonder if they are going to address potential limitations to their systems (bad weather, snow etc). Their tech seems absolutely impressive but I suppose there will always be failure. Does the system fail "gracefully" ?

New link: https://www.youtube.com/watch?v=tbgtGQIygZQ

(replay after the livestream ended)

It seems like Tesla is at a pivotal moment.

On one hand there are hyper-bulls who claim Tesla is a $4000 stock and the future of transportation. On the other, hyper-bears claim the equity should trade around $0-$10. There seems to be no middle ground.

It seems like they are almost betting the company on FSD. I don’t think FSD is really even close to a possibility over the next 5-10yrs. I hope I’m wrong, but if I’m right, I don’t see how Tesla keeps going on like this.

> On one hand there are hyper-bulls who claim Tesla is a $4000 stock and the future of transportation. On the other, hyper-bears claim the equity should trade around $0-$10. There seems to be no middle ground.

Well, other than actual investors.

>On one hand there are hyper-bulls who claim Tesla is a $4000 stock and the future of transportation. On the other, hyper-bears claim the equity should trade around $0-$10. There seems to be no middle ground.

Well the middle ground is the actual stock market where Tesla is trading around $265.

> It seems like they are almost betting the company on FSD.

IMO they don't have any choice. The more and longer they operate like a car company shipping bigger and bigger volumes, the more their financials will be undeniably trend towards those of the existing car companies and the more their existing market valuation will be hard to justify.

They need something like this to not get traded at traditional car industry multiples.

I think FSD would be cherry on the top. If they can continue slashing down the prices and improve battery life it would be revolution in itself. Even just continuing with current pace if they can produce $18K electric car with 600 miles range in next 3-5 years, you could be golden as shareholder. Their advantage, like Apple, is attention to details and awesome design that other car manufacturers have failed to replicate.

I have several questions about Karpathy's presentation: 1. How to label automatically using fleet learning? An example he gave is detecting cut-ins. However, I am wondering if it could be applied to more general and basic cases, for example, labeling lane marking/vehicles. 2. How do you know if someone is a good driver (which is essential for imitation learning)?

I read through all comments. Very informative.

How many people and companies are going to freak out if Elon and his team deliver FSD without LIDAR? Say in 2022, not even 2020. My layman logic is that AI is improving exponentially, and we do not know what artificial vision can do in 2 years.

This is the most exciting technology (software & hardware) and business event in a long time.

They use the batch size as 1. That is surprising to me as all the online course i see the batch size b/w 32 and 256.

Elon just promised Level 5 FSD this year, by the end of it.

If by this year he means 2050, then I'd still be kinda surprised.

I just spent the weekend racking up several hundred miles in a brand new Model P3D. Phenomenal performance, I am completely sold on the future of EV. I was underwhelmed by autopilot however. Drives like a drunk. The adaptive cruise control part actually made me motion sick. I had high hopes going in, so it was a pretty big let down.

I suspect Tesla is going to find themselves on the other end of a class action lawsuit over all the FSD upgrades they've been selling.

I wish someone will ask what gaps remain for full self driving? It seems like Elon is saying that it just a data collection issue at this point.

Absolutely great talk! Especially impressive how big the fleet is and how well they can use the data they have.

15 minutes in and it is still just showing stock footage. This thing was supposed to start at 11am PST, right?

I feel Tesla livestreams are very much like their cars - they never launch on time, but when they eventually do, they're pretty good and worth the wait.

11am Pacific Standard Tesla Time, the conversion to PST is iffy

The Verge says "The event was slated to start at 2PM ET (but is running a little late)". Still showing car commercial type clips.

is anyone knowledgeable with how this compares to Waymo's solutions? I'm curious.

They are fundamentally different: to summarize, Waymo follows the HD maps approach when you have a very precise map of a given city, with both metric (actual 3D shape of the environment) and semantic (lanes, sidewalks, signs) information, in which you localize yourself with centimeter-level accuracy (with GPS, SLAM etc). When you're localized you can combine information that you see now such as cars around you and high quality information coming from the map.

Tesla, on the other hand, thinks this is too fragile to changing environments and works with regular (think google maps level of detail, probably a little better) maps, combined with local semantic information such as signs and lane markings. This also means that they put less pressure on localization, because they don't use the map to detect i.e. speed limit (I'm pretty sure Waymo can detect those signs, it's just that the existence and position of the sign is already known in a global world frame).

link to the livestream video on youtube: https://www.youtube.com/watch?v=Ucp0TTmvqOE&t=4150s

I wonder how commaai is taking the news. Fleet + imitation was their angle too.

Uber and Lyft are fucked.

How so?

Ok, I am late to the show. Where can I watch the recording?

https://www.youtube.com/watch?v=tbgtGQIygZQ

It is pretty techincal. Wow. Great. Thank you Tesla.

How much b-roll do they have?

Maybe it is a test to see when the AI generating the video breaks

Did that roadster just pass two model 3s on a double yellow blind turn? Smh.... but that was awesome

Stream just went offline. Anyone have a link?

Here it is: https://www.youtube.com/watch?v=Ucp0TTmvqOE

doesn't portend well the way it is going, looks like they don't have anything to show here.

Musk: "Lidar is a fools errand"

Let's hope he is right.

"It's like having two appendices. One appendix is bad"

Reply on news.ycombinator.com