Tesla AI Day Live Stream [video]
youtube.com
youtube.com
Where do these chips get made? TSMC?
I would love to know how their new chip compares with the FSD chip in terms of power usage per teraflop.
Also, it seems that what they call "vector space" is really a 2d plane, so how do they deal with 3d scenarios such as parking garages and complex overpasses. They are rare, but they do exist.
The news will of course be all about the humanoid robot and the somewhat silly dancer, giving ammo to the detractors...
Also, did I catch it right? One of those plates is 15kW of power? That's some crazy heat dissipation!
[1] https://electrek.co/2021/05/03/how-tesla-pivoted-avoid-globa....
The whole presentation (not just dojo) felt like the kind of presentation you give internally to a VP when you haven't been making much progress. Flood them with challenges and implementation detail decisions they can't follow and don't talk about results.
That hurts me a bit, because I feel like I've done that sometimes. But that's what makes it easy for me to pick up on, when other people do that sort of stuff.
Good presentations are more direct. Ex: Here's our ResNet-50 benchmark numbers. Or at very least, "We didn't run ResNet50, because we don't think its realistic. Instead, we ran this other benchmark along with NVidia A100 GPUs and are proving that our system is faster "
----------
You need to prove your system has a benchmark score worth considering before diving into architectural details... at least if you're trying to attract engineering talent to your project.
I thought the fact they have 1,000 data labelers was interesting.
I'm unsure about their dojo efforts. Seems cool, seems better than Nvidia but not 10x better (hard to tell it's not apples to apples). For example their dojo cluster us supposed to have over an exaflop vs a A100 superpod at 700pb (seems like I/O is over 2x better which I understand conceptually why it matters but not real world implications). By the time this gets production ready (next year) seems possible Nvidia will have a replacement that is about as good or better.
Network architecture was not using video before, it was using 2-3 frames, which seemed obviously not going to work so seems better now.
The updated network architecture plus the fact they need to train with a supercomputer to me means they will almost certainly need a new inference chip. They didn't announce a v3 of their FSD computer, but that is probably the physical component blocking FSD rollout.
My take on timeline for actual compete everywhere self driving (robotaxi) seems to be: -dojo isn't even in production until next year, so no earlier than that. -FSD chip v3 probably not going to be design finalized until they use some dojo models in it. That plus chip shortages means it's unlikely to happen at scale until 2023.
So I think this presentation was to make it clear the large about of progress they are making (very impressive), but IMO this was also a subtle way of making 2023 the new target FSD date.
The alternative is they don't think they need dojo to get FSD, and if that is the case why bother building a fully integrated chip manufacturing process?
1. Training requires them to make use of ridiculous amount of data which requires large GPU compute + RAM and super high bandwidth to keep the GPUs fed at all times. Backpropagation requires a LOT of bookkeeping that adds on a lot of RAM requirement at train-time.
2. Inference is a lot easier. You just need to worry about keeping up with your 8 cameras and running them through your network. You also aren't calculating any gradients and dealing with needing to do any of the bookkeeping for backpropagation so you need a LOT less RAM
3. There are a ton of tricks you can do to make a trained network a lot more efficient for inference such as lower precision models (int8 or in their case, CFP8), network pruning, compiler tricks, etc.
It is quite impressive that they've managed to fit all this into their current V3 hardware. That being said, they are definitely starting to really hit the limits of V3 and in the long-run will likely need to upgrade the FSD hardware on newer cars. They are probably trying to avoid having to provide a retrofit for all cars in the fleet as that would be prohibitively expensive.
The demo was an obvious troll but it looks pretty exciting. I'd rather have resources put into actual innovation than learning how to increase the likelihood someone will click a button by 0.01%
I just can't imagine how Waymo or any other company can compete with the first-principles full stack engineering and scaling of Tesla, which touches virtually every important dimension of consideration. And that is today, not even a year from now.
The foundation Tesla has built and shown on AI day will progressively increase their lead in the years to come, I just don't see how anyone can compete with that level of forward thinking. To me it just seems obvious, so what am I missing about Waymo?
Tesla/musk self driving claims require extensive independent verification, as do the other fsd wannabes (they are all wannabes right now).
I still would like primary focus on highway driving and safety. That is the big efficiency win for logistics and slot easier than urban streets
Musk is a visionary that has delivered amazing promises. That doesn't mean all of his promises are amazing. Some are quite clearly BS (for example, I don't think robotaxis are impossible in general, but announcing a million of them on a 15-month time frame absolutely was).
[1] eg: https://www.engadget.com/2019-04-22-tesla-elon-musk-self-dri...
His only contributions have been early investment in Tesla and hiring the right people for SpaceX to achieve some decent cost savings and vertical landing on Earth. It's definitely not nothing, but it really shouldn't make anyone believe 99% of what he says, or consider him some kind of genius.
Apart from that, there are chances that Spaceship will be a good product, probably making it cheaper to put stuff in orbit than it is today. On the other hand, this may wildly accelerate problems with space junk, especially if the irresponsible plans for putting more than twice the amount of objects in LEO are allowed to go through.
Define "Mars colony". A thriving metropolis with millions of residents? Or a permanently crewed research station with a few dozen staff? The former is not going to happen in our lifetimes. The later isn't certain but I think it has some chance of happening. But Musk is going to try to set up the later and call it a "Mars colony", with the declared ambition of eventually developing it into the former. Aspirational naming.
> there are too little resources on Mars, and too harsh conditions, for it to make any kind of sense.
In 2021, Musk has a net worth of $170 billion. He's only 50, he could easily live for another 30 or 40 years. It is entirely plausible that by the time he dies, he will be a multi-trillionaire. To whom, or to what, is he going to leave his trillions? (Heck, forty years from now, quadrillionaires could even be a thing, and old man Musk could be one of them.) He might leave them to a charitable foundation dedicated to paying for the colonisation of Mars. If that happens – well, even if you are right that colonising Mars is pointless, if a multi-trillion, or even multi-quadrillion, dollar foundation is willing to pay for it, it may actually happen, no matter how pointless it may be.
Another possible scenario: sometime in the next few decades, the US successfully lands on Mars, and sets up a modestly sized US-led international research base. The US government continues its policy of excluding China, so China responds by landing on Mars independently and establishing their own competing Mars base. China and the US then become trapped in a new space race, in which each seeks to outdo the other with the size and capabilities of their respective Mars bases, not motivated by any practical economic or military benefits, but simply by national pride and prestige. A US-led Mars base could grow into thousands of people, and US Congress might become politically locked into paying for them out of the federal budget, even if their existence doesn't actually benefit the average US citizen in any tangible way.
(The above two scenarios are not mutually exclusive, it is possible both might happen at the same time.)
But if you take his statements as "thinking aloud", I much prefer this mode of operation. On the other end of the spectrum is Apple, with zero "thinking aloud" or sharing _anything_ until it's ready to ship. It's ok... but much less fun.
>thinking aloud
He's a CEO and is legally bound to not lie or mislead the investors though. For example Skreli is in part serving a federal sentence for lying to the investors who didn't lose any money.
We still have no laws preventing another greedy person doing the same.
They went after him for PR purposes.
Personally, I wish they left him alone, and instead changed drug law that will prevent these huge price hikes in the future.
In a weird way, lonely Hoody Boy exposed a huge flaw in the system, and we should be thanking him?
You need to create new social media accounts and try to look for information supporting the opposite idea. I'm not suggesting that Elon is totally honest about everything he says, but there is more to him than what you seem to have heard.
I'm saying this to help you, because, you have been misinformed by these feedback loop algorithms. If you keep your identity tied to this idea that Elon is a total fraud you are going to become more and more upset overtime as the evidence to the contrary accumulates.
Definitely the TSLA stock price increase is being reflected into the massive investments into AI.
Actually the best indicator of stock price increase has been if an unknown buyer comes in an buys huge amounts of OTM calls expiring that week with the hope of causing a gamma squeeze.
Watch the option flow.
You'll note also that it just hasn't been the same since Archegos imploded for example, but there is definitely another smaller buyer doing the same. It's almost certainly a group effort given the amount of money at play.