Hacker News Reader

Comment by TaylorAlexander | Hacker News Reader

As a robotics engineer (though I don’t work on AI systems) I was bullish on Tesla’s architecture for a long time. I’ve watched all their presentations. Their hydra network. Their massive custom AI chip. I really thought they had a chance at making it all work. But the long tail is killing them. They’re walking back their claims for fully autonomous operation. Andrej Karpathy (their AI lead) left. Musk is distracted and showing that he doesn’t have all the technical chops we thought he did (if reports from inside twitter are to be believed). It feels like the company is all promises and no delivery now. Many have been saying this for years, but I really thought they were going to pull it off. Now I’m beginning to feel that they’re not organized appropriately to do it. You just can’t have a single guy who is all ego running a project as complex as this and find success. I think of how they must be doing things at Waymo or Cruise and they’re going to have a team of competent people in charge who don’t make rash decisions on major architectural issues. And given the scope of this problem I’m really starting to believe that’s what’s needed to make this work. There’s just no silver bullet. I’m really questioning what led me to believe in all this in the first place.

View on HN

Tesla's approach has always been the least credible of the self-driving companies: they seemed to belive that if you just iterate on driver assist enough you get self-driving, and they seemed to want to make the system work on far more constrained hardware than their competition (because they were for some reason aiming at selling direct to consumers, which is not a particularly good business strategy for self-driving. A cynic would suggest it was vaporware for marketing all along). Other groups working on the problem (especially Waymo) seemed to actually grok that this is by far the most complex safety-critical system ever envisioned and approached the problem accordingly (with a lot of caution, laying of groundwork, a plan to roll out the system gradually, and bulding the it from the ground up to work on its own, not as a driver assist).

When Tesla got rid of radar and switched to "vision only" I cynically assumed it was a way to compensate for supply chain issues rather than radar being as useless as Musk was saying. I still think this is a possibility.

It's also possible their radar was functionally useless: while it gave good range information it had poor angular resolution, a fact which seemed to contribute to Autopilot's issue with recognising stationary obstacles. it's worth keeping in mind that LIDAR gives much higher quality data than low-cost radar. (It also doesn't follow that if their radar was useless that vision alone would be good enough)

That might have made more sense to me if they hadn't followed it up by removing their ultrasonic sensors, which were functionally quite useful, since without them the car cannot see things at low speeds that are below the plane visible from the front-facing camera mounted opposite the rear view mirror.

Except cameras don't work in low-vis situations. Their range is shorter.

In fact, when Tesla removed the radar their max-speed with self-driving (or what is it called) dropped from 80mph to 75mph.

What's worse is that they disabled the radar for everybody. How is that not a class action? You're losing features.

The argument from Andrej Karpathy is that humans use vision and vision alone, so if we can learn to drive by vision, then so should an ML model. A radar is a sensor a human doesn't have, but we can gauge distance, and they also do introduce potential supply chain issues. So limiting the number of sensors is beneficial there. He also makes the argument that you have a tonne of data already. 8 1.2mp cameras, so 9.6 million pixels or effectively 9.6 million separate data points used to train the model. When you have that much data, what does including some relatively low resolution radar information give you that you couldn't infer from vision?

I don't know enough about AI to know if he's right, but it makes some sense to me. I think my biggest issue with it is that human eyes are insanely high quality, able to adapt to huge changes in light levels and clean themselves every few seconds. I know Teslas have issues with cameras being blocked by sunlight and things like that. A human can just move their head and use the sun visor.

With my limited knowledge I think vision probably is the ultimate way forward, but maybe they're too early. Perhaps there's some breakthrough we need to simplify the problem. I don't think Tesla will be the first to start making money from autonomous vehicles. I think Waymo will be the first. LIDAR isn't scalable but it simplifies a big part of the problem to the point that you can generate revenue from autonomous taxis while you train and perfect the general purpose vision model in the long term.

It doesn't make sense to me at all.

We have an iris, this allows us to limit the amount of light hitting our retina, or increase it when conditions are darker. Do the cameras on a Tesla have shutters that can do this? Otherwise you're going to have fun when there is glare off a wet road.

Did you even read my comment?

The issue is that self driving needs to be better than human drivers by a considerable margin. The public will not tolerate an AI driver that makes the same mistakes as humans, they expect better. To do this it will need access to sensors that humans don't have

A good human driver won't cause an accident though. The majority of accidents are caused through distraction or inhibition. A car can't get distracted or drunk.

In such a safety-critical system, it's important to have backups. You need at least two cameras to accurately estimate depth, and cameras can fail for a variety of reasons (sun glare, low lighting, heavy fog). Radar, at the very least, is a backup for the cameras. Also, with vision, the best you can do is estimate distance, whereas with radar and LIDAR you are explicitly measuring it.

>Also, with vision, the best you can do is estimate distance, whereas with radar and LIDAR you are explicitly measuring it.

But is there any evidence you need to measure distances? We humans can navigate the world without walking into walls, so long as we're looking where we're going. For a machine to navigate the world it should be possible to do it via vision. And Tesla's do have multiple cameras to be able to measure depth.

And radar is not a backup for cameras. The resolution of the data is terrible and you can not rely on it to do any sort of driving except braking if it thinks there's an obstacle. Radar is also susceptible to problems as well, which is why Tesla's and other cars with radar can often go crazy thinking you're gonna crash randomly.

> The argument from Andrej Karpathy is that humans use vision and vision alone, so if we can learn to drive by vision, then so should an ML model.

Humans also learn social dynamics through numerous other senses and in situations that Tesla's models never encounter or code for. Those other cars on the road do have human drivers, after all.

Humans also have ears, etc.

Plenty of unusual road or lot situations where those things come in handy.

>The argument from Andrej Karpathy is that humans use vision and vision alone, so if we can learn to drive by vision, then so should an ML model.

Yes ... and? Is this guy like an oracle with all the right answers wrt to AI?

You're also appealing to the authority of a guy that didn't deliver much in 4 years and then just left the company. Then after he leaves Tesla brings back the radars lol, but that's definitely a coincidence, right? Because he's Andrek Karpathy, and he was definitely not wrong about that, at all.

I'm not appealing to anyone. I'm just stating his argument since he was one of the main guys at Tesla working on this problem.

He might have a point, he might not. I stated the obvious problems with it in my original comment.

If you limit yourself to the equipment that humans have, how do you expect to outperform humans? Would we accept 40k deaths every year in the US from self-driving cars, too?

For that matter, I don't think the cameras installed in Teslas work as well as a good human eye, with dynamic range and depth perception.

So it seems like a bad premise to start with, and the proof of the pudding is in the eating, so to speak. So many years in, and when they pulled the radar out for supply chain reasons they had to disable self-parking, too.

The majority of road accidents are caused by distractions and inhibition. If you perform like the best drivers on the road, then you won't have accidents.

That last statement is an assertion lacking evidence, and as counter-evidence I offer every accident Tesla’s “autopilot” has been involved in.

If drivers always paid good attention, I’m sure we would have fewer collisions. I’m not sure we would have none.

> The argument from Andrej Karpathy is that humans use vision and vision alone, so if we can learn to drive by vision, then so should an ML model.

By that argument, we should abandon silicon for our chips, because we think without it. Heck, by that argument, we should be building androids and putting them into the driver's seat, because humans use hands to drive.

What an incredibly stupid argument. My respect for this guy just went right into the toilet.

>By that argument, we should abandon silicon for our chips, because we think without it

Evolution has created some of the greatest computational machines. If it was possible to build a programmable biological brain, we'd do it in a heartbeat if that's what it took to develop true artificial general intelligence.

>Heck, by that argument, we should be building androids and putting them into the driver's seat, because humans use hands to drive

Humans don't need to use hands to drive though. We could input steering and throttle controls though many different ways. We have vehicles that are controlled by tilting our bodies in different directions. But the fact of that matter is, that we use vision to see where we're going and the infrastructure is designed for vision.

I'm not sure what your point is, it doesn't seem like you understand his.

It is certainly possible to make a self-driving system with vision alone.

But it is also extremely likely that this is much harder than doing it with radar and lidar added to vision. Since nobody knows how to do any of them yet, you can't expect the company trying to solve the much harder problem to be one of the first to get it.

Their communication is a constant denial of this fact. So they either have no idea what they are doing, or are lying through their teeth.

I think it sounds like Musk looked at humans and went "we do fine with just vision, so make the computer do it with just vision, that has to be good enough"

Which underappreciates the fact that it is such a hard problem (nature had 500M years to iterate on the reference design) that you want to cheat as hard as possible by giving yourself the best sensor information you can.

Automatic cars need to be better drivers than us, not just good enough, as another thing.

If they were as good as the average sober and undistracted human they'd probably be a massive improvement.

But yeah, they need to hit the highest bar possible. As we've seen with vaccines anything developed by humans has to be nearly perfect, while a "natural" massive clusterfuck with 40,000 deaths per year won't count for anything by comparison.

> If they were as good as the average sober and undistracted human they'd probably be a massive improvement.

How do you convince the better half of drivers that they should use a system that is worse than themselves, personally?

If you had a system that drove roughly as well as I do at my 90% best and cut out the worst 10% of my driving, and let me not have to deal with the road that'd be fine for me.

And I probably should have said "sober, undistracted and competent human" since some people just shouldn't drive, and some people shouldn't be driving under certain conditions (which may not be tired or drunk, but going through a rough break up with late night emotional blowups).

Everybody got bullish on this when Google first publicly presented their autonomous car and everybody from IT said: 10 years from now.

Except all the automotive people said: Nay, way to complex to solve.

The automotive guys worked since the 1960s/1970s on this. Yes it got boosted by the advances in AI, which where mainly driven by the now available computing power/memory. But AI and its limitation is known since the 1980s/1990s. I learned AI in hte mid 1990s in the university as part of my degree as an electrical engineer.

Now, we are at a point where even the Waymo guys (former Google Car) admit, that the fully self driving car will be years away. It will take much more years then everybody expected. The cars the Waymo runs, are running in very dedicated areas, which are highly mapped. They still have problems with temporay obstacles, like road construction. They still have problems in unexpected weather situations.

So it is, that even all the tech people who believed in the sayings from Elon, now realizes that its just vaporware.

It's not vaporware, it's just taken significantly longer than expected.

Tesla's solution looks more like vaporware because of how many times Musk has promised fully driverless operation -- sometimes even promising a cross country hands off trip -- within a year or two. Other companies have also had overly optimistic predictions, but generally not "we'll be able to go anywhere in the country with no intervention within the next year" level of extreme.

Waymo's progress has been slow relative to "ten years" predictions, but they've also made steady progress, and it looks like it's continuing.

Tesla is also selling "full self driving" as a feature, right now. Not "coming in a year or two" They are taking people's money for it right now, and have been for years.

It's not vaporware, it's just taken significantly longer than expected.

EDIT: nevermind, parent clarified. Leaving original comment so follow-ups make sense.

"Taking longer...": what we call it when it doesn't work, despite all of the marketing. So, IOW, "vaporware".

When Tesla ships a working product, where "working" something remotely close to what has been promised, we can call it something else.

Sorry, I interpreted GP as saying that self driving in general is vaporware. If they meant specifically just Tesla is vaporware, then yeah I largely agree.

Oh, okay, I get ya. Yeah, I could see self-driving being a thing...some day, and wouldn't put the whole thing in the vaporware bucket. Thanks for the clarification.

CPU/GPU processing power is clearly making steady progress and looks like it will continue. Yet nobody thinks we will be able to brute-force 2048-bit RSA with the best sieve techniques even if the progress continues for 100 years.

It is clearly not sufficient just to have some measure of growth to achieve some goal. It is necessary to have the growth actually matter compared to a measure of the problem difficulty. I would argue for self-driving cars as in Level 4/5 autonomy, nobody even has any useful measure or understanding of the problem difficulty.

It's really easy to understand the autonomous driving levels. It's also really easy to understand the Kardashev scale. It might just turn out that both are of equally little practical relevance in our lifetimes.

I know this is a boring situation, but there is no guarantee the universe shouldn't be boring.

Waymo has an operational SDC taxi service in Phoenix with no safety drivers. Obviously that's a fairly easy metro as far as driving goes, but it's enough progress to be long term optimistic imo, especially considering they're doing a closed beta in SF currently.

Robotaxis are on the road today in multiple cities. That's nothing to sneeze at, large fleets of totally autonomous vehicles in complex city environments is something something roboticists have never done before!

Those environments have been mapped out to a very high level of detail. It's a similar "long-tail" problem — all very well and good to have self-driving cars running around downtown SF after spending tons of effort and time creating a virtual map of the city, but it's quite something else to get general self-driving that will work across the entire country.

I don’t think doing it for the entire country is even a goal for these AV only companies. For trucking theyll HD map the highways. For city center taxi service HD maps and remote assistance are fine.

Cruise went from nothing to a public driverless service in Austin in 90 days, and that involves a LOT more than just mapping the area. Waymo had a comparable timeline for expanding to downtown Phoenix from the suburbs. Mapping is not the albatross you think it is. It's essentially just Google Maps/Streetview (which covers the whole world) with extra sensors and larger hard drives. A problem that's been solved for over 15 years already.

Did you read and understood what I wrote: "The cars the Waymo runs, are running in very dedicated areas, which are highly mapped."

I simply don't see the problem with areas being highly mapped. For the record, operational areas can be mapped very quickly now - I think I read a Waymo presser that said they mapped their LA operational area from 0 in a few weeks.

    Musk is distracted and showing that he 
    doesn’t have all the technical chops we 
    thought he did (if reports from inside 
    twitter are to be believed)

Who thought he had technical chops? I don't mean this as snarky or glib. But it was surprising to read.

The rosiest take is that he's a Steve Jobs type as opposed to a Bill Gates type. A guy who assembles and rallies teams of engineers and designers around a singular vision and often manages to pull, push, or drag them across the finish line.

I'm not a fan of Musk in particular, although I am also not demeaning this sort of CEO. The ability to form/lead/motivate teams that execute on this level doesn't exactly grow on trees. And it's all but impossible for a CEO to have deep, current "technical chops."

It's just that I've never actually heard Musk credited with having technical chops.

Musk regularly repeats that he is an "engineer, not a businessman"

This video edits together examples from various interviews: https://youtu.be/e7ez_WF40hY

Ah, okay. That explains some things. He's the one saying it, and I guess some folks bought into it and believed him.

> Who thought he had technical chops?

Many, many people liked Musk, or called him more credible, for being the one CEO who also knows engineering. It also made him aspirational or a role model for many engineering types. It's a bit like when people mention that Angela Merkel has a PhD in quantum chemistry, it confers legitimacy in context.

The difference, of course, is that Merkel also actually has her degree.

> But the long tail is killing them.

I imagine their engineers have said they’re “90% finished” once or twice in the last few years…

Always a red flag!

To be fair the last 10% of any project is generally 90% of the work. So saying you are 90% done yet being far off is truthful, in a sense.

Is this a Zeno's paradox, where you can never actually finish?

Have heard a twist/joke on this where an engineer says something like "i'll never get all the way there but close enough for it to work"

Its dishonest to not scale by effort required when reporting project progress.

As a leader of software teams at various points of my career, I have to completely reject this notion.

In a poorly managed project, yes, the last 10% of a project is 90% of the work. A healthy project puts 90% of the work at the front: the work of planning, narrowing scope, thinking creatively about what could go wrong, leaving oneself off-ramps in case of unexpected situations. The speed or cadence of a project should only increase as you near release; momentum should build. There should be a natural period of "dust settling" before production, rather than scrambling to pull the system into a healthy state.

If an engineer repeatedly says they're "close," but cannot quantify such a statement, I find they're (generally speaking) completely full of it.

Perhaps Tesla's situation is unique (or perhaps not). Musk has repeatedly said it's difficult to gauge completion date because no one has built this software before, so there is no benchmark. Having just sold my Tesla in favor of a competitor, I suppose I won't be there for the release anyway!

I think it depends on the project. 90% of SWEs are using pre-packaged databases, analytics, logging, APIs and whatnot. There is no 10% or 1% tail bogging them down because they're just configuring tools. But if you're in the business of innovation, it's really hard to know what lies in the darkness ahead, and it it very common to attack the easiest parts first, and what you thought was a 1% that would take up half the time is a 1% that cannot be completed ever.

I think the point is that if that final 1% is essential to the project then the project was poorly planned (like the comment you replied to said). Why spend untold time tinkering on the fun easy stuff ignoring a 1% that could kill the whole thing? If that 1% isn’t essential then you should just not do it — and if you don’t make that decision then the project is also poorly planned.

I think if every single great discovery in the course of humanity had to have been "well planned," they would have never happened. Like I said, most of the "disruption" and "innovation" happening in the startup ecosystem is far from. And projects of that scale involve little to no unknowns so long as a properly experienced team is attacking them.

But when you're talking about AV or fusion power (or even much smaller discoveries that sit on the vanguard of human knowledge), it requires the kind of leap into the unknown that all great discoveries have necessitated.

> To be fair the last 10% of any project is generally 90% of the work.

No, that means a pathologically badly planned project. If some part of your estimates are off by an order of magnitude, you need to learn to do a lot better at estimation.

The thing is human intelligence shines in the long tail and stochastic parrots thrash and die there.

This is absolutely correct and I'm not sure why people are downvoting you, unless they're not familiar with the stochastic parrots paper. https://dl.acm.org/doi/10.1145/3442188.3445922

> I'm not sure why people are downvoting you

I am not sure too. I also want AGI to happen but we need to acknowledge the limitations of current formalisms and I think people are married to formalisms.

Isn´t what they have right now safer than human driving?

This is the type of question that will start arguments on both sides, but the truth is no one outside Tesla could possibly have the data to say one way or the other.

Safer than human driving isn't the bar that needs to be cleared. I consider myself a much safer driver than the average human (which also includes 70 year olds with driver's licenses) and I don't drive when I'm sleepy or drunk or even a little impaired. Even if the claim that it's "safer than human driving" is true, I would only ever use it if it was safer than my driving.

Fun fact: most people believe they're a safer than the average driver. Many people even believe this while driving drunk. Unless we start putting interlocks on all cars (which isn't entirely hypothetical[0]), they're just going to keep driving around drunk. Would you rather they drive drunk, or have an imperfect system that's better than drunk them, driving a car that also has an AEB (Automatic Emergency Braking) system?

[0] https://www.motortrend.com/news/anti-drunk-driving-technolog...

Practically speaking, even if self-driving technology was widely available today, it would take decades for manual cars to get off the road.

I know I am not as good as the average driver, simply because I have much less practice, but I make up for it by extreme vigilance and caution, and I've never been in any sort of accident or moving violation.

I think the world would be a better place if most people believed they were bad drivers like I do, and were much more vigilant and less aggressive.

And of course the pragmatic reality is that nearly everybody considers themselves "better than average."

Tesla isn't merely competing with the actual average human driver, they're competing with the average human driver's unrealistically rosy perception of their own driving prowess.

It's not necessarily unrealistic. It only takes one terrible driver to drag down the mean and make everyone else better than average!

A good example of how nearly everyone can be above average is number of arms. If, like 99+% of the population, you have two arms then you have an above average number of arms.

Yeah, no.

Driving skill is almost certainly normally distributed, and if so, your answer is wrong.

From personal experience, I can say what they have now drives worse than a drunk teenager.

They haven't demonstrated that in a like-for-like comparison. Tesla's autopilot, in circumstances where it can be engaged, and with human supervision, has more miles between accidents than average human drivers in all circumstances (such as older cars, worse roads, and worse weather). But in the circumstances where Autopilot can be used humans are also much better than that average.

From watching a number of videos, I'm also under the impression that Autopilot is currently still so unreliable that the average beta tester of the system is overall _more_ attentive (and often more stressed) to the road than they would otherwise be, so they can intervene in questionable cases or even just override otherwise embarrassing road maneuvers.

Given that the typical Autopilot ride has multiple disengagements that also makes it really hard to judge the performance of the system properly - if you always have to have a hyper-ready human waiting in the wings to catch mistakes to produce nice stats, that's certainly still far away from what most people want in such a system. You'd need to demonstrate better safety without disengagements and without a human backup, as the human drivers you compare to don't have one, either.

Marques Brownlee demonstrated this nicely in a recent video review of his daily work commute using Autopilot.

I believe you're confusing autopilot with FSD beta.

I did, even though the video I cited taught me the difference. Thanks. :-)

Hard to say. Are there published comparisons to affluent humans driving relatively new cars manually?

What they have now is a small subset of "driving" so not really useful to compare as it's misleading at best. Otherwise you could say student drivers are the best drivers since they almost never cause accidents.

Downvoted for doing a question that nobody denies. I love Hacker News these days.

Reply on news.ycombinator.com