Tesla FSD Beta tried to kill me last night
electrek.co
electrek.co
The real problem is that roads are vastly unconstrained systems. An analogy is you can't take a sieve and fill all the holes in it with AI because it has no determinism and you don't know what all the constraints/holes are in the first place. Humans can actually reason in these situations fairly competently, even the stupid ones.
Give up, produce a half decent low price car with basic features that a human drives that frees us from the insane EV pricing jump and perhaps the world will be changed. Self driving cars are not important other than for a few exec egos and companies who want rid of the cost of a human workforce.
This is another fucking race to the bottom and the only innovation is getting people to think it's important.
Self-driving cars should be smart enough to say "You specified a destination downtown. I will wait 4 minutes 23 seconds before departure because we'll have a peak cohort of people driving downtown, and can maximize fuel economy by drafting and traveling as a convoy." Potentially we even get to the point where the cars couple together and form transient "trains" to to spread the wear on the drive systems.
We could even have controls like "A parade/diplomatic mission/emergency vehicle is travelling down XYZ Street; to ensure their security, a two-block radius around it been flagged as no-go for any autonomous vehicles."
He sits on the sidelines for all the billions of problems which are not heavily subsidised by the government in one way or another.
The fearless QB who only takes the field when the team has a 48 points lead and for Uncle Sam's team with uncapped salary ceiling.
A true underdog story. Move over Kurt Warner. Not.
This absurdist comparison isn't to say Musk is as bad as ISIS, but rather action shouldn't be confused with helping.
What's the argument then? Is everyone expected to wait around until public transit is fixed before they work on something else?
AVs are complementary to public transit. You're not going to take a robotaxi from SF to SJ. You might take one to the Caltrain station on either end and avoid needing a personal vehicle at all.
It would help if local leadership could say "You're building a business that has high flexibility in where you site it, so you should place it near a transport hub, rather than off in a suburb that causes a longer average commute for everyone."
Yes, you can't do this for every business-- some have natural dependencies on specific site characteristics, but those seem likely to shape the rest of the city's transport decisions (how many cities have major thoroughfares that parallel existing heavy rail, for example)
Autonomous vehicles promise to expand individual car ownership to some groups that would normally be forced to rely on public transit (people disabled in a way that prevents them from driving, or repeat drunk drivers, for example). For the able-bodied, they promise the relaxed luxury of "someone/thing else is driving and I can work/read/relax" that public transport offers, without having to actually build public transport.
This means the people with the strongest vested interests in public transit are already relatively outcast and underclass, and it will only get worse.
Make the rich and powerful have to take a bus, and suddenly it will become a significant political concern that the schedule is a loose suggestion at best, and that nobody has cleaned the bus since George VI was on the throne.
It would be interesting to expand on your last point-- an AV fleet that compliments public transit as a last-mile offering-- but it doesn't seem to be the model we're seeing sold right now. It's the same old vehicle designs clearly intended for single-family use, long trip times, and low duty cycle. I would imagine an AV system built as a "compliment" might involve small, slow vehicles, almost glorified bumper cars, with like 25km battery range, since the target application is to drive from the train station to a home/office, then return to the station for a battery swap and service. While each trip is a light task, a single vehicle might go on dozens of trips per day and as a result get a completely different wear pattern than your typical Hyundai.
From my exp autonomous driving seems entirely plausible within large cities at slower speeds that don't instantly kill you or others, and outside that just as good a lane assist + cruise control as you can get– the rest is branding FSD and walking back a claim or two around it. It all seems very malleable.
So, what speed would that be that is safe for pedestrians?
And how useful is FSD that only drives at that speed?
See where this is going?
As for instantaneous death, a collision between a person an automobile at 25 mph is enough to do serious damage to the person. What you've described is autonomous driving at a walking pace, and somehow that doesn't sound so appealing.
https://ktla.com/news/california/company-refutes-claim-that-...
Where one party is motivated by profit and the other is not, I'm inclined to believe the latter. Having seen Cruise vehicles get wedged in traffic first hand I'm inclined to believe the fire department.
> An internal San Francisco Fire Department report said a Cruise autonomous vehicle delayed an ambulance from transporting Davis to hospital, contributing to his death.
> Now, the fire chief says the report was inaccurate in a joint statement with the SFMTA in which they call Davis' death "an all-around heartbreaking and tragic incident."
> "The San Francisco Fire Chief has not attributed this pedestrian death to Cruise AVs," the statement said. "The city sends our deepest condolences to [the] family and friends of the victim."
https://sfstandard.com/2023/09/12/muni-bus-killed-san-franci...
Cost effective, I don't know what this means in this context. Development costs, rider costs? I don't see why this matters.
Safety is easy to measure. Just measure crash rates.
If your concern is can we engineer a self-driving car with acceptable safety performance, there is only one way to figure that out and it's to try to build one. You're not going to somehow prove it out on paper before trying it.
Seems like it'll be impossible for Tesla. It's less clear for others though, as some of them (Mercedes?) seem to be taking a more responsible development approach.
And taking legal liability in very narrow (essentially useless) circumstances is exactly why it’s a marketing thing. It’s to make it seem like they have an advanced system when there is no evidence of them having it or having any realistic path forward.
After reading that, it's hard to tell what Mercedes is doing. :/
Hopefully it's not just marketing bullshit, but I guess we'll all only find out over time.
If they're taking a measured approach, and it's working, then it seems like they've either bought in the expertise or managed to develop it. :)
That is my new favourite expression and describes 85% of startups.
However on new cars, £28k for a car with 149 miles range or £20k for a car with 400 mile range that the difference in price buys you 77,000 miles of fuel?
This is where you really go off the rails. You can't possibly believe this.
It's ego because it's a gold rush revenue source if it plays off and various CEOs have been pushing because while stock price is rising for the bubble, they're earning.
Self driving cars could potentially end car accidents outright, saving 40,000 lives in just the US per year. But that would put some mechanics out of business.
If we (tech fans) really wanted to help alleviate the problems roads have (traffic, accidents, etc) - public mass transit is the way to go.
But I guess .. where’s the immediate hype-based profit in that? (think: “omg blockchain… I mean nft… I mean ai!!1!!”)
It is freedom. Life is "energy wasting" too, but don't let that get in the way of ideological virtue-signaling...
Those working in the mass transit industry call passengers "cattle" for a reason.
Roads may have problems but the alternative you're proposing is far worse.
Yes, one can get into the car and drive and live “the way of the road” like Ricky’s dad on TPB but again:
How is that freedom?
“Freedom” is american marketing.
It is freedom
Sorry, but "far worse" is simply excessive.
Living in a large European city, namely Paris, and having experienced both car and mass transit, I prefer the second option a lot, to the point I sold my car without replacement about 6 years ago after 2 years of barely using it.
There is of course the ecological aspects, but even in terms of quality of life, I much prefer public transports: it's cheaper, it's less stressful, it's more predictable and I'm able to do other things like reading while transiting.
By comparison, in this context (dense urban area), cars, with their needs (maintenance, parking, etc) are actually more constrain than freedom.
And that's not like I'm a unique case, most of my colleagues either use public transports or (e)bikes, car users are a minority (~20%).
In Paris, there is even a large proportion of the population who didn't even have a driving license at all.
Doesn't it means that mass transit is the perfect solution? no. But in dense urban area, it's probably the best compromise.
I think mostly people are talking at cross purposes though. I too live in a large European city and getting around by car is incredibly constraining (traffic, parking, other drivers etc) while using the public transport system, while not perfect, is smooth in comparison.
However, for less dense areas, eg going on holiday somewhere in the countryside, it is much more convenient to have a car and I suspect that many people are coming from this angle. In those circumstances, I will rent a car as necessary. It doesn’t happen often so I still save in comparison with the cost of owning a car.
Living in sparse suburbs is probably hell without a car. This is partly because of poor town planning but also, I imagine, because a certain set of people want to have that much space. The trouble is that the externalities of the car travel are generally concentrated elsewhere, so it’s a bit unfair.
I don't see how it's pretend. Sure its not true freedom in the sense there's no cost or rules, but cars allow for spontaneity and a way to easily go off the beaten track.
Cars are liberating in the same way a horse would have been hundreds of years ago. A car allows me to go where I want, when I want and that's worth any cost.
They make the average person's life better and I wouldn't want to live in a world where I couldn't drive.
Currently, Americans spend about 100 billion hours driving every year, which is about 11 million man-years.
If you can't see the transformative impacts of freeing up that much time for people to enjoy any other activities, whether it be doing work, reading a book, sleeping, or eating chicken nuggets, then I honestly have no idea what to say.
I just found it somewhat ironic that instead of considering alternatives, people would rather hold out for self driving cars, which may never happen, to reduce time wasted in driving.
Before the damn'demic had the thought that the reason people commute long distance is partly because company VIP's all like to live a) close to work. b) next to each other. Which due to suburbanization forces everyone else to commute. This seems like an incredible waste of societies resources whose costs are born by poor and middle class people. I think the whole work from home thing is a rebellion against that basic unfairness.
If you think about it getting misted because your car plows into oncoming traffic is actually not that bad because some day others will be able to pay $15,000 for DLC in their car that unlocks “road head”. It’s basically on the same level as antibiotics or the Haber-Bosch process in its necessity for the flourishing of mankind
Somehow, when we build a $2 billion Central Subway in San Francisco, through some of the very densest neighborhoods in all of America, with a realized daily ridership of ~3,000, there are no jokes about how unimportant the train is compared to fertilizer.
Far too many people are so selfish that they get hung up on questions like “Could this cause death or grevious bodily harm to me or others on the road?” and do not have the ethical fortitude or wherewithal to focus on “What could the abstract concept of ‘transformative’ mean at an unknown point in the future?”
> there are no jokes about how unimportant the train is compared to fertilizer.
This is a good point. There is basically no meaningful difference between a self driving Tesla and a train. The only explanation for why you don’t hear people discussing the myriad times that commuter trains veer into traffic or otherwise behave in dangerous and unpredictable ways must be collective ignorance and hypocrisy.
I'll happily admit there are problems with autonomous vehicles today, and that is not what I dispute. What I dispute is that autonomous vehicles are not important. And of course they are. They would free up more human time than any other invention on the horizon.
However, to the non sequitur point about safety, we are not designing a system from scratch. The existing system has two million crashes per year in the US alone. If the new system has one million crashes, that's a massive improvement.
> There is basically no meaningful difference between a self driving Tesla and a train. The only explanation for why you don’t hear people discussing the myriad times that commuter trains veer into traffic or otherwise behave in dangerous and unpredictable ways must be collective ignorance and hypocrisy.
Once again, you've quickly switched the point to something different. The question was if an astronomically expensive train that carries a mere 3,000 riders a day is also worthy of derision, since few people use it, and all it does is replace walking, freeing them up to play Wordle during their commute.
I also liked how you brought up trains and then said I’ve switched the point when I responded to your point about a train.
I concede that your point about San Francisco’s train ridership is unassailable proof that self driving cars are as important as water treatment and germ theory. Since it is the only train in the world and has less than stellar ridership, it is proof positive that the very concept is worse than a car that drives itself into oncoming traffic.
It is a shame that the author of the article did not take a moment to consider ridership numbers of the San Francisco rail project as his car was attempting a u turn on the freeway. A small minded person could find themself thinking about impending death and not specific train projects worthy of derision.
The main problem is lack of restraint: "Give me $15,000 non-refundable and someday I will deliver the means to make a profit on the car you bought. It already has the hardware." is building up a big potential liability. But, even if forced to refund that FSD money, it's still been cheap capital to grow Tesla.
Besides the fact that the next OTA update may well give you a completely different batch of problems.
I don’t care for arguments about safety statistics or the ends justifying the means. My kids are on these streets. It’s not your autonomy test track.
>I don’t care for arguments about safety statistics
is insane. If it's safer it's safer, and the only way to know is statistics.
The fact that (seemingly) no government department is collecting and publishing safety statistics on every self driving car manufacturer defies belief.
But I think what you’re talking about is if autonomy makes my kids safer in the immediate, and not as an end result of public testing. Yes, I support that. I don’t think this is that yet. The word beta is right there.
If FSD has more accidents then ban it.
You can do something wrong but if it works out, hey, it’s fine right?
This also creates a wonderfully sloppy veil. Hey it’s proven safe. So we’re going to go ahead and fuck with the settings more.”
This experiment is sure to be unbiased.
The claim here is that an overnight minor update from v11.4.4 to v11.4.7 caused a significant regression.
I’d rather have distracted driver on autopilot over distracted driver in a car with crude ADAS.
(There are Autopilot/ADAS related deaths, though not a shocking number relatively, but that is not the same type of system and is also deployed on many other vehicles.)
I'm saying that there are very little of perfect drivers. Even without phone and alcohol, which is a criminal negligence - there's kids, partners, mind wandering, daydreaming, poor vehicle maintenance, crap road designs or surfaces, medical conditions, vision impairments, weather, food, insects, etc which on overall means we are generally very distracted and impaired.
Yes. Many people who've been riding motorcycles more than about 6 months or so (aka and lived), have learned to. A few years on from that, defensive^W paranoid riding is well ingrained.
What really gets me is that people would then not return their vehicle as defective (or never ever again use such a feature). I sold my previous car after it twice false triggered on the auto emergency braking failure and I would expect other car owners to do the same thing.
> half a lane wide sudden moves
In 10k autopilot km's I've never seen that. Yes there is some breaking sometimes, but no swerves yet.
And yes panic breaking pisses me off and lane departure alarms almost caused to rage and return the car.
That includes those of us who worked on safety critical engineering years ago and know exactly what the fuck we are talking about.
FTFY. We're socially conditioned to ignore these risks, but it's still objectively insane that I can easily kill myself, my entire family, and another group of innocent bystanders by rotating a steering wheel a few inches in the wrong direction.
(I still don't and would never use Tesla FSD, though.)
So maybe we should worry more about the average American?
No, I don't believe any of those p-hacked/biased stats that conveniently portray something, that is sold as BETA SOFTWARE by a company with a multi-billion market cap and a world-class marketing division, to be "safer" than an average driver.
FWIW I'm also not thrilled with the idea of the average person being permitted to get behind the wheel of a 5 tonne death machine, either.
> So maybe we should worry more about the average American?
As an outsider (Australian) watching the American empire decline so rapidly in recent years, I assure you I'm also quite worried about America, and the average American.
Say for example a utility company pushed out an update that flipped the power contactor between you and the grid too many times that it caught fire, the company would be taken to court.
I simply just don't understand how Tesla can get away with it.
(And that's not even counting the "Elon Mode" we've since learned is hidden in the firmware.)
The "Elon Mode" thing is an internal-only flag that nobody has been able to activate outside Tesla, and is not actually called that it's just what some guy on Twitter called it.
The other factor is that human attention may be needed in a situation where it is very hard to actively fix the situation if you're not already in the OODA loop. If you suddenly realize you need to override the car, and you're in the middle of a complex merge and the reason you need to override the car is because someone else screwed up, it may not be possible to solve the problem with the extra cost of your attention spinning up.
And the empirical data available (which could definitely be higher fidelity, granted) appears to suggest that the actual safety record is quite good.
So we're left with a trade-off: there are some scenarios where a more featureful system could elevate risk, but there are other scenarios where it lowers them. It's not obvious on paper how to balance them, and the empirical (and, for me, experiential) data indicates the trade-off is a reasonable one.
Perhaps a large amount of the concern around this comes from people not being aware of what it's like to operate a car in this mode. Maybe people think you really don't have to pay attention? It knows when you've got hands on the wheel, it knows when you're looking at your watch or your phone or the car's display, and it will pretty aggressively prevent you from getting complacent. Not only will it beep, it might lock you out from using it entirely. (Lockouts never happens to me, but I have seen tweets from it happening to others.) There was an old pre-Waymo Google talk about how partial driving assistance was dangerous for this reason, and I bought that argument but I no longer find it convincing having used such a system a lot.
Yet it doesn't disengage when you take your hands off the wheel
> What makes it different from lane-keeping assist in other vehicles?
lane assist, assists you in keeping in the lane. It doesn't pretend to be able to drive for you. It also, as far as I can see, doesn't attempt to fling you into cars you're overtaking. (unless you don't use your indicators....)
> bog-standard cruise control?
That again isn't pretending to drive for you. The instructions are generally very clear, like on my car it says in huge text "this aid will not break for you, you must maintain distance as you normally would". Also, it doesn't attempt to steer you into a central reservation. There are shit cruise controls out there, like on the VW golf. That stops you if you attempt to drive past much slower traffic.
> we accept that it is the driver's responsibility to not let that happen.
Yes, because they have been told in no uncertain terms that if you obliterate someone in your car, its your fault and you're going to jail. More over, you were the one driving, so its down to you to not do it. Tesla are shipping untested systems and telling you, as an untrained roboticist supervisor: "don't worry, its safe, it drives it's self!" give you the impression that you don't really need to pay that much attention.
> What is the difference here other than the system is _less likely_ to kill you because it is smarter?
Its not that system. I will fully endorse a safe, tested, verifiable level 4 autonomous driving system. I will not endorse a barely credible level 2 system who's QA system involves YOLO deploying shit to unqualified customers.
Well you don't want it to immediately disengage, which would be insanely dangerous, but it knows if your hands are on the wheel. It will progressively alert you more prominently if it hasn't felt wheel torque from your hands. It will then begin beeping loudly and stop and lock you out from using Autopilot. It also knows whether you are looking at the road from an internal camera and will beep if you are looking at your phone, watch, or the car's screen. This system works quite well.
> lane assist, assists you in keeping in the lane. It doesn't pretend to be able to drive for you. It also, as far as I can see, doesn't attempt to fling you into cars you're overtaking. (unless you don't use your indicators....)
"Pretending to drive for you" is meaningless. It's a feature that takes over driving functions that requires active supervision and that if you do not pay attention can cause a crash. The conditions of how it might cause that crash are not particularly important conceptually. What matters is that there is an expectation you pay attention or you might crash. The primary difference is cruise control is worse at driving.
> Yes, because they have been told in no uncertain terms that if you obliterate someone in your car, its your fault and you're going to jail. More over, you were the one driving, so its down to you to not do it. Tesla are shipping untested systems and telling you, as an untrained roboticist supervisor: "don't worry, its safe, it drives it's self!" give you the impression that you don't really need to pay that much attention.
This is also the case with FSD. You're told its your fault. You're told it's your responsibility. They do not say "don't worry it drives itself" or give you the impression that you don't really need to pay that much attention. In fact it says the following, among other things you must explicitly accept, enable, and agree to:
> Full Self-Driving is in early limited access Beta and must be used with additional caution. It may do the wrong thing at the worst time, so you must always keep your hands on the wheel and pay extra attention to the road. Do not become complacent. When FSD is enabled your Model 3 will make lane changes off highway, select forks to follow your navigation route, navigate around other vehicles and objects, and make left and right turns. Use FSD in limited Beta only if you will pay constant attention to the road, and be prepared to act immediately, especially around blind corners, crossing intersections, and in narrow driving situations. Cabin Camera is used to detect driver attentiveness while FSD Beta is in-use to provide you with audible alerts, reminding you to keep your eyes on the road.
> I will not endorse a barely credible level 2 system who's QA system involves YOLO deploying shit to unqualified customers.
I'm not looking for your endorsement, I was trying to explain why the system is legal and probably less dangerous than you think. Also obviously they QA it before it goes out to "unqualified" customers.
Waymo, on the other hand, is actually very good. Waymo is (IMO) actually a bit better/safer than the average human driver, Tesla FSD is way, WAYYYYYYY worse than a human.
I’ve spoken with a number of ppl who’ve taken rides in both Tesla FSD and Waymo, and the reviews are unanimous. Tesla FSD makes major mistakes very regularly, you have to be taking control a lot, it’s a terrifying experience. Waymo actually drives safely and correctly, in a confidence inspiring way, with no need for human intervention.
People even compete to be allowed to perform free beta testing. Great from a business perspective, saves a lot of money on QA.
I prefer to use the software once it has been fully released and has been out for a while for the vendor to iron out any major issues.
To freely chose to beta test the self-driving feature of a huge automobile travelling at high speed on public roads (not a test track) is nearly impossible for me to understand.
Where it the upside to it? Being able to blog about how you nearly died?
Had it gone a bit farther this would have been a good candidate for a Darwin award.
Why - in this highly safety critical domain - is there ever anything close to “beta” software accessible to end-users? What about all other participants in traffic? This “perpetual beta” approach hasn’t been industry best practice for most of its existence.
There used to be years long development and testing procedures, all happening outside of common traffic.
This is all a big marketing racket attempted to as “efficiently as possible” transfer most of risks and costs on masterfully deceived consumers and the ever unquestioned public.
For this reason, I always stay clear of Teslas on the road.
My old Honda hasn't tried to kill me yet. I suppose the jury is out on the takata airbag recall but that's fixed now and our relationship can continue with a pristine record of zero murder attempts.
You could get rid of the car...
“This just tried to kill me. Great, better try it again.”
I mean, this seems horribly irresponsible?
The government needs to give Tesla a week to remove all Autopilot and FSD terms from their products or recall the entirety of the product line. People who paid extra for this unlock should get full refunds for it.
My Kia can provide an amount of self-steering in cruise control which is truly impressive and smoothes the driving experience, but nobody sells it as driving itself, much less charges an extra ten grand for the privilege.
True accountability should hone in on the “natural person”. This needs to change.
Also, the time it take to charge your EV compared to filling your gas car is silly after spending $50k to $200k!.
Also, Musk is very capabale of doing stupid things. Especially when it comes to social media likes. I mean he spent 44 billion for that same reason...
https://www.cnn.com/2023/01/17/tech/tesla-self-driving-video...
What are lawsuits and fines to someone who buys 40B companies out of boredom / like we comment on HN.
Responsibility and repercussions must become much clearer and more sharply felt to any so called “leadership”.
This is one, if not the main malaise of our times.
Like, wouldn't be too many deaths and accidents set back the goal? And wouldn't a previously acquired bad reputation slow down adoption when it's finally working?
And why not sacrifice short term gains by having say, LIDAR and better hardware at the start, to have the best performing system as fast as possible, and then gradually trim things down once everything works perfectly?
Regulation is a trailing thing, we need people not to be default evil for us to have a society.
My long term rental Nissan Rogue lane following system tries to kill me multiple times a day, and only disconnects with an alarm 2seconds+ after leaving the lanes.
Wonder if Tesla could somehow allow to drive with less steering wheel resistance but also autosteer reliably.
I just stopped using it.
Mercedes brought out their first actual self-driving car a few months ago. Unlike driver assistance, where the driver is expected to retain control at all times, it will give at least ten seconds notice -- so you're allowed to take your hands off the wheel, read a book, or watch a video. But it will only let you turn it on in very restricted circumstances.