The excuse is that this firmware version is not available to the public. Yet Elon is driving in public with other drivers putting everyone at risk of death. At what point are we going to stop this lunacy?
The left turn arrow turned green, the car inched forward a little, and Elon immediately put on the brakes. The situation was no more dangerous than when a human driver mistakes which light is theirs and does the same thing, which happens pretty regularly in my experience.
And that this is a test version of the software isn't irrelevant, it makes a huge difference—I am much less opposed to internal company testers who know what they're doing than I am to a public beta in the hands of people who believe Tesla's (really egregious) marketing.
Which category do you think Elon Musk is more likely to belong to?
When cars first debuted, they were incredibly dangerous and killed lots of people. Yet we persisted.
Airplanes, too.
Fully autonomous cars are on the streets of San Francisco. They run into trouble, yet the experiment persists.
We should allow for civil suits, especially when negligence is found, but the only way to progress forward is to make real world attempts.
The promise of autonomous driving is worth trillions of dollars for all of the opportunity it will unlock. And as macabre as it may be, there will continue to be deaths along that development path.
For example, the hand crank was known to be fatal and pushed the founder of Cadillac, Henry Leland, to get it rid of it when it his friend was killed after helping a lady on the side of road.
https://www.studebakermuseum.org/blog/the-1912-cadillac-a-se...
Similarly in England and Wales deaths from horse-drawn vehicles were around 70 per 1 million people per year in the early 1900s. That's in the same ballpark as motor vehicle deaths in the 1980s and '90s (80-100 per 1 million people per year) [2].
(It should be noted that medical technology is better now. A lot of those 1900 fatalities would probably have been preventable if they had current medical technology).
A lot of people on HN seem to view the horse era as some sort of idyllic safe, quiet, and unobtrusive pedestrian and rider paradise. It was not. From that second article:
> It is easy to imagine that a hundred years ago, when cars were first appearing on our roads, they replaced previously peaceful, gentle and safe forms of travel. In fact, motor vehicles were welcomed as the answer to a desperate state of affairs. In 1900 it was calculated that in England and Wales there were around 100,000 horse drawn public passenger vehicles, half a million trade vehicles and about half a million private carriages. Towns in England had to cope with over 100 million tons of horse droppings a year (much of it was dumped at night in the slums) and countless gallons of urine. Men wore spats and women favoured outdoor ankle-length coats not out of a sense of fashion but because of the splash of liquified manure; and it was so noisy that straw had to be put down outside hospitals to muffle the clatter of horses’ hooves. Worst of all, with horses and carriages locked in immovable traffic jams, transport was grinding to a halt in London and other cities.
and
> Motor vehicles were welcomed because they were faster, safer, unlikely to swerve or bolt, better able brake in an emergency, and took up less room: a single large lorry could pull a load that would take several teams of horses and wagons – and do so without producing any dung. By World War One industry had become dependent on lorries, traffic cruised freely down Oxford Street and Piccadilly, specialists parked their expensive cars ouside their houses in Harley and Wimpole Street, and the lives of general practitioners were transformed. By using even the cheapest of cars doctors no longer had to wake the stable lad and harness the horse to attend a night call. Instead it was ‘one pull of the handle and they were off’. Further, general practitioners could visit nearly twice as many patients in a day than they could in the days of the horse and trap.
[1] https://legallysociable.com/2012/09/07/figures-more-deaths-p...
[2] https://www.bmj.com/rapid-response/2011/10/31/cars-and-horse...
Back when we used horses for everything, people used to walk aimlessly in streets and children used to play in them. We advocate against that now.
Society chose a new mandate. It wasn't just the auto companies - it was all of us.
“We should kill people because it will unlock market opportunity”, is why we need to start charging the people in charge of these companies with murder when they inevitably do just that. I don’t care about Tesla stock, I care about not being killed by one of Musk’s “hype” projects.
There is absolutely no reason you should assume "testers" know what they are doing. I have met plenty of people with decades of experience in "testing" barely know what they are doing. Even in the case they know what they are doing, they shouldn't be testing a *deadly* vehicle with potentially broken software on heavily populated *public* roads.
This is another way of saying that self-driving vehicles shouldn't exist at all. At some point we have to test them on public roads, preferably before putting the software into the hands of regular users. If you ban even internal company testing, then what you're saying is that self-driving vehicles should never exist.
> This is another way of saying that self-driving vehicles shouldn't exist at all.
It's another way of saying that self-driving vehicles should be invented somewhere else, so that in 10 years you have to beg and plead for an overpriced second-rate implementation with a worse safety profile.
It's not surprising that it made a mistake during testing. That's what testing is for: rooting out mistakes.
Testing is for rooting out errors. Validation is for proving you achieve the desired specifications and failure rate. Validation occurs when you believe you are done or when testing becomes inadequate to discover failures. It is about “proving” the negative with respect to errors.
Broken, incomplete software where defects can be routinely discovered after light usage has no place being used by consumers on public roads.
No more weights: https://www.reddit.com/r/teslamotors/comments/zv5xp9/a_warni...
How strikes work: https://www.youtube.com/watch?v=otknSUQ4ICY
The inability to robustly solve simple, obvious cases and not failing safe is indicative of a sloppy development and validation process that is incompatible with the deployment of a safe safety-critical device.
It felt like more than a few inches. (I'm not splitting hairs here, I really do feel it was qualitatively significantly different from your description, which made it sound like the car was cautiously starting to move just a bit.)
> and Elon immediately put on the brakes. The situation was no more dangerous than when a human driver mistakes which light is theirs and does the same thing, which happens pretty regularly in my experience.
When humans make this mistake, they stop the car themselves. When this car made this mistake, someone else (Elon) had to stop it. These are not equivalent. In the former case you can argue no accident was going to happen, but in the latter you can't.
> that this is a test version of the software isn't irrelevant, it makes a huge difference—I am much less opposed to internal company testers who know what they're doing
This is on a public road. Try pulling this off (well, please don't) in medicine and see how it goes.
This is not nuanced enough. I've been in a cab to JFK in the snow and the driver was speeding around a turn that the car started sliding and eventually crashed into the side of the road.
There are now over 40,000 car accident fatalities in the US per year. So no, humans make mistakes that they cannot correct. https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
Note that these are based on auditable published statistics and are likely a overestimate of the risk as we must assume the worst when doing safety-critical analysis. Tesla could improve these numbers by not deliberately suppressing incident reports and by not deliberately choosing not investigate to avoid confirming fatality reports. But, until they do so we need to err on the side of caution and the consumer instead of the for-profit corporation.
[1] https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/SGO-2021-01_In...
It's hard to tell because the camera is at such a weird angle, but from what I can see the vehicle remains pretty firmly behind the stop line throughout the entire encounter. We can quibble about how fast it was accelerating, but I regularly see worse false starts at lights.
> When this car made this mistake, someone else (Elon) had to stop it. These are not equivalent. In the former case you can argue no accident was going to happen, but in the latter you can't.
I'm actually more worried about the human case than the human+autonomous case. In the human case it is up to the entity that made the mistake to correct their own mistake. In the autonomous vehicle case you effectively have a second set of eyes as long as the driver is paying attention (which they should be and Musk was). This is why I say that it makes a difference that this was internal testing—the driver wasn't a dumb consumer trusting the vehicle, it was the CEO of the company who knew he was using in-progress software.
> This is on a public road. Try pulling this off (well, please don't) in medicine and see how it goes.
Requiring that autonomous vehicles never be tested on a public road in real world conditions is another way of saying that you do not believe autonomous vehicles should ever exist. At some point they have to be tested in the real world, and they will make mistakes when they leave the controlled conditions of a closed course.
Sure, but that's a wildly different case than "it's ready for public roads we super promise"
External intervention was required to stop the car from entering the center of the intersection.
That's not the same thing as "inched forward" is my point.
> In the human case it is up to the entity that made the mistake to correct their own mistake.
You're completely ignoring how likely these events are or how severe the errors are in the first place. You can't jus count the number of correction points and measure safety solely based on that.
> Requiring that autonomous vehicles never be tested on a public road
I never said that. (How did you get from "look at how it's done in medicine" to "this should never be done"?) What I do expect is responsible testing, which implies you don't test in production until at least you yourself are damn sure that you've done everything you possibly can otherwise. Given everything in the video I see no reason to believe that was the case here.
For me this is the important part. Too many people already drive far too distracted. Imagine giving many of those same distracted drivers a great reason to ignore what they're doing even more. Very frequently when I'm stopped at a traffic light I'll see the person next to me on their phone waiting for the light. Would we trust that someone doing that would lookup when their car starts driving into the intersection?
I really do want self driving cars to be a thing but I wish we were going about it a bit differently and I wish other people who wanted it didn't play things down as much as they do.
Isn't this precisely why self-driving (even if it isn't perfect) could actually make the roads safer. People, as you say, are already super distracted (partly because driving is boring).
When I say that the technology makes people feel like they can be less attentive I really do mean it. There was the SF tech worker who was playing candy crush or something when his Tesla smashed into a barrier on the highway. I have friends who own Teslas and talk frequently about how they like taking them on road trips because they can relax a bit more and pay less attention to the flow of traffic (it'll brake for you!!! they say). In a world where these cars have to share the road with human drivers and drive on roads that are under construction or in poor weather conditions I just don't see how we can say this is safe.
How attention-monitoring strikes work: https://www.youtube.com/watch?v=otknSUQ4ICY
Top comment: "The cabin camera really does feel super solid at detecting when I’m distracted. Even if I’m just like searching a song on the infotainment it will get onto me which is annoying but I completely understand and am glad that it works so well ..."
First, let's assume that perfect law-abiding, self-driving cars exist. One one hand they would eliminate incidents caused by inattentive driving, on the other hand they would in fact create incidents where attentive driver in the right of way would yield for an inattentive driver. Change in total incidents would depend on proportion of these events. Anecdotal evidence is anecdotal, but in my own experience number of incidents I have avoided simply yielding when having right of way is much higher then the number of incidents I have gotten into due to my own mistake.
Second, actual "self driving" cars are far from that, especially in their interaction with other drivers.
Third, there second order effects. E.g. a car quickly, unexpectedly maneuvering could cause other car to break sharp, which could end up in an accident where the original car is not even involved. With an increase of cars behaving differently than the local custom such accidents are bound to happen.
Most probably we are going to see an increase in the number of accidents with proliferation of semi-autonomous vehicles before that number starts to dwindle.
the attentiveness monitoring and strike system? If they ignore it, they will quickly exhaust their strikes and get locked out for bad behavior until they learn to take it seriously.
Just imagine: Constant 100% always-on supervision, supervision of the supervisors with 3-strikes you're out attentiveness monitoring, automatic and manual reporting of possible anomalies with full system+surroundings snapshots to inform diagnostics and development, immediate feedback of these into the simulations that validate new versions, and staged rollout that starts at smaller N (driving simulators are actually pretty good) and continues intensive monitoring up to larger N. Even Phase 3 trials only involve thousands of people, while FSD beta is driving a million miles per day with monitoring that feels more like Phase 1 or mayyybe Phase 2.
One day drug development will be this sophisticated, and it will be glorious.
Beta software shouldn't be on the road where other people can be impacted by it. Want to run into the back of a fire truck? Fine by me. Just do it on a track (and record it). I'll laugh from my computer chair rather than being potentially exposed to it on the road.
Early Access also doesn't attempt to run a red light because 'cars are supposed to go straight' when they're taking a protected left hand turn.
From one of the posts above. The monitoring system could not detect if a person was in the seat or not, and would autonomously drive with a giant teddy bear, a giant unicorn and a completely empty seat and repeatedly hit a moving object the size of a child.
>Except for the part where they can not even detect trivial cases. One of the easiest possible cases, a giant teddy bear in the seat “holding” the wheel (by attaching a simple weight to the wheel), is determined to be a attentive driver [1]. A T-ball and they strike out. The inability to robustly solve simple, obvious cases and not failing safe is indicative of a sloppy development and validation process that is incompatible with the deployment of a safe safety-critical device. [1] https://www.youtube.com/watch?v=CPMoLmQgxTw
Still, clever bypasses need to be taken seriously. Fortunately, Tesla has a track record of doing just that and closing loopholes: https://www.reddit.com/r/teslamotors/comments/zv5xp9/a_warni
A bypass that requires completely vacating the driver seat is probably not easy enough to be widely exploited. If you have evidence to the contrary, please do tell, but while we are exchanging memes, let's review the Green Hills Software standard of excellence, just to contextualize the shade they throw: https://www.youtube.com/shorts/UdSVMgay0MI
lol, if you knew about it, then why is it on the road?
You'd need a simulation that captured all the edge cases of the real world, and that's logically impossible.
In safety critical systems design specifying operational domain is quite literally one of the first things that are done, even before any serious work on the design starts.
The only reasonable phase where safety critical systems can be interacted with closely monitored public are acceptance testing that either validates the model or drags the team back at drawing board.
Yes they do. When you get your license you have to past a test that talks extensively about defensive driving and how to deal with other drivers.
Also, why wouldn't they be able to test it in a simulator?
And this is more like a staging environment than it is production—production is when you put it in the hands of customers, this is an internal release available only to employees.
I'm firmly opposed to FSD beta, but this video doesn't concern me at all.
Why not? What's the limiting factor? Video bandwidth?
We can run video games at 4k 60fps. Is that not enough?
Even if you could perfectly simulate conditions for the sensors to pick up (incl. precipitation, light, dust) you cannot possibly simulate all possible scenarios that can happen in the real world.
Instead you run some basic must-pass scenarios, like the one in the video, where there is a red light in front of the car, and a left turn green, in clear conditions and daylight.
If the model doesn't pass, then don't ship it!
Yeah, that's what integration testing is for.
I would be surprised if they couldn't test the brains of FSD independently in a simulator. That's just QA 101.
You can if you invest in good simulators. See: https://waymo.com/blog/2021/06/SimulationCity.html
> One way we ensure statistical realism in our simulated world is by creating realistic conditions for our autonomous driving technology to experience. So, if the Waymo Driver is going through spring showers in Detroit at sunset in SimulationCity, we can recreate raindrops on our sensors and even simulate other minute details such as the dimming light and solar glare.
If they can simulate rain drops, it's definitely possible to simulate traffic lights. They are a basic scenario for a self driving car. What this video highlights is the extreme lack of validation culture within Tesla FSD.
We can't expect people to never test self-driving cars on public, populated roads unless we are just acknowledging that we don't believe self-driving cars should be a thing, because I sure as hell am not comfortable with the idea of cars going straight from simulation to production.
This is what you're advocating for by saying people should test these things on public roads. That is production, none of the other drivers have consented to be part of the "test".
Tesla FSD routinely fails at 1 in 100 mile safety-critical failure modes. The simulation does not need to be perfect, it just needs to be 100x better than what they have and FSD needs to be 100x better before they should be allowed to inflict it on general consumers.
If they want to test with trained safety drivers in a robust and well-controlled real world environment to identify failure modes that need to be addressed, that is fine. But non-criminal usage by consumers is at least a factor of 100x away.
Any version of SDC software that has a stoplight bug is proof that the QC process of that software is broken and that the makers don't care about safety.
Do you have examples of stoplights that aren't red circles? Or do you have examples of when a Tesla approaching a glowing red circle should not recognize it a stoplight?
I built a CV system in 2013 that detected vehicle makes/models in varying weather and time of day. It was a lot harder than stoplights, we had a team of one, and it was with technology that is ancient by today's standards. Tesla has no excuse.
Most Teslas have multiple cameras, so the viewing angle shouldn't matter as much to the vehicle as it does to a human.
1. https://twitter.com/RealDanODowd/status/1695268712910348471
I was just commenting about that.
Elon had his hands on the steering wheel ready to make any intervention, this is hardly some crazy and dangerous testing going on. I would be far more worried about the average American driver than Elon testing the latest version of FSD.
I’ve had an incident where a crazy guy fully stopped his car on the I5 fast-lane to LA trying to brake test me before getting out of his car to threaten me. I’m far more scared of people than I am of software.
(have lost friends to both drunk and distracted drivers)
2. Elon mode. https://www.theverge.com/2023/6/20/23767041/tesla-hacker-elo...
3. https://twitter.com/RealDanODowd/status/1687106791715573761
4. The video clearly shows no hands are on the wheel at various points when Elon lowers the camera in his hands or when making 90 degree turns so the side of the yoke is pointed up and visible with no hand on it. The video is literally direct video evidence that your claim is untrue and you have no idea what you are talking about.
In the "RealDanODowd" videos he has fitted an aftermarket accessory to the steering wheel to fake having someone touching and the other video "Elon Mode" had someone literally hack the car. Having Elon himself make a publicity video isn't a reflection on how all the cars work.
Anyway, Ford lets you use their system without hands on the wheel, as do the majority of other manufacturers. People just want to hate on Elon because of the "anti woke" crap but there's nothing demonstrably different to any other auto maker.
Way back in 2019 I rode in a Honda Civic that had lane keeping functionality (auto steering and braking) which let you take your hands of the wheel for short periods of time. I imagine the tech was about 1000 times less sophisticated, and in fact, I remember it leaving the lane and needing correcting often, where were the scary and inflammatory articles then?
Let's get real, there's one rule for any news about Tesla, and another rule for other car makers.
I said that Elon does not have his hands on the wheel. You then retorted saying that it will not let you drive without both hands on the wheel, implying that what I said is impossible.
I then provided 3 independent techniques available to Elon Musk that would allow him to drive without his hands on the wheel and pointed at direct video evidence proving your baseless assertion and implication are false.
As to your second point, Ford’s system is explicitly designed, advertised, and declared to be safe to operate hands free with attention being determined through a camera driver monitoring system (DMS). Tesla explicitly declares in the legal fine print that their system should only be used by a attentive driver with their hands on the wheel.
If you think it is unfair to hold Tesla to their own standards for what is safe usage then take that up with Tesla. If they just take liability for hands-free operation like Ford then Tesla can remove that requirement and there would be nothing to complain about if Tesla shows hands-free operation because that would be a legally supported mode. Until they do so they are absolutely guilty of demonstrating unsafe and unsupported operation in official, sanctioned, advertising.
I don't ride a Telsa or an Elon and I know that's not true. SMH.
(Source: I think I did do this once or twice.)
One of the things they talk about in the video is the hoops they had to jump through to get the car to make a complete stop at stop signs, since almost no human drivers actually do that.
Is he really driving if he isn't actually in control most of the time?
Are you trying to imply that a human driver monitoring a self driving system is somehow increasing the risk to others when its actually decreasing risk?
The question is what % of drivers will fall under 1 vs 3?
That video does nothing to change that. Why should it?
https://www.tesla.com/VehicleSafetyReport
https://www.teslarati.com/tesla-fsd-beta-safety-crash-statis...
Both autopilot (basically lane-keeping) and FSD have multiple times higher miles driven between crashes than the US average.
As per the company, a Tesla with FSD Beta engaged experienced an airbag-deployed crash about every 3.2 million miles in the last 12 months. This makes the system about five times safer than the most recently available US average of 600,000 miles per police-reported crash. It should be noted that FSD Beta specifically works for inner-city streets, which involve tons of edge cases and unpredictable driving behaviors from other vehicles.
I really find it hard to understand how anyone who has driven a Tesla with AP for any distance thinks it is safer than a human driver. Maybe just drivers who are always texting or drunk. AP is the least defensive driver on the road, it needs close supervision.
Do you trust VW if they first party report their emissions? Philip Morris if they first party report the effects of smoking on lung cancer? Unaudited first party reports are literally worthless. They have every incentive to lie or misrepresent.
For that matter, Tesla has repeatedly, actively lied to consumers and misrepresented their products.
The current director of Autopilot software, Ashok Elluswamy, admitted under oath to staging the initial “Paint It Black” Autopilot demo video claiming full self driving capability: “The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself.”.
The company misrepresented the range of their cars and actively suppressed complaints and gaslighted customers requesting service [1].
As for the “safety statistics” themselves, they are completely devoid of any information. The only information reported is a single ratio munging both Autopilot and FSD together.
They do not break out the products. They do not enumerate the crashes. They do not even report the number of crashes or number of miles used for calculating their “statistics”. They can not even be bothered to publish the damn numerator or denominator of their safety numbers. This is grade school level reporting and they are being allowed to drive 2 ton machines on public roads.
In contrast, this is what Waymo reports: https://storage.googleapis.com/waymo-uploads/files/documents...
A 30 page research paper itemizing the specific crashes. Methodology, references, evaluation models, comparisons, etc. That is what a proper safety report looks like, not the grade school bullshit Tesla marketing is pushing. Tesla safety and compliance should be ashamed of themselves for prioritizing marketing over human lives.
[1] https://www.reuters.com/investigates/special-report/tesla-ba...
This is obviously worse because you have to be paying attention and be able to quickly take over, but it's not like this type of thing would never happen with a human driver.
With their software, Tesla is more of an active participant in those ill-fated decisions and mistakes. So they deserve to bear more of the responsibility.
See here for pre-release addenda, all that's available on the public internet this moment:
https://www.reddit.com/r/pittsburgh/comments/dv9j08/what_are...
Now, if yinz'll excuse me, the driver in the opposite lane going straight has out-of-state plates. So, I need to stop typing this message to peel out / teach them a valuable lesson about right-of-way when the stop light changes in Yinzerville - as I head to a city-government-sanctioned hazard-light parking spot directly in front of Primanti's to pick up my slaw with fries and sandwich garnish (and sugar with coffee and cream).
Inb4 the bicycle crowd arrives to tell me that the average car driver commits 50 felonies a minute.
Sure. As long as Tesla itself, the company and its leaders and its shareholders, are held personally liable for any major autopilot or FSD fuckups.
Be it a monetary ticket or time in jail, have them all serve that responsibility each and every time. And if a long stint in "jail" - i.e. not being able the conduct business of any kind for that period - is too traumatic for the company to survive, then maybe they'd rethink their level of recklessness.
And certainly, if I'm in any sort of roadside screwup nearly regardless of fault, my insurance rates are going up a lot, for a long time. So too should that be the case for Tesla and their products whenever and wherever their software makes a decision that is involved in or the cause of the same.
Are you sure?
That would include everyone who owns S&P 500 index funds, probably most (if not all) pension funds, etc.
Fund managers? Triply so. They 1. knew better 2. enabled that 3. with other people's money
The entire issue of excessive CEO compensation for the sake of driving stock price, while being completely divorced from any repercussions of any knowingly harmful actions is exactly the capitalism part of "late stage capitalism" that sucks.
Let's restate your question -- Why should shareholders not get to benefit from blatant disregard for truth or safety without any liability on the line when that stance results in collateral damage?
edit: Upon looking up some stats I might have misremembered, in any case the median human drivers is definitely vastly safer than Autopilot.
Try to find the number of miles driven or crashes in the report, you know, the numerator and denominator. You will not find it because they seem to think that a “safety report” is a single number. You would fail a 6th grade science fair project if you did that.
And they seem to think it is okay to deliberately publish such a report for two ton machines with software that has killed dozens of people. They are either so stupid they should not be allowed near a safety-critical device or they are being deliberately deceptive to a level that would make Elizabeth Holmes blush.
So no, their safety report is anti-evidence for safety. Only somebody with something to hide would publish such a deficient and deceptive report.