Comment by daly | Hacker News Reader

Comment by daly | Hacker News Reader

Parent Full thread

Your claim is that Tesla's failures are due to incompetence and malfeasance.

To judge incompetence you need to be able to show what competent behavior is. Please be specific and clearly show what will, and HOW to, achieve competent behavior in an ADAS system. Tesla's approach to that problem is that the human is always responsible. It says so "right on the can". Write your "competent" code as open source and post it for all to use.

As for malfeasance, you need to show mens rea, the intention of wrongdoing, aka a "guilty mind". Please provide specific information you have that anyone at Tesla intended to cause harm. Name names. Be aware that falsely claiming malfeasance by an individual might be legally actionable defamation.

Safety critical systems fail. Nuke plants melt down, commercial and military aircraft crash, ...

ADAS saves lives, more lives statistically than average but not all lives.

You want perfection. You can't have it.

It seems wrong to cast aspersions on people at Tesla without showing how "you would do it right". Be sure to include a github link in your next reply so I can download and test your coded solution.

Literally every single statement you just made is completely antithetical to modern safety processes.

1. Competence is judged in accordance with an acceptance criteria. You do not need to be able to show competent behavior to deem something incompetent. You are not competent to develop a fusion reactor. Nobody can do that, so we know you can not and are thus incompetent. You are not competent to develop a manned lunar space mission. It was done historically, but not now, so we know you can not and are thus incompetent. You are not competent to develop a rocket that can reach GEO. Others can do it, but we know you can not and are thus incompetent.

2. A competent DMS must only allow ADAS operation when it can positively identify a human driver looking at the road in a manner comparable to a attentive driver. Before you go on about some sort of inane tangent demanding I, a non-expert on gaze monitoring, define the exact parameters of what constitutes a "attentive driver" and then seek loopholes in whatever criteria I suggest to declare "victory" over safety, the acceptance criteria I just put forward explicitly does not absolve the manufacturer of any liability for deficiencies in their design whether known or unknown. The use of unproven technology in safety-critical contexts does not get a free pass. It must be deployed with the utmost care until an adequate analysis of its safety properties can be determined.

3. Although I can not speak to what, specifically, constitutes a successful implementation, I can speak to what is an immediate failure. In much the same way that I can not make a rocket that reaches LEO, but can determine that a rocket that explodes on the launchpad is not successful, I can determine what is a clear failure. Allowing the ADAS to operate while there is no human in the driver seat is an immediate, egregious failure. It is completely unacceptable on its face even if there were no known solution. Even worse, human face detection is an undergraduate computer vision project. Basically every phone and camera in the world has human face detection. If a human face is not detected for a certain amount of time then the ADAS should relinquish control. This may cause excessive disengagement if your human face detector is poor, but the reverse fails unsafe which is unacceptable. Literally anybody with even the minimum of safety design training would know that.

4. You demand that I solve Tesla's problem, that their hundreds of paid engineers can not solve, for free and then post it as open source. For some reason you do not demand Tesla, the manufacturer of the product and a nearly trillion dollar company, to release their source code for evaluation before it is allowed to endanger the lives of the public. For that matter, Tesla does not disclose their code even to the regulatory agencies, yet you demand a random member of the public to engage in a more thorough and transparent process than the company with billions of dollars who is attempting to deploy an incomplete safety critical product on the public roadways. The double standard is appalling.

5. It is not necessary to specifically identify individuals to declare malfeasance. I do not know who was in charge of Philip Morris when they deliberately obfuscated evidence linking smoking to lung cancer, but I know that they engaged in malfeasance.

6. The actions of Tesla demonstrate such a disregard for accepted practices in the development of safety-critical products and are such a gross deviation from the normal standard of care and occur with such regularity that it is improbable that the actions are mere criminal recklessness. Either they have no staff who are trained in safety design or they override them to ship product. Both are unacceptable.

They are developing a safety-critical product. Their management knows they require qualified safety experts. If they choose to retain no such individuals, then they have intentionally engaged in conduct that will lead to no safety oversight despite knowing that they should. If they have safety experts and override them, it is self-explanatory how that is unacceptable.

7. You demand I specifically name individuals while also declaring that your demand incurs legal consequences on me while accepting no increased burden yourself. Your attempts to raise the standard of proof exclusively on your opponents and then hurl baseless legal threats are anti-social debate tactics. How about you make a notarized statement under penalty of perjury that you know Tesla is following adequate safety procedures, and if they are not then you are intentionally lying to harm the public. If Tesla is following adequate safety procedures, then the second clause has no effect. If Tesla is not following adequate safety procedures, then you are trivially guilty of perjury.

8. Your legal threat is asinine in any event, so I will go ahead and name some names. Ashok Elluswamy, the current director of Autopilot software, is guilty of manslaughter. In sworn testimony, [1] Ashok Elluswamy on page 70 declared that he was specifically involved in staging the Autopilot demo video to misrepresent the capabilities of Autopilot to customers. This has been relied on by customers as a faithful representation of its abilities and has lead to multiple avoidable deaths. In his sworn testimony he repeatedly declared he had no knowledge of human factors, human factors engineers, or basic safety design concepts. On page 36, he explicitly abdicates responsibility for overall software safety by declaring that he is just a "software engineer" despite being the director of Autopilot software. He is grossly unqualified, made no efforts to retain or listen to safety expertise demanded of his job, and has deliberately deployed incomplete software with known critical defects to the public.

Elon Musk is personally guilty of manslaughter. For the same reasons as above. In addition, Elon Musk was publicly informed that Tesla ADAS would not stop for a school bus with the stop arm extended, as is required by law, in 2022-10 [2]. He was separately informed again in 2023-02 [3]. He took no action to correct the behavior or to recall the Tesla ADAS until the behavior is corrected. On 2023-03-15, a Tesla with ADAS engaged hit a minor exiting a school bus with the stop arm extended [4] causing serious injuries. This is recorded in the NHTSA SGO database [5] as Report ID 13781-5100 which definitively confirms that the ADAS system was operating at the time of collision. Elon Musk was informed that the Tesla ADAS would engage in grossly illegal and reckless behavior and then it did that behavior and caused serious personal harm to a member of the public. He has been separately informed of numerous other life-threatening safety defects and exactly zero corrective action has been taken to mitigate them.

9. Safety critical systems do fail. This is despite great efforts and care taken to rigorously tackle safety problems, the decades of experience in developing high quality safety protocols, and a solemn regard for the challenging task of keeping humans safe. Tesla throws all of this away and tackles problems with a callous disregard for human life and a "ends justify the means" philosophy that is incompatible with safety. Instead of transparency, they deliberately obfuscate their raw data, allow no independent evaluation or auditing, and then publish bogus safety reports like this [6] which include no actionable data at all that are meant to mislead customers into trusting unsafe systems with their lives. They do not even include the count of crashes used or the number of miles driven. They publish a single ratio and call that a safety report. In contrast, Waymo published this safety report [7] at 1,000,000 driverless miles. It exhaustively lists, describes, and analyzes every crash that occurred. It discusses methodology, comparisons, confounding factors, and all the other basic things expected from a real safety report.

Waymo is registered with the CA DMV to do driverless testing [8] and reports all vehicles, mileage, and contact events to the DMV. Tesla is not registered to do driverless testing and has not done any safety driver testing despite claiming their customers are "safety drivers" doing testing. They deliberately misclassify their Full Self Driving, L4, Beta program as a L2 Beta program to avoid mandatory reporting requirements that would reflect badly on them.

10. You can present exactly zero audited evidence that Tesla ADAS saves lives. The only support for that position is the unaudited Tesla marketing statements I pointed to above drawn from private data that Tesla does not allow anyone access to. Unaudited reports by the manufacturer are literally worthless. They have the largest possible conflict of interest to lie or misrepresent the data. All publicly available evidence points to the opposite with Tesla reporting to NHTSA over 700 accidents with no known survivors attributable to FSD in the SGO database over an estimated 350M miles. That is over 150x more deadly than the average human driver.

11. I do not want perfection. I want proven, audited safety statistics, robust safety processes that minimize avoidable deaths and risks and ratchet toward improved safety, and exhaustive safety testing to avoid safety regressions. Tesla provides exactly none of these. I would also like the people making safety-critical systems to have a respect for the humans lives they are putting at risk. Tesla and Elon Musk do not have that either.

In conclusion, your opinions are wrong in every conceivable way. Literally every single statement derived from your beliefs is harmful and detrimental to safety design and has been repudiated by decades of work by actual safety experts making products fit to depend on. And your argumentation style is anti-consumer and anti-social as you seek to burden others with proving your unsupported positions wrong to a ridiculous standard of proof that you yourself are unwilling to attempt and then threaten them in an attempt to cow them.

[1] https://www.documentcloud.org/documents/23574198-elluswamy-d...

[2] https://dawnproject.com/tests-reveal-that-tesla-fsd-will-dri...

[3] https://www.youtube.com/watch?v=_ZiSZbWIrzA

[4] https://www.rflaw.net/howard-gene-yee-hits-10th-grader-halif...

[5] https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/SGO-2021-01_In...

[6] https://www.tesla.com/VehicleSafetyReport

[7] https://storage.googleapis.com/waymo-uploads/files/documents...

[8] https://www.dmv.ca.gov/portal/vehicle-industry-services/auto...

To be clear I didn't "threaten" anything. I intended no threats. Calm down.

Are you confusing ADAS with Full Self Driving? They are nowhere the same.

An ADAS system, like my cruise control, won't stop for a school bus, for example.

Do you insist that my cruise control shut off if I look away from the road? It currently doesn't.

I rented a Tesla 3 for a week. The ADAS system worked as expected. One instance does not a pattern make but I was quite impressed. I have a lifelong involvement with vision and robotics and understand how difficult it is to "follow the lane" in all kinds of lighting, weather, and traffic. In some driving conditions even people can't follow the lane.

As for engineering safety systems we had many debates about protecting human/robot interactions. This is the reason you see fences around robots in factories. Opening the gate will shut down the robot and that will shut down the production line. Workers have still been killed by being inside the fence despite all safety interlocks and worker training.

You claim to want "a ratchet toward improved safety". It seems to me that these early attempts at driver awareness IS a ratchet toward improved safety as they did not exist before. Ratchet safety requires failures and corrections. You claim that "Tesla provides none of these" but seem to ignore the new, previously non-existing, attempts at driver awareness.

Teslas are reasonably safe cars. To quote Euro NCAP safety rating results:

Following this test, Model S again received a 5-star safety rating and the highest Overall Score among any vehicle tested in this protocol. It also performed exceptionally well in more detailed tests across the board: 98% in Safety Assist. 91% in Child Occupant Protection.

Sandy Monro, an auto expert, did a review of the safety design of Tesla. Monro pointed out the engineering features that preserved passenger compartment integrity in a collision. The Tesla I rented had steering wheel airbags, side-curtain airbags, and driver/passenger separation airbags. My current car only has steering wheel / passenger airbags.

Even if you insist on confusing ADAS with FSD the driver is "pilot in command" and fully responsible for everything all the time. If there is an accident it is the drivers fault. Until there are no drivers that will always be true. I am unaware of Tesla claiming that you don't need a driver.

Having worked on an ambulance corp for 5 years I can state from personal experience that people die in car accidents. That will continue as long as there are cars.

You seem so passionate about this topic with a particular focus only on Tesla. That strikes me as odd. There are many car companies with ADAS systems and ADAS failures, most of which don't include an attempt at driver awareness. Where is your review of other systems? Comma, for example, at the Comma Con 2 conference explicitly said they are NOT liable for anything.

If ADAS/FSD safety is your issue please post safety issues of non-Tesla cars. There are 91 car companies in China with ADAS and most of the worldwide majors have ADAS. So lets have a more general, non-Tesla safety discussion.

Self-driving Uber death https://www.washingtonpost.com/nation/2023/07/31/uber-self-d...

Kid ran in front of a Tesla: https://youtu.be/TzdXYn-nA74?t=219

Passenger compartment nearly intact after crash: https://youtu.be/e_OCFBl1r7U?t=3352

Autopilot avoided a crash: https://youtu.be/e_OCFBl1r7U?t=4540

AAA study finds ADAS could save more than 8000 lives per year

https://youtu.be/UVfDrivSm58?t=754

Reply on news.ycombinator.com