Hacker News Reader

Comment by slg | Hacker News Reader

But haven't we all seen Fight Club? It isn't a question a confidence, it is a question of financial math.

This decision tells us nothing about the safety of the Mercedes system compared to its competitors. All it tells us is that adding these limitations to ensure the system is only used in the safest possible scenarios makes taking over liability more reasonable. That isn't surprising. Their competitors' systems are also very safe if used in this manner. The only difference is that the competitors are not satisfied with releasing a system with enough limitations that it only works in stop-and-go highway traffic in clear weather. It is that added functionality that is more dangerous and the reason other manufacturers don't take on liability.

Odds are the marketing and accounting wings of Mercedes had just as much if not more influence on this decision than the tech team.

View on HN

It’s both, right? The competitors may very well be just as safe in those conditions, but we wouldn’t know based on their liability stance; the Fight Club equation simply doesn’t apply.

With Mercedes, the Fight Club equation gives something like a mathematical guarantee of their estimated confidence of the safety of the system. There are no mathematical guarantees from the competitors.

>It’s both, right? The competitors may very well be just as safe in those conditions, but we wouldn’t know based on their liability stance; the Fight Club equation simply doesn’t apply.

I was referencing Fight Club as an example of an auto manufacturer making a life and death decision based off their financial incentives and not the best interest of the customer. The decision to take on liability is about money, not confidence in safety.

>With Mercedes, the Fight Club equation gives something like a mathematical guarantee of their estimated confidence of the safety of the system. There are no mathematical guarantees from the competitors.

You also have to factor in the marketing aspects. I'll reference another movie here in Tommy Boy[1]. Mercedes knows a move like this is attractive to consumers. People will look at it and think it means the system is safer. This decision will sell cars. But a guarantee doesn't tell you anything about the quality of the product. As Chris Farley's character says, you can "take a dump in a box and mark it guaranteed".

Maybe the system is truly dangerous and taking on this liability would be a losing proposition alone, yet adding in these additional sales from the marketing of this liability coverage yields a net positive for the decision. Or maybe the system truly is incredibly safe. There is no way for us to know. I am simply pointing out that this decision about liability is largely meaningless when judging safety because safety is only one of numerous criteria used to make the decision.

[1] - https://www.youtube.com/watch?v=mEB7WbTTlu4&t=49s

> I was referencing Fight Club as an example of an auto manufacturer making a life and death decision based off their financial incentives and not the best interest of the customer. The decision to take on liability is about money, not confidence in safety.

The Fight Club scene is about how these two things are integrated: their confidence in safety defines their ability to choose to take on liability.

Yes, its intent in the story is to horrify: there's a lack of humanity, a reliance on a simple function relating those two variables.

However, that doesn't imply the two variables are unrelated, in fact, it implies they are completely correlated.

This real life example is more complicated than the Fight Club version. It includes more variables like the added sales I mentioned and all these variables are unknown. How can you draw conclusions about one variable in a formula in which you don't know the value of any of the variables?

Not sure what you mean: the movie scene has the same property. It's not about the risk of individual failures of components, it's risk of a payout

Strong indicator I believe my anti-flood machine is good at preventing floods is I'm willing to take on paying for any liabilities you incur from flooding

You are only thinking about payouts and not the change in sales. Imagine you make $100m selling your anti-flood machines. Maybe your machine fails 10% of the time and a failure costs 2x the unit price. Taking on liability in that situation would bring you down to $80m. Bad deal for you. But what if someone in marketing comes and tells you that market research suggests taking on liability leads to an extra $30m in sales. You come out ahead because the $30m in new revenue exceeds the $26m in new liability. That isn’t confidence in your machine. It is marketing and accounting.

Occam's Razor: would only work in the short run, crash data goes to regulators.

Many things can be marketing - the drain cleaner sold in a bottle in a plastic bag doesn’t need it for safety - it needs it because it makes the product look more “dangerous”.

The interesting part is the balance they have to strike - be too lax and everyone uses it and you get the Tesla “autopilot did a big bad” articles; make it too restricted and you get “the damn thing never lets you turn it on”.

I don't really see what it has to do with fight club.

suppose car A and car B have autonomous driving that perform identically across a wide range of conditions. the manufacturer A enables FSD whenever the customer feels like it, but accepts no liability. manufacturer B accepts full liability for FSD use, but restricts it to situations where that's a good bet. car B is safer for the average customer, because it doesn't let them use FSD when it is especially risky. unless I understood a lot more about ML, CV, etc, I would pick car B every time.

No, this reasoning is flawed.

The baseline safety of cars is an absolute shit show. 30,000 people dying every year and trillion dollars of damages and lost productivity.

Car A can enable FSD in all cases, be safer in all cases, but still not be a good deal economically for the manufacturer to accept liability.

Car B could be making drivers less safe overall by preventing them from enabling FSD in most cases, but at least they aren’t liable.

Your comment is predicated on the assumption that FSD (the actual system installed in the car, not a future theoretical perfect system) is safer than the average driver in the situations where Mercedes currently disables it.

I'm not sure we have data to support that? We know Tesla's autopilot is safer on average, but most of those miles will have been driven in the situations where Mercedes allows it to be used.

I am saying such a scenario may possibly exist, not necessarily that it does exist.

Mercedes could be increasing the number of overall deaths by limiting the availability of the feature and still be reducing their liability for when the system is in use.

Let’s say with FSD on all the time that instead of 30,000 people a year dying that only 20,000 people a year die. Would a company accept the liability?

What if the death rate was 10,000? 1,000? 100?

If FSD could prevent 29,900 deaths a year but still see deadly failures 100 times a year, would a company say “I accept the liability”?

So you see, perhaps it’s crucial that companies not be able to be sued out of existence even if a few hundred people a year are dying in exceptional cases under their software, in order to prevent over a quarter million deaths and untold number of maimings every decade.

Also consider in this ethical and legal liability dilemma that these populations are not necessarily subsets, but could be disjoint populations.

> We know Tesla's autopilot is safer on average,

We don't even know this (even if you restrict it to highway miles), since it's not an apples to apples comparison. General safety statistics include old cars with fewer safety features independent of who's driving the car.

"We know Tesla's autopilot is safer on average"

Am I allowed to just use "LOL" on hackernews?

Well, unless you are going to rebut the statement, I don't see the point.

If you are just basing your point of view on the widely reported Tesla crashes, you might want to look up some actual safety stats. Crashes of human-driven cars happen every day, and they're often fatal.

But as I pointed out, most Tesla autopilot use is presumably in "easy" conditions, which complicates comparisons.

You were responding to someone saying that Tesla’s autopilot is safer (based on crash stats per million miles), not FSD. FSD and autopilot are two different features.

Fair enough. I cannot believe those related are not in tight conversation considering the AI element of FSD (or am I mistaken there?).

Either way, in summary, I cannot trust FSD until it is 100% reliable (impossible) and the temporary situation for some time to come (regulated/supervised FSD) drains all the life out of what I enjoy, actual engaged driving! ...

The bits we don't enjoy (stop-start traffic and some motorway driving) have already been taken care off more than a decade ago.

I'd love the option of FSD but ... either FSD will never fully be realised, or will be adopted widely and there'll be some hold outs like me who actually enjoy their driving.

We'll see

> The bits we don't enjoy (stop-start traffic and some motorway driving) have already been taken care off more than a decade ago.

Not really. You're referring to assistance features that require continuous driver attention. I think that highway driving, and perhaps even city driving in some parts of the world, could be completely automated to a level of safety that is far higher than humans can achieve.

I am deeply skeptical that we will ever see a system that can drive in all current road conditions though; I think it's more likely that road systems will eventually co-evolve with automated driving to a point that the automated systems simply never encounter the kind of emergent highly complex road situations that currently exist which they would be unable to handle.

I also enjoy driving, and my 40yo car doesn't even have a radio, let alone Autopilot, but I think it's likely that within our lifetimes, the kind of driving that you and I enjoy will be seen as a (probably expensive) hobby rather than something anyone does to get to work or the shops every day.

I think the majority of people enjoy driving. Driving is fun. Sure, traffic sucks, but the actual act of driving comes with lots of pleasures. Most people don’t seem eager to give up driving, nor are many people ready to hand over control to AI.

I’m a transportation planner, and for many years my specialty was bicyclist & pedestrian planning and safety. I would follow autonomous vehicle news, but always through that lens. In addition, I have sat through lectures, webinars, and sales pitches that tout our wonderful autonomous future. And lemme tell ya, there is little to no mention of all the road users who are not in vehicles. Countless renderings and animations that do no account for our most vulnerable users. It smells like mid-20th century transportation planning mentalities that is completely engineer-driven. Very narrow-focused and regressive.

My coworkers and I enjoyed sitting around and coming up with countless difficult-to-solve scenarios (that my tech friends would look at and say “eh, sounds interesting and solvable”) for AV developers to contend with. And despite pressure from our “future forward” marketing coworkers to focus on this sector, it feels nowhere close to really being ready (20-30 years maybe?).

Anyway, I do think the focus on “allowed in some places” is interesting. I have some trouble seeing “road systems will eventually co-evolve with automated driving” coming to fruition given the glacial pace of road system evolution.

I guess by "road systems will eventually co-evolve with automated driving" I would also include relatively minor interventions like increasing the proportion of controlled intersections, which are much easier for autonomous systems to deal with.

I have spent a lot of time in parts of Asia where massive evolution of transportation infrastructure has happened on a scale of a few decades (or less), so it seems less crazy to me that large-scale road evolution could happen along with autonomous vehicle development than it might seem to someone working in the West.

> This decision tells us nothing about the safety of the Mercedes system compared to its competitors

Mercedes is just taking notice that grandstanding and PR worked for Tesla, so they are doing the same thing.

Everybody who is serious about this knows that unless you get Level 5 it's all just grandstanding.

Level 5 won't come from unleashing Level 3 into the world and throwing deers , cyclists and pedestrians at it (hopefully not literally).

It's just a weird hill that brainpower and capital decided to die on. Deaths on the road are tragic but airbags, seatbelts and ultimately bigger cars and lower alcohol intake are something that is practical, whereas FSD is something like a pie-in-the-sky thing

I don't know how you came to this conclusion.

Level 4 is a serious goal that provides very useful benefits.

Bigger cars? Have you seen the new pickups, what do you want to drive, a tank?

Also bigger cars kill more pedestrians

People who say "bigger cars are safer" mean "the person driving the bigger car is safer". It's an arms race to them and they want the bigger car.

Not all crashes involve multiple cars. Bigger cars reduce the amount of deaths because there is more metal mass between you and the tree/barrier that you hit.

I also said that bigger cars is just one element in the equation, with seatbelt compliance, zero alcohol tolerance, airbags etc.

All those things are much more feasible and practical solutions compared to pie-in-the-sky FSD

US and Germany might be at different stages of this process, though. For example Germany has almost 2x less deaths per km driven and the situation is even better in countries like Switzerland or Norway.

Also we’re talking about the perspective of the car manufacturer. New cars are already significantly safer than an average car on the road so it’s relatively little they can do to address many of the points since there seems to bet not so much room left for non marginal improvements in safety and FSD is a much more attractive proposition to most car buyers than “3.5% safer than the competing brand”.

> FSD is a much more attractive proposition to most car buyers

So is donating to the church hoping to secure a place in Heaven. We all fool ourselves for the sake of peace of mind or immediate gratification, the important thing is to be aware of it.

It's important to be honest with yourself and decide who are the people who can take you for a ride.

For me it's my younger siblings, the sport franchises I support, and attractive women. Because at least in such circumstances it would be a fun and worthy ride.

Being taken for a ride by Musk, Mary Barra, Herbert Diess, (insert automotive CEO), or even Elizabeth Holmes, Bernie Madoff...that's pathetic, not fun and will leave you with regrets...but maybe techno-utopianists have a different mindset and their calculus is the opposite of mine, meaning they have a contempt for the small things in life and only get excited about moonshots and pie-in-the-sky ideas even if it means getting scammed.

I disagree. If often had to spend time in highway traffic jams (which is what Mercedes seems to be offering here) I’d rather pay an extra 10k to get 30 minutes of my life back every day instead of buying a car which in which I’d be 1.5% less likely to die during a crash.

The “Fight Club” math is incomplete for the sake of a good story. Regulators can force a recall on manufacturers and insurance companies can make a car uneconomical to buyers by appropriately pricing in risk. You may argue that the former has been undone by regulatory capture (something I would dispute), but I think we all recognize that insurance companies aren’t particularly charitable.

Wouldn’t this logic suggest that the economic value of taking on this liability is minimal considering insurers haven’t made cars with competing tech unaffordable for buyers to insure?

The Fight Club math is complete. They specifically quote the cost of out of court settlements, implying hush money.

This has been done successfully before. One model of elevators used to turn kids into ground beef (under a dozen a year -- the gap between the inner and outer door was too big).

Eventually they were almost all replaced, the deaths tapered off and everyone involved retired.

One year, much later, one of the few left in service killed a kid. No one working at the elevator company knew what was going on, and all hell broke loose. The remaining surviving culprit was in an old folks home at that point. The company ended up recalling something like two or three antique elevators.

>But haven't we all seen Fight Club?

???

http://inaneexplained.blogspot.com/2011/03/fight-club-car-re...

TL;DR; If there's a defect that kills or maims people, car manufacturers compute if it's cheaper to recall or just deal with the lawsuits.

And this happens regularly: https://en.wikipedia.org/wiki/Unsafe_at_Any_Speed

There's always a tradeoff between cost and safety, how else would you imagine this to work?

Maybe cars with known technical/mechanical defects that will result in death could be recalled even in cases where it eats a bit more into the profits of the auto manufacturer than it would if they just paid settlements to the families who lost loved ones. That seems like a pretty good way to imagine this to work.

The proposed alternative, I think, is that the car company values a customer's life at more than the expected cost of settling.

The cost of settling is supposed to approximate the value of a human life, but it sounds pretty bad to say "yeah, we knew this defect would cost $50MM to fix, but we estimated only 5 people would die, and each person's life is worth less than $10MM because the settlement is only like $400k per life".

The proposed alternative is that the car company has to say "We calculated this defect will probably kill 5 people, so we will spend any amount of money to fix it, up to and including us having no profit"

I imagine that they'd ddmit that the car as designed is bad and fix or replace the bad cars. Leaving cars on the road that have a design flaw that randomly kills people, because it's cheaper to pay off dead people's families is reprehensible.

There's a question about what a design flaw means though. What about a car that was built without a backup camera because it was older than when they were commonly included? Or built when they were commonly available, but not before they were mandated? Is that a design flaw?

What about something that's more accidental that causes fewer deaths than the lack of a backup camera, but also costs more to fix than retrofitting a backup camera?

Legally, it’s up to the jury.

Each side has the right to bring in experts to testify about what were reasonable design choices, what was greed, and what was just a bone headed mistake.

If the jury concludes that the product wasn’t unreasonably dangerous or defective, defendant wins. If they find that is was, plaintiff wins.

By treating any lies to the customer as criminal fraud.

For some reason misleading investors in a way that causes them to loose money lands you in jail very quickly. You must provide extensive 'inverstment risk' report with the shares you are selling.

But when selling a car or tickets to faulty and deadly airplane, you don't have to inform customers of all the flaws you've discovered.

The lawsuits of Theranos case and Anna Sorokin make it clear that our society has one set of laws for owners of capital, and another one for the plebs.

> And this happens regularly

Do you have an example to demonstrate regularity that's not nearly 60 years old?

Thesr lawyers claim they got a multi million settlement out of a manufacturer of a defective automobile recently:

https://www.raphaelsonlaw.com/questions/how-much-to-expect-f...

Since it's out-of-court, there is probably a non disclosure agreement. Anyway, it happens enough for lawyers to think it's worth targeting the cases, at least.

Honestly, I didn't read that link extensively. It went straight to a sleezey looking "How Much to Expect From a Car Accident Settlement?" That didn't sound like it justified the usage of "regularly" in the statement I responded to.

Feel free to correct me on how its related to the use of "regularly" tho.

Have you seen the movie, if I remember the scene correctly Norton discusses the financial incentives leading to auto product recalls. I don’t think this is accurate in real life although Boeing hasn’t inspired much confidence.

>I don’t think this is accurate in real life

Ehhhh I mean they often don’t say the quiet part out loud but many companies definitely run a variation of the equation he describes.

The "Pinto Memo" [1] being a notable example. Although, as the article says, the cost / benefit analysis was against "societal costs" of the safety issues, not just the cost of litigation.

[1] https://en.wikipedia.org/wiki/Ford_Pinto#Cost%E2%80%93benefi...

Yeah, very good example and explanation

https://youtu.be/SiB8GVMNJkE

Narrator: A new car built by my company leaves somewhere traveling at 60 mph. The rear differential locks up. The car crashes and burns with everyone trapped inside. Now, should we initiate a recall? Take the number of vehicles in the field, A, multiply by the probable rate of failure, B, multiply by the average out-of-court settlement, C. A times B times C equals X. If X is less than the cost of a recall, we don't do one.

Business woman on plane: Are there a lot of these kinds of accidents?

Narrator: You wouldn't believe.

Business woman on plane: Which car company do you work for?

Narrator: A major one.

Reply on news.ycombinator.com