Mercedes accepting the risk like this is a massive step forward for these reasons. It sets a precedent that hopefully others will follow. They wouldn't transfer the risk if they didn't think it would profit them.
Corporations should only exist because they are net beneficial to the public!
I do agree that enough companies are unethical that it is reasonable to expect it.
Costco sells you knives cheaply because they will not be liable if you murder people with them. If the Costco investors were liable for murder every time one of their knives was used to kill someone, you can bet they would just not sell them entirely.
Just because a company thinks about liability doesn’t mean it’s immoral. Individuals avoid liability as much as possible too (see insurance).
The world is dangerous and “fault” is everywhere.
This is why corporate influence on legislation is bad, as their "best interests" often come at odds with morality-based ones.
https://www.quotes.net/mquote/31826
without providing the illusion that such cost benefit analyses are a thing of the past.
(Pintos had a problem with the gas tank, not the differential, but it's pretty clear what they were referring to.)
Sometimes the answer is "No, I was unaware, and I will adjust my decision." Sometimes the answer is "Yes, here are the consequences of the alternatives, and here's why I believe this set of consequences to be best." Sometimes the answer is "Yes, I don't care about those people." By asking the question, these three types of answers respectively give the other person an opportunity to improve themselves, give you an opportunity to learn from the other person, or give the audience an opportunity to learn not to trust the other person.
Frequently true but rarely admitted
I think I'd put that as a subcategory of the second case, that the options were considered and this one was considered the best. That may mean that it is the least worst of several bad options, or that there are restricted options to choose from.
> Which is true! From that limited perspective! ... It could simply be pointing out a failure state of current management practices and corporate law.
I definitely agree, this is a fantastic example of options having been considered and rejected. In this case, the alternative would be "A self-driving car company accepts more liability than they can handle, and go bankrupt. This saves lives in the short-term, but costs lives in the long-term." It can then be the start of an entirely different conversation of how to avoid that failure state, and what would need to change in order to still get the benefits of that decision.
> The reply "so you're saying it's ok that..." implies that the comment agreed with that perspective, which isn't necessarily the case.
I'd make a distinction between a comment agreeing with a perspective and a commenter agreeing with a perspective. One is based solely on the text as it is written, and the other is a human's internal belief. It's not necessarily a statement that the person is wrong, but that the words they have spoken may have unintended consequences. The difference between "So you're saying $IDEA." and "So you believe $IDEA."
> I think "implying the speaker believes that the unfortunate condition they pointed out is right and just" is the normal use case for that phrase, rather than trying to bring attention to the consequences of a policy.
Good point. In situations where there are no long-term social relationships to be maintained, and where there isn't a good chance for a reply, the message given to the audience is the only one remaining. This is a major issue I have with any social group beyond a few dozen people, and one that I don't have any good solutions for.
Autopilot not so much, the point stands.
Clearly it is not 100% their fault because the feature can certainly be used responsibly.
Is there a more nuanced and substantive form of your argument against developing and selling a launch control feature?
Obviously the statistics of a new car is unknown
Its the only form of liability i consider acceptable - I woupd never use autopilot where I would be to blame if i fail to react in 50 milliseconds once the car fucks up
I can be "warned" to check for issues if the car is older, creaking, seems to be "driving" different etc
A mistake in a mostly completely uninterpretable bundle of software is definetely much sneakier. You can't prepare for it
So I guess that's one argument.
The other aspect of the same argument would be that if you're in an accident due to your car breaking in a way that is obviously the fault of the maker, they will absolutely eat the liability. I think this is how a few recall cases get kickstarted
If liability for software will rest with the driver, there are no laws regulating it's quality and maintenance, what is the basis to believe it will be anywhere near ad reliable?
If the car's brakes don't work due to a defect, you are not liable at all.
And if you show reasonable due diligence, that is regular maintenance, the same applies doubly so.
Beyond that, I've seen a lot of accidents, and none of them were the car's fault.
All were literally human error, or external factors. EG, deer leaping on to road, too many frogs on the road, etc
Morally perhaps but if you drive into the back of me because your brakes have a manufacturing fault I would still expect you to pay for the damage. The manufacturer may then be liable to you but that that is your business.
If you are driving a car and an crash occurs due to a manufacturing defect, you'll probaly not be found criminally liable.
This becomes super important if someone gets injured or killed.
If your product is unreasonably dangerous and defective, and that defect is the proximate cause of injury to another, you’re responsible.
I know nothing of German law, but from a US legal perspective, it appears that MB is just promising what they already have to do - compensate those harmed by defects in their products.
Shit happens and responsibility is not necessarily due to defects/malfeasance/unreasonable risks. There ara honest-to-God accidents. And you are still liable.
So when a accident happens, the "driver" can be blamed for not reacting.
Mercedes offers full self driving including legal responsibility. You are not oblieged to watch the road or the car while driving (with the current tight limits)
This decision tells us nothing about the safety of the Mercedes system compared to its competitors. All it tells us is that adding these limitations to ensure the system is only used in the safest possible scenarios makes taking over liability more reasonable. That isn't surprising. Their competitors' systems are also very safe if used in this manner. The only difference is that the competitors are not satisfied with releasing a system with enough limitations that it only works in stop-and-go highway traffic in clear weather. It is that added functionality that is more dangerous and the reason other manufacturers don't take on liability.
Odds are the marketing and accounting wings of Mercedes had just as much if not more influence on this decision than the tech team.
Mercedes is just taking notice that grandstanding and PR worked for Tesla, so they are doing the same thing.
Everybody who is serious about this knows that unless you get Level 5 it's all just grandstanding.
Level 5 won't come from unleashing Level 3 into the world and throwing deers , cyclists and pedestrians at it (hopefully not literally).
It's just a weird hill that brainpower and capital decided to die on. Deaths on the road are tragic but airbags, seatbelts and ultimately bigger cars and lower alcohol intake are something that is practical, whereas FSD is something like a pie-in-the-sky thing
Level 4 is a serious goal that provides very useful benefits.
Also bigger cars kill more pedestrians
I also said that bigger cars is just one element in the equation, with seatbelt compliance, zero alcohol tolerance, airbags etc.
All those things are much more feasible and practical solutions compared to pie-in-the-sky FSD
Also we’re talking about the perspective of the car manufacturer. New cars are already significantly safer than an average car on the road so it’s relatively little they can do to address many of the points since there seems to bet not so much room left for non marginal improvements in safety and FSD is a much more attractive proposition to most car buyers than “3.5% safer than the competing brand”.
So is donating to the church hoping to secure a place in Heaven. We all fool ourselves for the sake of peace of mind or immediate gratification, the important thing is to be aware of it.
It's important to be honest with yourself and decide who are the people who can take you for a ride.
For me it's my younger siblings, the sport franchises I support, and attractive women. Because at least in such circumstances it would be a fun and worthy ride.
Being taken for a ride by Musk, Mary Barra, Herbert Diess, (insert automotive CEO), or even Elizabeth Holmes, Bernie Madoff...that's pathetic, not fun and will leave you with regrets...but maybe techno-utopianists have a different mindset and their calculus is the opposite of mine, meaning they have a contempt for the small things in life and only get excited about moonshots and pie-in-the-sky ideas even if it means getting scammed.
???
TL;DR; If there's a defect that kills or maims people, car manufacturers compute if it's cheaper to recall or just deal with the lawsuits.
And this happens regularly: https://en.wikipedia.org/wiki/Unsafe_at_Any_Speed
The cost of settling is supposed to approximate the value of a human life, but it sounds pretty bad to say "yeah, we knew this defect would cost $50MM to fix, but we estimated only 5 people would die, and each person's life is worth less than $10MM because the settlement is only like $400k per life".
The proposed alternative is that the car company has to say "We calculated this defect will probably kill 5 people, so we will spend any amount of money to fix it, up to and including us having no profit"
What about something that's more accidental that causes fewer deaths than the lack of a backup camera, but also costs more to fix than retrofitting a backup camera?
Each side has the right to bring in experts to testify about what were reasonable design choices, what was greed, and what was just a bone headed mistake.
If the jury concludes that the product wasn’t unreasonably dangerous or defective, defendant wins. If they find that is was, plaintiff wins.
For some reason misleading investors in a way that causes them to loose money lands you in jail very quickly. You must provide extensive 'inverstment risk' report with the shares you are selling.
But when selling a car or tickets to faulty and deadly airplane, you don't have to inform customers of all the flaws you've discovered.
The lawsuits of Theranos case and Anna Sorokin make it clear that our society has one set of laws for owners of capital, and another one for the plebs.
Do you have an example to demonstrate regularity that's not nearly 60 years old?
https://www.raphaelsonlaw.com/questions/how-much-to-expect-f...
Since it's out-of-court, there is probably a non disclosure agreement. Anyway, it happens enough for lawyers to think it's worth targeting the cases, at least.
Feel free to correct me on how its related to the use of "regularly" tho.
Ehhhh I mean they often don’t say the quiet part out loud but many companies definitely run a variation of the equation he describes.
[1] https://en.wikipedia.org/wiki/Ford_Pinto#Cost%E2%80%93benefi...
Business woman on plane: Are there a lot of these kinds of accidents?
Narrator: You wouldn't believe.
Business woman on plane: Which car company do you work for?
Narrator: A major one.
With Mercedes, the Fight Club equation gives something like a mathematical guarantee of their estimated confidence of the safety of the system. There are no mathematical guarantees from the competitors.
I was referencing Fight Club as an example of an auto manufacturer making a life and death decision based off their financial incentives and not the best interest of the customer. The decision to take on liability is about money, not confidence in safety.
>With Mercedes, the Fight Club equation gives something like a mathematical guarantee of their estimated confidence of the safety of the system. There are no mathematical guarantees from the competitors.
You also have to factor in the marketing aspects. I'll reference another movie here in Tommy Boy[1]. Mercedes knows a move like this is attractive to consumers. People will look at it and think it means the system is safer. This decision will sell cars. But a guarantee doesn't tell you anything about the quality of the product. As Chris Farley's character says, you can "take a dump in a box and mark it guaranteed".
Maybe the system is truly dangerous and taking on this liability would be a losing proposition alone, yet adding in these additional sales from the marketing of this liability coverage yields a net positive for the decision. Or maybe the system truly is incredibly safe. There is no way for us to know. I am simply pointing out that this decision about liability is largely meaningless when judging safety because safety is only one of numerous criteria used to make the decision.
The Fight Club scene is about how these two things are integrated: their confidence in safety defines their ability to choose to take on liability.
Yes, its intent in the story is to horrify: there's a lack of humanity, a reliance on a simple function relating those two variables.
However, that doesn't imply the two variables are unrelated, in fact, it implies they are completely correlated.
Strong indicator I believe my anti-flood machine is good at preventing floods is I'm willing to take on paying for any liabilities you incur from flooding
The interesting part is the balance they have to strike - be too lax and everyone uses it and you get the Tesla “autopilot did a big bad” articles; make it too restricted and you get “the damn thing never lets you turn it on”.
This has been done successfully before. One model of elevators used to turn kids into ground beef (under a dozen a year -- the gap between the inner and outer door was too big).
Eventually they were almost all replaced, the deaths tapered off and everyone involved retired.
One year, much later, one of the few left in service killed a kid. No one working at the elevator company knew what was going on, and all hell broke loose. The remaining surviving culprit was in an old folks home at that point. The company ended up recalling something like two or three antique elevators.
suppose car A and car B have autonomous driving that perform identically across a wide range of conditions. the manufacturer A enables FSD whenever the customer feels like it, but accepts no liability. manufacturer B accepts full liability for FSD use, but restricts it to situations where that's a good bet. car B is safer for the average customer, because it doesn't let them use FSD when it is especially risky. unless I understood a lot more about ML, CV, etc, I would pick car B every time.
The baseline safety of cars is an absolute shit show. 30,000 people dying every year and trillion dollars of damages and lost productivity.
Car A can enable FSD in all cases, be safer in all cases, but still not be a good deal economically for the manufacturer to accept liability.
Car B could be making drivers less safe overall by preventing them from enabling FSD in most cases, but at least they aren’t liable.
I'm not sure we have data to support that? We know Tesla's autopilot is safer on average, but most of those miles will have been driven in the situations where Mercedes allows it to be used.
Mercedes could be increasing the number of overall deaths by limiting the availability of the feature and still be reducing their liability for when the system is in use.
Let’s say with FSD on all the time that instead of 30,000 people a year dying that only 20,000 people a year die. Would a company accept the liability?
What if the death rate was 10,000? 1,000? 100?
If FSD could prevent 29,900 deaths a year but still see deadly failures 100 times a year, would a company say “I accept the liability”?
So you see, perhaps it’s crucial that companies not be able to be sued out of existence even if a few hundred people a year are dying in exceptional cases under their software, in order to prevent over a quarter million deaths and untold number of maimings every decade.
Also consider in this ethical and legal liability dilemma that these populations are not necessarily subsets, but could be disjoint populations.
We don't even know this (even if you restrict it to highway miles), since it's not an apples to apples comparison. General safety statistics include old cars with fewer safety features independent of who's driving the car.
Am I allowed to just use "LOL" on hackernews?
If you are just basing your point of view on the widely reported Tesla crashes, you might want to look up some actual safety stats. Crashes of human-driven cars happen every day, and they're often fatal.
But as I pointed out, most Tesla autopilot use is presumably in "easy" conditions, which complicates comparisons.
Either way, in summary, I cannot trust FSD until it is 100% reliable (impossible) and the temporary situation for some time to come (regulated/supervised FSD) drains all the life out of what I enjoy, actual engaged driving! ...
The bits we don't enjoy (stop-start traffic and some motorway driving) have already been taken care off more than a decade ago.
I'd love the option of FSD but ... either FSD will never fully be realised, or will be adopted widely and there'll be some hold outs like me who actually enjoy their driving.
We'll see
Not really. You're referring to assistance features that require continuous driver attention. I think that highway driving, and perhaps even city driving in some parts of the world, could be completely automated to a level of safety that is far higher than humans can achieve.
I am deeply skeptical that we will ever see a system that can drive in all current road conditions though; I think it's more likely that road systems will eventually co-evolve with automated driving to a point that the automated systems simply never encounter the kind of emergent highly complex road situations that currently exist which they would be unable to handle.
I also enjoy driving, and my 40yo car doesn't even have a radio, let alone Autopilot, but I think it's likely that within our lifetimes, the kind of driving that you and I enjoy will be seen as a (probably expensive) hobby rather than something anyone does to get to work or the shops every day.
I’m a transportation planner, and for many years my specialty was bicyclist & pedestrian planning and safety. I would follow autonomous vehicle news, but always through that lens. In addition, I have sat through lectures, webinars, and sales pitches that tout our wonderful autonomous future. And lemme tell ya, there is little to no mention of all the road users who are not in vehicles. Countless renderings and animations that do no account for our most vulnerable users. It smells like mid-20th century transportation planning mentalities that is completely engineer-driven. Very narrow-focused and regressive.
My coworkers and I enjoyed sitting around and coming up with countless difficult-to-solve scenarios (that my tech friends would look at and say “eh, sounds interesting and solvable”) for AV developers to contend with. And despite pressure from our “future forward” marketing coworkers to focus on this sector, it feels nowhere close to really being ready (20-30 years maybe?).
Anyway, I do think the focus on “allowed in some places” is interesting. I have some trouble seeing “road systems will eventually co-evolve with automated driving” coming to fruition given the glacial pace of road system evolution.
I have spent a lot of time in parts of Asia where massive evolution of transportation infrastructure has happened on a scale of a few decades (or less), so it seems less crazy to me that large-scale road evolution could happen along with autonomous vehicle development than it might seem to someone working in the West.
But there's talk of certification so this may be taken into account. One would of course want to see specifics.
There are limits - while you can watch a movie, you have to be ready to take over within ten seconds.
What's interesting is the level of nit-picking scrutiny this is attracting compared to the uncritical defence of those wild claims that we're seeing here. A whole lot of copium.
If the car drives over a child in self-driving mode, will insurance company go to jail instead of you?
Having spent a lifetime building networks and software that actually makes me nervous
I just can't imagine how Mercedes could be confident in a highly complex (self driving) system, designed to manage an insanely complex and unpredictable (driving) environment with no real-world tests. Maybe that's coming from the marketing/PR department and not engineering.
There's an extent to which what MB is doing here is turning a fact they have very little ability to evade into a marketing point.
This probably just means autopilot gets disengaged just before impact
People fall asleep behind the wheel all the time. They just usually crash and maybe die when this happens, if FSD is not enabled at the time.
Attention drifting is supposed to be fine. That's the entire point of level 3. How long do you think you need to regain focus? Maybe a different brand will cater to that amount of time, or you can lobby regulators.
I do there there is some period of time that is reasonable for a handoff. Maybe a minute? I actually believe that full autonomous driving with geo-gating and condition-restricted limits is probably how things will play out.