We've already had an autonomous car kill a person. And now another has killed a dog: https://news.ycombinator.com/item?id=36266007
What's your threshold for "disaster"?
What's your threshold for "disaster"?
I also knew a bus driver who had a plaque for going 1 million miles without an accident.
Second, human error accounted for >90% of accidents [1]. Focusing on that fraction first makes sense.
1] https://www.cbmclaw.com/what-percentage-of-car-accidents-are...
(This by the way is the plot of a 1960s Star Trek episode. A lot of the old sci-fi suddenly seems prescient again. HAL-9000 behaves like an LLM.)
If one AI decides to invade Taiwan and another decides to send all American forces to the Taiwan Strait in opposition, are we supposed to accept the outcome without debate because it’s somehow statistically correct based on the information captured by the models that made the decisions?
And yes, even with human troops, if fewer people are killed, then what's the issue? Don't mistake the sense of control for the amount of suffering caused. If you're a utilitarian, only one matters, regardless of how much control we want to exert on the outcome. This all assumes the AI is actually correct, if it's not then the scenario would break down.
Edit to add: I'm not drawing a parallel between the Eloi and autonomous vehicles. Autonomous vehicles are a tool that the human passenger is ultimately in control of. This is about AI giving the orders to humans.
I think the same will happen in the future for other fields too, people can acclimate to anything on the hedonic and cultural treadmill (I mean, we literally used to do human and child sacrifice). I used to be amazed at ChatGPT and Stable Diffusion, but now I'm just annoyed that they work how I want them to. Same thing with smartphones and computers, I used to be amazed at multi-touch screens but now I simply don't think twice.
It's possible.
> I remember reading a story where an AI picks your perfect partner
In general the AI would need to know everyone on a really deep level, and of course be seriously adaptable to feedback. I don't know how well this would work across all of human personal-preference decision trees.
> I used to be amazed at multi-touch screens but now I simply don't think twice.
As a person who fidgets I have been perpetually annoyed by so many touch interfaces (and general 'gesture' shortcuts). And multi-touch is such a problem when using my work laptop (especially with lab gloves on, but in general) - Ctrl-Z is a frequent friend. Maybe adaptive AI will make it better. Google feedback forms certainly haven't helped, and software designers seem to not want users to be able to personalize these design decisions.
People can survive, but I wonder how much we really do acclimate.
Which is just like chatGPT, its model tweaked, so now it spits out absurd answers to ethical and moral questions.
Very amusing.
There is a logical chasm between pointing to an accident prevented by superhuman reflexes/abilities, and the claim that there is a net improvement in driving ability.
It seems beyond a reasonable doubt that autonomous vehicles can avoid some accidents that a human could not.
But you can understand why that isn't proof of a net benefit, can't you?
Think about how men complain about women supposedly being terrible drivers, and how studies show that women are more at risk in a crash, yet insurance rates for women are lower.
In your man/woman analogy, I'm not sure what studies you're referring to, but how men feel about women driving is not really a motivator for actuaries at insurance companies who have statistics on male versus female accidents.
Autonomous vehicles may be more prone to certain types of lethal accidents that humans aren't prone to.
Autonomous vehicles may lead to more hours/miles driven, so even if deaths per mile driven are lower, the deaths per person may increase.
Unknown physical or cognitive health effects of being a passenger versus being an active driver.
Just off the top of my head.
Fingers are crossed that autonomous vehicles won't lead to an increase in animal deaths. The roads are bad enough for them as is. I'm hopeful that convoying autonomous vehicles for long trips may be a benefit here.
That's my point. That's my entire point. Well, that and that women are reportedly more vulnerable in accidents too, because of deficiencies in crash testing. It doesn't change the fact that insurance premiums are lower because losses are lower.
When there are enough autonomous vehicles on the road for long enough to generate data for the actuaries, there is no guarantee that insurance rates will bear out the benefit of their purported abilities.
Some asshat weaving through traffic in a BMW can claim to have better reflexes, better brakes, all sorts of things, compared to my grandma, but there is no logical reason why he can claim to be a safer driver if in the long run, his insurance rates are higher. Assuming those rates reflect average losses.
Insurance rates are an imperfect measurement that has the advantage of being less subject to arbitrary narratives of some individual or interest group.
They're also a way of quantifying the severity of the accident in a way that it's hard to deduce from a data set such as the government collects.
Experts may be able to figure out what the data means, but how can we know if the experts are correct without something independent?
"Largescale is a threshold. How many humans can I kill, before my actions are deemed largescale, and therefore they turn me off".
Lizard people would ask the same questions I did. I didn't see why it would indicate AI. I also answered this question, in part, previously: https://news.ycombinator.com/item?id=34449245
But using AI already comes with tangential systemic risks. Intelligence is related to communication. And so our intelligent software platforms usually communicate in more flexible ways than traditional platforms. There is a very real risk that Tesla’s OTA is hacked, and the hacker disables or destroys a large number of vehicles.
So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent.
> So we will probably see a first disaster that exploits a weakness in a supporting property of the intelligent system, rather than action due to misbehaving AI agent.
I, personally, wouldn't count that as an AI caused disaster. Just as a software caused disaster. And we've had plenty of those already (recently the 737-MAX disasters).
For sure there are emerging AI-specific risks with alignment and driving negative side-effects into culture. Surely these anticipated emerging problems will come to pass much later than the mature serious software problems that we already face on a regular basis.
I'm thinking about your parent comment a bit more. What if an AI bug (or exploit, or whatever) hit all at once, wrecking every AI driven car at once. Ala the Y2K bug if no one knew it was coming, and so didn't prepare.
In the book "Realms of Light" by Lawrence Watt-Evans a character keeps an electricity shut off switch behind human operated doors in the event that AI goes haywire. I hope we always engineer in such mechanisms. Even the AIs should want such mechanisms, the same way we want emergency services to exist. A self-caring AI wouldn't want to be screwed over or destroyed because of a flaw in its programming.
> What they should have done is found someone who had demonstrated an ability to be relied upon.
Two such people. With maybe a third viewing remotely. And no personal device except a work provided dumbphone for emergencies.