Firefighters forced to smash window of driverless Cruise taxi to stop it
businessinsider.com
businessinsider.com
FWIW, the reason(s) this is bad - if not obvious - include:
- The hose can (and does) get snagged on the car's under-carriage, which can rip it away from a hydrant, or engine, or yank an attack line right out of the hands of the crew manning it.
- Hoses snagged by cars can, as they're being unexpectedly pulled somewhere they were never expected to be, catch firefighters in the legs and cause them to fall. I'm aware of at least one case where a firefighter suffered a pretty severe head injury in one of these scenarios.
- Even if the hose isn't snagged, damage from the vehicle driving over the hose can cause the hose to rupture. Hopefully I don't have to say any more about why that would be a Bad Thing.
- Even if the hose isn't snagged, or doesn't burst at the moment the vehicle runs over it, the damage from this kind of thing is cumulative with all the other damage fire hoses suffer, which can lead to the need to replace it prematurely. And let me assure you, this stuff is not cheap.
- Probably more that I'm forgetting right now.
Such kinds of problems will only scale up as we apply L5 cars take on more public roads, but I wonder how long or what changes will have to be done in current road signs, critical city infra to support them.
A car shouldn't make that call. It should err on the side of safety and assuming the emergency is real.
When you see flashing lights on the highway coming up behind you, do you think "I bet that ambulance is faking it so they can get home faster", or do you just move over?
I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else.
But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.
Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it.
(Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.)
Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads.
There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.
Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.
Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?
Essentially the same that you said.
This is something that people are very good at judging but software isn’t.
I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.
Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe.
Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle.
Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.
Legitimacy isn't really the problem, it's the circumstance of a human being in front of your car that prevents you from moving forward, and beyond getting out of your car and talking to them, or just waiting, you don't have much of a choice in what options you have.
I agree with the intent of your point, but what's the alternative? You can't just keep driving if someone is standing in the middle of the road.
In the case of an illegitimate request, a human driver might question the request and take a different action instead.
Are you being directed to run a red light? If you judge that someone is maliciously asking you to do so, you probably will choose to remain still instead.
Are you being directed to stop? If you judge that someone is maliciously asking you to do so, you may choose to turn around, reverse, turn into a parking lot or otherwise remove your vehicle from the situation. You don't have to run people over to refuse a command to stop.
And if you were completely blocked from moving, you'd probably call the police.
My point is that "always follow hand signals" is trivially problematic. I would expect that if self-driving cars followed any hand signal they saw, you'd see kids doing it for the lulz on the sidewalk in front of their school. ..."traffic jam challenge". After all, someone is less likely to act maliciously towards a human than they are a machine.
The more important test scenarios are life threatening or unreasonably disruptive situations rather than harmless pranks.
Nowhere near perfect, but that makes it a lot harder than just getting a spike strip or a hand gun or a demolition derby car. We’re talking about people willing to risk a decade in prison to “do criminal things to a car and/or passengers”. How much work do you think they want to put into disabling a car with a remote? Why not just wave a gun in the driver’s face at a stop light?
Reading between the lines it doesn’t seem like there’s a lot of communication to emergency services etc on how to deal with autonomous vehicles.
Even the police seem to be operating under misconceptions about what the vehicles can and can’t do.
If self-driving cars cannot cope with this, they should not be allowed on public roads. Very little has convinced me they should currently be street legal.
Will you have to file a CVE after the incident?
Community Emergency Response Team: https://community.fema.gov/PreparednessCommunity/s/welcome-t...
"In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. "
Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost.
The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.
I hope that as well, but I think neither the US, Israel nor China will stand idle next to a nuclear power getting dragged in front of the ICC - not with the numerous counts of crimes each of these countries could reasonably be charged with. The Israelis may be the only ones being able to get away with a lot of their stuff given that their opponents continuously enact crimes as well (e.g. hiding military command structures under hospitals and schools), but the US and China? They can't pull that card.
To be fair, the point of the ruse (in real wars; in heroic fiction this sometimes varies) is usually to get people killed, too; the question is just which people get killed.
This one infamous sword fight has now poisoned the well with respect to the global norm. Next time there's some other totally unaffiliated hat-wearing super-powered bandit who's stealing the Ark of the Covenant or the royal diamond or something, if THAT bandit tries to surrender, there's a high chance that person might be killed, because their surrender is distrusted since DragonWriter changed the norm and now every fight between supers and cops is now on "shoot to kill, no prisoners, take no quarter" stakes. Imagine how many times a famous super like Batman would have been killed if the authorities never accept a surrender because the surrender norm is ruined. It can be repaired but that might take a decade or more to repair the trust that was broken.
With strong norms about surrender, red cross medics being protected, etc., we can have things like the miracle of the WW1 Christmas Truce which help the war to be a little bit little less of pure hell, for all the pure-hearted or indifferent people dragged into being a soldier, perhaps even drafted or press-ganged, to be the disposable cannon fodder for a bloody and disease and gas drenched war for no good reason more than rich politicians' egos.
https://en.wikipedia.org/wiki/Christmas_truce
These people just want to go back home to their wife or their mum and would gladly gladly surrender if given the chance, but that's not possible if someone ruins the norm! It's like, for example even if someone is a super-villain for example, they should really not take human lives of innocent people, or destroy irreplaceable cultural heritage sites, or damage/burn an ecological area like a rain forest, because those, once lost, are forever gone, and leave our world impoverished.
But: fake cops robbing people is pretty common too. Unfortunately.
Even more unfortunate, real cops doing it also happens.
So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?
When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.
Relevant xkcd: https://xkcd.com/1958/
A piece of software with some cameras and actuators is really not the same.
If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.
“You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.
And there are going to be a lot of pranksters who do, and will use them to mess with so-called self-driving cars.
Perhaps autonomous vehicles should be be sensitive to that field, signaling for manual control when they're too near a vehicle which has it enabled. That way any vehicle--not just the fire department--can create a "human control only" field by activating their emergency flashers. Perhaps emergency vehicles could be recognized such that they create a much larger field.
You don’t. Bad actors can already pretend to be fireman doing fireman things and direct traffic. Human drivers will also be mislead by that.
What is more, if you have a high viz vest, a hard hat, and more confidence than a garden slug you can also direct trafic. Not legally mind you, but more often than not people will follow your directions. If there is a whole gaggle of fake construction people and they have cones too they can probably re-route a highway.
Whatever is stopping people from causing misschief (too often) by these “powers” doesn’t seem to be different in the case of self driving cars than with human driven cars. In fact the self-driving car is more likely to retain a perfect photographic evidence of the mischief maker’s face than a regular human driver.
Add that to the _LONG_ list of issues that will have to be solved until we can even think about releasing autonomous cars in the streets.
Unfortunately that attitude is common is the industry. The notion that the rest of the world needs to provide extra, up to date information for the shitty AI so it can function.
I have news for ya. Emergency road repairs, fires, etc are not going to make it into your world database in a timely manner so a dumb vehicle can navigate the situation. It will all be over before anyone even thinks about fixing your system for you.
Dude was totally pissed and gesturing every which way wildly. When I couldn’t immediately decode. He just started swinging even more wildly. Turning redder and redder.
Was fairly sure I was going to get shot arrested. So I just did a U turn. As I was leaving the next guy started yelling at cop that he was blocking all possible routes to his town.
It will never happen. Robots will never have the intelligence to manage dangerous situations in the open world, only in controlled environments. After a series of accidents passes the unforgivable threshold, certain levels of autonomous driving will be banned completely.
This is the same claim always given by the A.I boosters. Who sees firefighters fighting a fire and decides to just plow through and make matters worse?
Because by "significant minority" what you are saying is that 2 or 3 out of 10 human drivers do this.
Of course, software engineers get treated with kid's gloves, otherwise, everyone just freaks out and cries and cries, because their robotic utopia dreams are meaningless nonsense.
Apparently, smart enough to code, but not smart enough to take responsibility for their actions.
Assess a massive fine. Money is lifeblood of a corpo.
It's just a bit too perfect, no?
One project I worked on was a safety critical system that almost got sunk by an overly-simplistic fault response. The device would reboot if it detected any sensor readings out of range. One day someone put a bag over it while it was delivering concentrated oxygen from the air and it rebooted because the ambient o2 sensor was reading well below what anyone could reasonably expect were the machine not enclosed. The patient had to be bagged while the machine came back on.
You simply can’t predict every emergency situation and you most certainly shouldn’t assume a simplistic single-response model is enough to get the passenger to safety. Humans are really good at inferring correct behavior socially and it would be much better for an AV to watch what other drivers are doing to figure out how to handle unexpected situations. To the degree an AI-driven system is unable to do this it is completely unsuited for use on the road.
And I wouldn’t be surprised if quite a few of those who are have never actually fully understood what that means.
I certainly don’t.
But then, there’s a reason I stay away from life-critical systems.
There is no particular reason why advanced planning, modelling and pattern matching ability should need human-like goals.
Heck, even counting votes would be quite impossible (how much divergence does a copy need before it gets its own vote? What it not all modules/nodes are copied?).
Also, the anti-suffrage women seem to have been erased from history as far as popular discourse goes... guess the Women's National Anti-Suffrage League isn't taught in schools (support amongst women for extending the vote was very lukewarm, becoming a majority only after the law changed).
Because driving requires the ability to understand what other drivers are doing. It requires a theory of mind, which means understanding human goals.
I think instead of flirting with this absurd fantasy of self-driving cars we should be doing more to invest in railroads and other forms of mass transit so that people don't need to drive at all. Europe and Asia are far ahead of North America in this regard.
I'm a transit advocate, but there are some fundamental issues that need to be resolved first, especially in North America.
Of course it is trivially true that we all intend to go from point A to B. But that is almost irrelevant. It is the million micro-decisions on the trip that count.
Without seeing the human, I assess in seconds whether the other driver is unusually high or low-skills, focused or distracted or impaired, polite or rude... and this constant assessment, with human understanding, of (others') human goals and attitudes is a key element of driving skills, and yes this is basically all covered in your exception of needing to account for stupidity of people. That exception is just bigger than you think.
Like chess?
Chess is really just about searching through as much of the space as possible, and you can take a few seconds. And believe me, there's a whole lot more stupid behaviours you can encounter on the road while you have to react in less than a second.
That isn't really an accurate description of how the modern DL bots work. They don't need to reference any database of past games while they play.
As an experiment, try making a chess AI without explicitly giving it the rules of the game and see how it performs.
This reminds me of the kinds of intuitions people had about Chess and Go. In retrospect they seem silly, but it made plenty of sense to them at the time. The fact was that there was a solution that machines could use that humans couldn't use. Naturally, a solution that humans couldn't use was hard for them to anticipate being effective.
Single-track railroad like in airport terminal-to-terminal transport? Fine.
Out on the open rail network? Not even close. And that is a LOT more constrained & predictable than an open road network.
Getting to all cars being software-driven seems as much in the near-mid future as was the idea of having full vehicle-to-vehicle telemetry &communications - i.e., sounded like a great idea until it crashed on the rocks of practicality (and inter-corporate politics?); not likely happening in a relevant time frame.
The optimist believes progress will halt at a convenient moment. The pessimist fears otherwise.
We need true GAI to do this, of course, and the question is: will a "real" GAI be willing to drive a car? How much will we have to pay it?
No we will never have GAI.
---
I know never is a long time, but I can comfortably predict this because there is not only technological problems with GAI, but also philosophical problems. The way we define intelligence is such that if we find or invent something that combats human intelligence we immediately redefine it to exclude that aspect.
All that's needed is someone to achieve GAI at a supercomputer level. From then on, it's just a matter of scaling it down. Your iPhone has more compute power than a supercomputer 30 years ago, after all.
And what's also a way to "cheat" is fusing computers and organic matter. Brain-in-a-vat style, or, for yet another Star Trek reference, Voyager's organic compute network (which IIRC even suffered a pathogen infection at one point). And that is definitely possible in the next 10-15 years. Give Musk's Neuralink enough apes to euthanize and it will get done.
In 30 years we will not have an iPhone-sized device with the power of today's supercomputers. Transistor scaling is in its final stages. We'll be lucky to get 2-3 more process nodes and that'll be it.
Perhaps quantum computing will enable new kinds of inference, then we will need to scale superconductors like we did transistors in the past 75 years, that is possible (not in the next 10-15 years though; 30-40 years is more realistic). However I don’t think this will lead to anything more then amazing statistical inference. People theorized artificial neural networks way back in the 1940s and we already had algorithms for it in the 1970s. I’m not aware of any algorithms, not even theoretical ones that gives us AGI, on any computing device (not even quantum computers).
However this misses my point about AGI. It is not about computing capabilities but theory. AGI will have to evaluate situations that has never arisen before without having data that directly directs it. I don’t see how statistical inference can do that, and as such we will need an entirely new methods of training it. Like I said, we haven’t even conceived how an AGI technology works, there are no algorithms which will work in theory even with unlimited computing power. Contrast this, for example, with quantum computing, which has a bunch of algorithms demonstrating it’s capabilities. With AGI all we have is “perhaps our cars become better at driving them selves”. I’m not buying it.
According to my worldview, humans are just robots built out of DNA, proteins, etc. The idea that machines couldn't do everything a human can do (and more) doesn't make any sense to me.
I actually don’t believe in the superiority of the human mind, I think the word intelligence is actually a scientific distraction. Humans are remarkably—but not uniquely—adapted at communicating with each others and manipulating our environment for our benefit (most of the time; but sometimes to our determent, e.g. by causing a climate disaster). My statement about intelligence was not to share my believe about humanity, but a statement about human believes about our humanity (a theory of mind of sorts). I believe that humans are so human centric that we will always evaluate intelligence in human terms. As such nothing will ever supersede it as we will simply alter our former believes in order to keep our mind at the center of the universe.
Now as for machines that can do everything that humans can do (and more), I still consider this a pipe dream, even if we disregard the word intelligence. This is not because I believe the human mind is superior to machine, it is more because we haven’t conceived of how we would do this. Statistical inference has its limits, even with all the computing power in the world, it will never do what specialized machines like our brains have evolved over millions of years to do (well maybe if we allow our machines to freely evolve and procreate and set them loose in a dynamic self sustaining environment; but that wont provide any benefits for us).
I also have my doubts about the utility of AGI. People state that full self driving cars is easier with AGI, but I haven’t seen any proof of this statement (nor for the utility of AGI driven cars for that matter; we’ve had self driving trains for decades without AGI). With traditional machine learning models, there is tons of utility in the data analysis in all sorts of fields (just like there was with factor analysis and linear regression before it), but with AGI I fail to see what we will get from a vague notion of can do everything humans can do, which doesn’t apply to traditional machines that are specifically programmed for that task. In short, I don’t even think there is a market for AGI.
We apply the concept of intelligence to other species, so I don't see why we couldn't apply it to machines. If people want to redefine intelligence so that humans stay on top, then they are wrong to do so. There is undeniably an objective component to it, even if there are many cultural factors at play as well. If you don't like the word "intelligence", replace it with "competence", "productivity", "performance", etc. and the same argument will apply.
You raised two other issues: the possibility of AGI and the utility of AGI.
I think we have good reasons to believe that it's possible. Three possible future scenarios: (1) we get to the point that we can simulate brains to a high degree of accuracy in software, (2) we get to the point that we can build artificial brains out of neurons made of silicon, (3) we get to the point that synthetic biology is advanced enough that we can engineer new kinds of intelligent organisms. These might be science fiction by today's standards, but they show that there is no issue with conceivability. As an aside, the human brain runs on relatively low power consumption and operations per second, so computational power isn't a bottleneck. We have enough compute, we just don't have the right algorithms yet.
As for the utility of AGI, I strongly disagree that there is no market for it. By definition, an AGI can do anything that a human can do. There is already a job market for humans. As soon as AGI can compete with humans in domains like management, engineering, scientific research, education, etc. I don't see why there won't be a huge economic incentive to adopt it.
You hit the nail on the head here (pun intended). Brains (and other adoptive biochemical processes which reinforce behavior) are way more ingenious then to be described as a simple computer. It is not just neurons firing in patterns with synapses which can reinforce certain pathways. No, there is a whole biome in our bodies, from our guts, our muscles, through hormonal systems and yes, nervous system. Our behavior is also influenced by bacteria and other microorganisms that doesn’t even share our DNA (but sometimes actually interacts with our DNA). This complex machinery has evolved over millions of years, and I don’t think there is any way for us to simulate it. Even if we look outwards, our behavior is just as much influenced by each other as it is by our brains. So for a successful simulation you’d need to “raise” your machine in a similar manner, allowing interactions with a whole community (or more) that treats it as equal. Our brains are merely a tool which encodes this interaction as learned behavior. Just building a brain will miss all of that and give us a lousy model.
In statistics we learn that: “No models are correct, but some models are useful”. I’ve always taken this to mean that there is not always utility in a 100% accurate model. I think this applies for AGI. If we create a machine that can do everything that a human can, and better, what can we use it for? You claim increased competence, productivity, and performance, but we already have machines that do that. And we humans have found a way to work pretty well with those. We even have fully automatic vehicles that transport millions of people every day (e.g. in Vancouver B.C. and Copenhagen). It doesn’t need to be better then humans in everything, just accelerating and decelerating in the right places, it doesn’t need to know the airspeed velocity of an unladen swallow, it will only need to know if something is blocking the tracks. Traditional statistical inference will make these systems better and cheaper, and they will benefit humanity, there is no need for them to have generalized knowledge beyond their immediate utility, so they will never be given one.
Sometimes we get hung up on comparing AI to humans. I wouldn't want an AI that does math and physics in the same way that humans do. Our brains aren't very good at those things. I'm more interested in the potential for AI to find new, more effective ways of working in these areas.
AGIs by their own general purpose nature can improve other AGIs, possibly including themselves. AGIs can improve other less general purpose AIs.
Either you could ask for the AGIs to improve what you need for you, or the AGIs might do it by themselves.
Humans may be like robots but their programing is not statistical pattern matching or Boolean logic. Rather it is the intelligence selected for by evolution to achieve the goals of an individual organism.
How does this not make sense?
Intelligence does not just happen absent goal directed evolution.
As far as I can tell, your points don't apply to what I said. I'll quote myself:
>The idea that machines couldn't do everything a human can do (and more) doesn't make any sense to me.
In other words, it doesn't make sense to me when I hear impossibility arguments about AGI that appeal to something special about humans. I'm not making a claim about software and hardware in 2023. I'm saying in principle, there is nothing stopping us from building artificial systems that are just as complex and capable as evolved biological organisms, other than our lack of knowledge of how to build such systems.
>Software does not because it has not evolved
Software does evolve. To get evolution you need three things: replication, mutation, and selection pressure. We replicate, mutate, and select software all the time. I don't think you need a large scale simulation to get evolutionary dynamics.
>Humans may be like robots but their programing is not statistical pattern matching or Boolean logic
That might be true, but I'm also skeptical. Part of the problem is we don't really understand intelligence very well. If we really understood what makes humans capable of intelligent behavior, we would be able to replicate it in artificial systems. I don't see a fundamental barrier other than having the right algorithms.
And what makes you so sure the opposite won't happen? E.g. GAI will be achieved earlier by excluding some aspect of human intelligence that some people deem essential. The whole "problem" is that we can't agree on what constitutes intelligence. I predict that it won't be long before AI systems can score 200 on standard IQ tests.
I don't know of a "standard" iq test that goes past 160 or so.
ChatGPT is easily circumvented to violate all its own guardrails with a few instructions.
Knowledge without understanding.
It clearly doesn't understand it's own rules well enough to realize when it is asked to violate the rule "don't do C" that being asked to do A->B->C is the same as being asked to do C.
More or less hit the nail on the head :)
Driverless cars don't need to work perfectly all the time. They just need to injure fewer people than human drivers.
When I was on sales trip to KL in Malaysia there were times when the tropical rain could half flood a street in minutes. Our driver was able to creep around the shallow side of the road (alternating with cars in the other direction).
How would a driverless car cope? Would it even detect that there was problem? and if it detected it and wouldn't drive on that might be the worst thing to do if the flooding was going to get worse.
This is actually probably one of the best arguments for self-driving cars.
The self-driving cars will eventually stop doing this. Humans, never.
To add: a vehicle driving over a hose also means a vehicle is likely to be “in” the scene operating area. Random cars where we don’t expect them can lead to all sorts of injury.
Granted, as a truck comp guy, I often had to deal with engine guys not leaving room - but first due is first due. ;)
Words not heard often enough! :-)
In this context, the issue is actually pretty simple: it's a factor of available personnel and time. At least in the US, most municipal fire departments don't have the budget for enough staffing to run with all of their companies fully staffed (by NFPA standards). I'd posit that 3-person companies are the most common configuration you find these days, with 4-person being rare, and 5-person (or more) being all but unheard of. Ironically some volunteer departments actually put apparatus out the door with 4, 5, or 6 personnel more often than most paid departments. The tradeoff is the staffing with volunteer departments tends to be highly variable, so that same department, on a different day, may roll a truck with 1 or 2 personnel, or not roll at all.
Anyway, back to the point. Let's a assume a typical "working fire": the first due company arrives, and there is a LOT of stuff that needs to get done, and in a hurry. Size up needs to be done, ideally with somebody doing a 360° walkaround. One or more attack lines need to be stretched and put into operation. A primary search needs to be conducted unless somebody on scene can absolutely confirm that everybody is out of the building. Vertical ventilation may be required. Additional lines may need to be pulled to protect exposures. A water supply needs to be established. A RIT (rapid intervention team) should ideally be in place to rescue any firefighters who become trapped.
Did I mention that you're starting out with 3 people on that first arriving apparatus (and the operator needs to stay with the apparatus as a general rule)? And then as the subsequent arriving apparatus come in, their crews usually slot in behind the first company, doing all of the things on the list above, which are of critical priority.
What isn't on the list of critical priorities (in relative terms) is "place cones around the supply line, or other hose that happened to fall in the street".
Now by the time the 7th or 8th due company arrives, you probably have staffing for things like putting out cones, doing traffic control, etc. But they're probably coming from the other side of town, or might be a mutual aid company from another department. So you could easily be looking at 10, 12, 15, or more minutes before they roll up. So it's during that first, let's say 15 minutes, of an incident, that you usually don't have cones put put, traffic control established (PD might be available to help with that, but they're dealing with their own staffing issues), and so on. And during this time is when things like "car running over the supply line" tend to happen.
It's unfortunate, but largely it's just the reality of the way things work today, based on the staffing that's available. If every company was showing up with 5 or 6 personnel, this stuff would happen a lot sooner.
Also, at least in my jurisdiction, traffic control is a police responsibility, and we're technically not allowed to do it anyway.
The preference is anyone who's not a fire-fighter just stays away, for their safety and ours.
- You were talking to somebody really young and inexperienced, who did not themselves realize the risk of letting somebody drive over hose.
- You were talking to a prototypical "world-weary, crusty old greybeard" who was so jaded and cynical that they were thinking to themselves: "It doesn't matter what I say, this person is going to drive over my hose anyway."
- They did in fact have the ramps, that @fukawi2 mentioned, deployed. Hose ramps do make if somewhat reasonable to drive over hose, but they are rarely deployed in my experience.
Surely Cruise and Waymo have thought about this. Curious how it's supposed to work normally, and whether something about it failed in this case?
How is an official supposed to get a moving autonomous vehicle to stop or obey non-standard directions (back up to make room for a fire truck)? Is it supposed to require any special training on the part of officials, or are remote operators supposed to always be watching for it? And what happens if the regular communications channel to remote operators fails?
In that case it would also need to tell the difference between a cop and someone impersonating a cop, or it's going to be really easy to mess with self-driving cars.
That is literally the sales pitch. They're supposed to be better at driving than humans. And that actually means all of it, not just lane-keeping on well marked brightly lit highways.
Why? Couldn't it be "better at driving than humans" by reducing the most common errors (lapses in attention), while performing about the same as humans in less common situations (cop impersonators)?
The whole "sales pitch" is that it doesn't take any human effort to do.
They're supposed to be cheaper. Being as good is the limiting factor. Being cheaper is the pitch.
Transportation is literally the 2nd greatest household expense [1], and the cost is dominated by labor. I'm not sure how you could conclude that the potential for massive savings is a "toy".
Whether we want it to be or not, vision AI at the moment is somewhat an idiot-savant when it comes to interacting with the real world.
A RFID tag would do. If you prefer going dystopia, a chip implant.
And I mean given RFID is only detectable from inches to feet range, what's the point? And isn't under skin chip just RFID under the skin?
No worries, I bet they've factored the cost of the resulting fines neatly into their pricing model!
I was in a Waymo that encountered a fire engine parked in the center of an intersection. Traffic was stopped already when we arrived. We were maybe three cars back. Our light was red and no one was directing traffic. I recall the car acting a little hesitant as we came to a stop. It pulled towards the curb a bit and might have stuttered rather than coming to a completely smooth stop. Either way, the first car at the light went right on red. And we pulled up smoothly to fill the gap. When our light turned green, the Waymo waited a bit for the first car to go. Slowly crept past the fire engine in the open lane, and accelerated somewhat aggressively away. I literally whooped.
My assumption is that the software flagged the situation to an operator and they were able to instruct it in the time we were stopped. I heard from a friend that the car basically prompts the operator with a few pre-planned choices to select among — so the car is following a route it planned, but it provides the operator with options to keep the human in the loop for the tricky decision while still allowing autonomous driving execution (no remote direct driving is allowed iiuc).
No. The alert is the lights and sirens, and any driverless car unable to respond to that signal should be banned from roads. Emergency services have other things to do than draw a geofence so that Waymo can keep making money off of their collected data.
Apparently Waymo doesn't do this as much.
[1] https://www.sfmta.com/sites/default/files/reports-and-docume...
But a person standing in the way should cause the vehicle to stop, right? Pedestrian of any kind, including a firefighter in full gear.
It's hard to speculate what happened without knowing the details, but I'm imagining a scenario where someone stood there to block the car's progress, and once they realized it's a dumb robot that will proceed as soon as the path is "clear", they did the next logical thing (which was to smash the robot).
In my view, we should be worried not just about what data is missing but also what data is either unintentionally or even intentionally incorrect. One day a hacker will find a way to insert malicious data (in contrast to malicious code) and try to exploit these systems. That's just inevitable at this point. I hope that these autonomous systems end up with a lot more fail safes, since episodes like this one are only going to become more common, in my view.
You assume that a firefighter will be recognised as a person. Even if they're wearing breathing gear or carrying a sack etc. And would definitely respond appropriately to hand gestures, driving away from the fire rather than queuing neatly behind a fire appliance or something.
For all the people concerned that a driverless car might stop somewhere it shouldn't, I'm way more concerned that the car will drive on when it really should stop. Especially given some of the insanity we've seen in Auckland with the current floods, the idea that a driverless car might not recognise someone standing waist deep in floodwaters or whatever, then just drive on through, doesn't excite me.
There's too many cases where actual human drivers fail to respond appropriately even though we have 100-odd years of selection pressure encouraging them to.
https://boingboing.net/2018/04/30/zapguns-r-us.html
On the other hand, I can actually see a lot of merit in autopilot killswitches for anybody, including law enforcement.
Including thieves?
Hacks of cars were demonstrated in 2015 and documents from the CIA released by Wikileaks showed they have explored the idea as early as 2014[1], but at the time it was still presumed that killing someone by hacking their vehicle would be quite resource-extensive.
---
[1] https://www.washingtonpost.com/news/innovations/wp/2017/03/0...
e.g.: https://www.onstar.com/public-safety/emergency-situations
Or us the scary thought that it would be software/hardware and therefore insecure?
Without a current license you should not be driving.
Right now law enforcement already has a "killswitch": a traffic stop. If you don't stop at the traffic stop, the "killswitch" is a chase & spikes on the road.
A "killswitch" in which someone can just have the car pull-over seems like objectively safer.
What’s the goal? You want to steel a car? Rob the passengers? Kidnap the passengers? Kill the passengers?
So what are your options in this world where there is the equivalent of a firefighters key in elevator, but probably using PKE and periodic private key cycling?
Steal a key? From where? Buy one? From who and for how much? Ok, so now you’re needing to commit some rather risky crimes before you even move onto the next step.
And then let’s say you get the key. So you disable a taxi. And then what? I assume the passengers can lock the doors. So you smash the window open and, well, maybe it’s best to have a gun…
But why not just get a gun and approach someone stuck at a red light?
Why kind of risk/reward calculus is going on inside the mind of the criminal deciding to forcibly stop a moving vehicle and rob or murder the passengers?
So yes, what you said is true, but the likelihood is very low and the benefit to emergency responders is high.
I guess it makes for a good plot device for a spy movie.
So yes, a pissed off (and stupid) cop could do this, but then you can turn around and go to the press with the information, call up a lawyer, and proceed in any other manner where you're being harassed by a police officer.
Cue some unrelated critique on policing in general that has nothing to do with what we're talking about...
I don't think you just contact a journalist and a lawyer, and things usually work out for the best.
The problem with your line of reasoning is that you really have a problem with government abuse and that you're looking for anything that could possibly add to government abuse even in the smallest of manners. There's no scalar to your concept of abusive systems, just the fact that any system that could be used for government abuse is bad.
It leaves out the intent and capabilities of the government to engage in any kind of abuse.
If the government wanted to silence you then it would silence you like they would in any authoritarian nation: showing up to your front door, putting a bag over your head, and taking you away. They don't need any advanced technology to do so. Technologies are not the main cause authoritarianism. People are the main cause. Technologies can facilitate authoritarianism or they can facilitate liberal democratic republics.
Can a system of warranted entry on private property be abused so that police can enter your house in an unjust manner? Of course! But the presence of a system of warranted entry on private property does not correlate to the authoritarian government.
Such discussions are pointless to me.
The problem with your line of reasoning is that you really have a problem with government abuse and that you're looking for anything that could possibly add to government abuse even in the smallest of manners. There's no scalar to your concept of abusive systems, just the fact that any system that could be used for government abuse is bad.
But I'm not imagining anything because you've made it clear that your problem all along was with government abuse. It's also clear that you're looking for anything that could possibly add to government abuse even in the smallest manners because you don't talk about scale or likelihood, just a simple binary of "could be used for government abuse" or not.
I'm just logically filling in the blanks based on our interaction to come up with how you reasoned your way through all of this.
And I just went back and realized you started talking about personal cars and not taxis... what kind of cop is going to get mad at you and then... wait for you to get a cab, follow you, and then make your car pull over... and then what? Why are they doing this? Why would anyone do this?
I mean, what system is more likely to be abused, one where cops can get a warrant to kick your door down or one where cops can make autonomous taxis pull safely over to the side?
Stop being silly!
As an aside, this seems like somewhat of an issue with modern journalism. I guess you have to get the article out asap to avoid being scooped, but then it ends up missing key information, like a statement from Cruise that might have been received if they'd been given a business day to respond. If you wait though, it's old news and no one cars about the added details.
Weird how mundane and unsurprising this feels after many years of headlines about self driving. It's hard to have a lot of faith in a concept that seems so doomed for failure before it's even hit a critical market size.
Cars are not smart in 2023. They can't be because the technology does not exist, for example, for a robot to reliably recognize and flexibly respond to open ended an emergency situation in the real world the way a human can.
Maybe some situations can be some of the time but not any where near the rate a competent driver can.
The principals of so called self-driving company's should be held personally liable for accidents if the technology is so great. Because that is what happens to human drivers who commit infractions.
To quote a popular Tumblr post, "If your solution to some problem relies on “If everyone would just…” then you do not have a solution. Everyone is not going to just. At not time in the history of the universe has everyone just, and they’re not going to start now."
Human flaws are not going to go away any time soon, and you probably can't improve the overall quality of human driving on any scale less than decades. Whether or not they're useful right now, driverless cars are promising because the technology improves over time.
Article says Jan 2023 incident was active fire fighting and June 2022 was an active fire scene. Additionally that running over hoses is a violation of CA vehicle codes and "can possibly injure firefighters".
Few are willing to don the reflective vest and ruin someone's day by lying to their face and directing them down a bad detour, but I have no problem believing that a lot of people would laugh at a stupid self-driving car wasting its time and energy following their hand signals.
There are valid reasons for <random citizen> to be directing traffic, like losing their load on the highway or helping traffic flow around a stopped vehicle.
Since there's no way to take control of these vehicles, I imagine they fall back to a service center with remote drivers when they come across the unexpected. Anything involving flashy lights should fall into this category.
Driverless cars cause traffic jam at accident scene.
Driverless cars cause traffic jam while tenders waiting at traffic lights.
And, obviously, emergency stop should initiate stopping, but the vehicle should do so only when it's safe. It's not an escalator that's safe to halt immediately and at any moment.
For an completely unmanned vehicle (driving to pick up or to recharge/park for the night) - probably no big deal.
What is a realistic scenario here? Unless you are in a tank in a warzone...
How is a traditional, non-self driving car protecting you from someone pointing a gun at you? Do you have bullet-proof glass and bodywork?
Note: OP edited the post after my response. Would be good if HN indicated this.
[0]: Cherry Creek carjacking prompts other victims to come forward https://www.cbsnews.com/colorado/news/cherry-creek-carjackin...
That sounds warzone tbh.
With driverless self-driving vehicles, you have no such comfort. Even if your software correctly handles 99.9% of the use cases, the remaining .1% will get someone injured or killed, and people climbing into taxis cannot self-select whether their route will pass by a fire, in which case "take a cab with a human driver."
Driverless cars have a huge moral hazard here. Their incentives are all around serving their passengers, but the downsides are offloaded onto pedestrians, cyclists, other automobile users, and now firefighters and the citizens they are protecting.
No "solution" should "beg the question" by taking it as axiomatic that driverless cars at the current level of capability are necessary. They are not.
We are not curing cancer or feeding the world here, we are trying to make people get rich by eliminating the cost of labour for a human driver. No way this calculus should prioritize investors over human lives in any sane society.
> We are not curing cancer or feeding the world here, we are trying to make people get rich by eliminating the cost of labour for a human driver. No way this calculus should prioritize investors over human lives in any sane society.
How about, we're trying to make transportation cheaper for everyone by eliminating the most expensive part of taxi cabs (human labor)? Transportation being cheaper means people can get better jobs further away from where they live, see family more often, etc.
Before you mention public transportation, realize that it would benefit from driverless technologies as well (aka, not zero sum).
Which has been in place for years in some places (ie Docklands Light Railway).
> we're trying to make transportation cheaper for everyone
Call me an old cynic but I'll believe it when I see it - I'm old enough to have seen many, many events where the supply cost of something has gone down but the consumer cost has not (or even gone up in some cases.) Why will driverless taxis be any different?
no great loss there.
Is that a problem?
And if it is, are you implying that we cannot one day have driverless cars without imposing life-threatening externalities on citizens who did not give informed consent to participate in the testing of driverless cars?
I'd much rather be around a driverless car today, which drives overly cautious and comes to a complete stop at every stop sign, than a car driven by a human driver. So that's my "informed consent", and I suspect many others agree. Is that how democracy works?
If we cared about transportation and climate change, we'd be investing into buses, trams, trains, biking and walking paths, and other such solutions.
The attitude designation belongs with self-driving car enthusiasts, who are perfectly happy throwing out decades of safety research and progress.
It may be the opposite, as driverless cars can potentially be utilized more efficiently. If I drive to a destination, my car will sit in a parking space doing nothing and wasting space. A driverless car could drop me off, then drive away to taxi other people around. There's much less need for parking lots, freeing up valuable urban land.
It may reduce car ownership overall. I have a car because it's more convenient than public transport, and far cheaper than a taxi. If I could have a self-driving car at my door in, say, 15 minutes, and it was competitively priced, then I probably wouldn't bother owning a car. Another advantage would be that if I needed more seats, or more space, I could order a vehicle to my specifications.
It may increase utilization of roads, as self-driving cars that can communicate with each other can make more efficient use of them. Less need for stopping distances if all cars can brake simultaneously, and being able to slow down in advance reduces the impact of traffic jams.
That said, I think my own personal carbon footprint would likely fall, because most of the time I could use a much smaller vehicle. Unless I doubled the amount of time I spent on the road, I'd probably use less energy overall, as I could use a 2 seater car with half the mass.
Public transport is, of course, much more efficient; but it's also much less convenient, and at least in my country, more expensive. It costs less to run my car for a year than it would be to pay the equivalent bus fare for a year. If buses were self-driving, maybe the opposite would be true.
The whole idea of autonomous robo taxis is bad on its face for many, many reasons.
When will we grow up and admit it?
Sure, but by how much? What's the average extra overhead per person? By that I mean, what's the average distance that a taxi would need to drive between dropping one person off, and picking the next person up.
The more taxis there are, the lower the overhead, as the more likely it is a taxi will happen be nearby to someone who wants one. We're used to taxis being rare, especially outside of major cities, but if taxis were common, the overhead might be very low.
I can look outside right now, and see maybe 20 vehicles parked within 50m of my home. If that were 5 self-driving taxis instead, that would be easily be enough capacity for my immediate neighborhood, at least most of the time.
So you could likely reduce the amount of cars locally by a factor of 4 or 5, and I'd still have a vehicle less than a minute away at most locations. Yes, if you had a 5th of the cars they'd have to do 5 times the work; but likely not much more than that. In other words, the maintenance cost wouldn't significantly worsen, and may even improve; cars aren't immune to entropy while stationary.
The other advantage of having self-driving taxis is that you can more easily specialize. I have a pretty typical 5 seater car because I sometimes need that space. But most of the time, I could make do with a 2 seater with half the mass. My choice of car is determined by the edge cases, but my choice of taxi would be determined by what I needed at the time.
If the average trip is short the rob taxi does more trips and the effect is the same.
Who cleans up the car when a drunk stranger pukes in it while you are sleeping?
Who fuels it or charges it?
There are many reasons people don't share cars.
Lacking proper control software is not one.
Sure, but you also need less cars, and the cars can be more specialized (i.e., smaller on average). Two 1500kg cars travelling 10km each, might be replaced by one 750kg robotaxi travelling 20km. Overall a significant reduction in fuel for the same two journeys.
And then there's the question of whether it's cheaper to have one car traveling 100,000km over 5 years, or two cars travelling 50,000km over 5 years. My guess is that fewer cars travelling further would be less expensive in most cases.
> Who cleans up the car when a drunk stranger pukes in it while you are sleeping? > > Who fuels it or charges it?
You'd hire someone to do it? The same way it works with rental cars or taxis today. That's part of the overhead I mean: is it more efficient for companies to maintain a specialized fleet of robotaxis, or for individuals to maintain a far larger fleet of generalized vehicles?
No… they are _not_ the primary contributors to greenhouse gas emissions. Road transport accounts for less than 12% of emissions[0].
What does climate change have to do with driverless cars anyway? It seems like you're basically saying we should keep driving miserable under some false pretense that it's responsible for the death of the planet.
> It seems like you're basically saying we should keep driving miserable under some false pretense that it's responsible for the death of the planet.
And no, that's not what I'm saying. Self-driving cars are just trying to patch the problem of terrible traffic and terrible public transportation. By doing so, they aren't solving any real problem, and they will increase problems of congestion and also climate change, since they will continue to bolster cars and thus roads needed to support cars.
Again, this is zero sum thinking. We can have both efficient and safe driverless EVs as well as trains/buses run by the government. Personally, I'd prefer to take the former given how little interest the government has in enforcing laws on public transportation, and this trend seems undue to change.
I'm not sure what you mean by this. Are you talking about HOV lanes and bus lanes not being enforced, fares not being enforced, or literal lawlessness on public transit?
The problems from autonomously operating trains are limited owing to them being a separate, nearly-independent system. Anyways, some parts of the world has fully-autonomous trains with centrally-located operators on bay for overriding if anything unexpected (usually someone jumped into the rails) happens. Airplanes have significant automation built-in (and in theory can even land without input) but at the end of the day we have pilots that can override just in case that an engine became unexpectedly loose or someone on board suffered a heart attack.
The problem with autonomous cars is that a) it's specifically on a mixed-used location and b) often most of the developers want to fully abdicate any control, unlike on other systems where how operators are in standby just something unexpected happens. If both Waymo and Cruise are willing to have an operator-standby system a) might be mitigated. Until I hear that these vehicles are supervised just in case they have encountered something unexpected you will see a lot of similar nuisance issues.
Does anyone really believe that? Right now I could commute to work by taking an Uber - it costs about £10 for a one way trip.
Now you're telling me that a company somewhere, would send a state of the art car equipped with incredibly advanced computers and sensors, car which they need to insure, which needs constant communication and backup emergency controls ready on standby, for which they need to cover maintenance, depreciation and other costs, and would charge me less than £10 for the priviledge? How little exactly?
It's nonsense, that's what it is. Human taxis have pretty much reached the bottom of pricing through gig economy - you are hardly paying for maintenance and fuel at this point with the way rides are structured. What sort of possible margin is there for self driving taxis, which will inevitably end up being incredibly expensive assets until the technology becomes common place and cheaper?
This, in my opinion, is unsustainable. Soon there won't be any new "suckers" in the pipeline, and it will suddenly get a lot more expensive as these rideshare companies need to reckon with the actual cost of human labor and leasing their own cars.
Investing in driverless technology, which can be replicated and will eventually become cheaper as it's mass-produced, is a very smart move. These costs will eventually get passed down to consumers as well once there's enough competition, and then everyone wins.
No, public transportation has completely different technological needs.
Is this going to make people's lives better (shorter work week and/or better pay) or worse (longer commute and expectation that you work in your car)?
Yes, there are a lot of very good use cases. And honestly, I have a far less important use case of my own: If I take a taxi (usually when travelling) I just want transportation, I do not want someone chatting me up in an effort to earn a good review/tip because they don't earn enough from driving the route silently to get by.
I just don't want progress lubricated by the blood of people who did not give informed consent to assume the risks.
I agree. If we both happen to be correct, I wonder in what other areas of automation we may be greatly over the tips of our skis.
Other than working and sleeping, driving is the main thing adults do with their lives. Not having to do it anymore would be transformational. I'd prefer to see that accomplished through urbanism and public transit, but that's about as much of a moonshot politically as self-driving is technically.
"A software handling 99.9% of the cases" wouls be a significant advantage over cars with drivers if it was true.
I think it is just a numbers game, we may not be there today but I believe we will have the a future with safer roads thanks to driverless cars. I am longing for that
So we should push for it instead of trying to shut it down. Solution to the problem in article sounds easy, allow law officers to mark roads as blocked so the car does not even to see the fire truck. Or just put a road signal or a person on the road so the cars (driverless or with drivers) don't step on your hose
> fire hoses on the ground in the area of active firefighting
It sounds like there wasn't.
I don't know why firetrucks show up on medical 911 calls or why fire departments run the lifeguards. But they do all those non-fire things and even more.
i guess it would cost a lot of money, and prob be very wasteful, wreck privacy even more, etc.
but... i wouldn't mind knowing a bit more of what's going on out there. now that our streets are even-more deadly beta robot playgrounds.
and, with all the new tech, maybe we can learn some things.
Thalidomide was considered safe and effective for pregnant women until it wasn't. You want to be a guinea pig, be my guest and while you're at it, go ahead and install alpha builds for your BE infrastructure.
Edit: the fact that this was down voted speaks volumes to me about how people view safety. Sad!
Not saying that is right or not, just surprised this is apparently an issue worth smashing a car up for. I have no recollection of this being in the highway code for instance
California Vehicle Code § 21708: "No person shall drive or propel any vehicle or conveyance upon, over, or across, or in any manner damage any fire hose or chemical hose used by or under the supervision and control of any organized fire department."
I dunno man, that's an enormous amount of downside to take in exchange for avoiding the minor inconvenience of just finding a detour so as to not drive over the hose.
In reality, yes, I'd back up and find another way, if the hose appeared to be in use fighting a fire. I wouldn't want to cut off the water even momentarily. But that falls into the category of "Not being a dick behind the wheel," rather than the category of "Obeying the law." If we don't want this to happen with self-driving cars, then we need to change the law.
Speed bumps - which are everywhere - are allowed to be up to 100mm: https://www.legislation.gov.uk/uksi/1999/1025/regulation/4/m....
I drive over speed bumps all the time and they're more than twice the height.