Why emergency braking systems sometimes hit parked cars and lane dividers
arstechnica.com
arstechnica.com
>Not only might it take much longer to arrive than the company has ever indicated—as long as 30 years, said Urmson—but the early commercial versions might well be limited to certain geographies and weather conditions. Self-driving cars are much easier to engineer for sunny weather and wide-open roads, and Urmson suggested the cars might be sold for those markets first.
Urmson put it this way in his speech. "How quickly can we get this into people's hands? If you read the papers, you see maybe it's three years, maybe it's thirty years. And I am here to tell you that honestly, it's a bit of both."
He went on to say, "this technology is almost certainly going to come out incrementally. We imagine we are going to find places where the weather is good, where the roads are easy to drive — the technology might come there first. And then once we have confidence with that, we will move to more challenging locations."
https://web.archive.org/web/20160422140357/https://spectrum....
Using US fatality rates you would expect ~34 deaths from that. https://en.m.wikipedia.org/wiki/Transportation_safety_in_the... Edit, my numbers where low.
So, as far as I can tell they are close to human safty though perhaps not up to defensive driver in good conditions safe.
If you have better data than I would love to see it. But, IMO using ballpark numbers is much better than simply having a bad feeling about something.
These cars are not going to fail the same way people do. Computers really fuck up in broad daylight, people get drunk and drive down highways in the wrong direction.
Right now I would trust this over me driving tired or drunk but not sober. Give it 5-10 years and better hardware and I think it will be a better driver than I am.
That's probably because Tesla is liable. And it could be that at this point they don't mind operating at a loss in this area.
That's ridiculous, they might base this on some other benifit such as type of owner, but the industry is not just going to subsidize 100,000+ plus people. I am not saying it's fool proof, but your bias is showing.
By the way, the black box recorder in a Tesla might also contribute to the discount given by insurers.
It will be an interesting question once systems are on the road that claim to be fully autonomous--and no one's sure how this will work yet. If I'm sitting in a car that's sold (and regulated) as completely hands off I'm certainly not going to accept liability if it does something dumb and kills someone.
I wonder if it’s mainly another way of getting access to telemetry data? Insurers definitely do give you a discount if you install a recording device. I assume that lets them filter out some bogus insurance claims. (Or maybe people just drive better when they know the insurance company is watching!)
- Tesla’s cars are newer and more expensive than the average car on US roads (making them safer)
- Tesla’s Autopilot mostly, if not only, gets used on roads that are safer than average.
- Tesla drivers may have had to intervene a lot to prevent more deaths.
- not all those Tesla’s ended up on US roads (probably not something with large impact)
The trend of things machines can do better than people really only goes in one direction. Further people are vastly worse at driving than they think, over 1.25 million people die per year due to car accidents. That's insane.
On top of that, when you factor in the utility of driving a car, the risk/reward ratio looks stellar. Humans are better drivers than given credit for, and it will be some time (much longer than a lot of enthusiasts will admit) before computers manage to match that.
That's one of the top killers anyway you slice it and every single one of those deaths is preventable.
PS: Truckers drive up to 3,000 miles per week * 48 weeks per year * 47 years (65-18) that's ~6.7 million miles. US death rate is ~1:88 million miles on average. So if they where as dangerous as average that's ~1 death per 13 truckers. However, they are safer than this and average less miles. Still, it's one of the most dangerous US jobs.
And every single one of them is a preventable tragedy.
I don’t see how that’s an argument in favor of letting today’s self driving cars loose on the road.
We don’t let people without any driving experience drive cars either because, if we let them, they “will improve soon”.
Or are you suggesting letting self-driving cars loose now is a necessary sacrifice we have to make to make progress?
Yes we do, that's exactly what a learner's permit is for.
If you're measuring whether the actual experience of using autopilot meets the prevailing standards by which we judge "safe enough" as a country, you don't want to correct for those first three.
Especially the "the cars are new and safe overall" factor. Correcting for that is like "correcting for" the fact that all the dangerous lithium ion cells are carefully packed and armored, and coming to the conclusion that sitting in a Tesla is less safe than sitting in a bonfire.
It makes the arguments a lot worse when people don't distinguish those two questions.
If (and that’s likely) other features than Autopilot contribute to Teslas killing and injuring fewer people, we must compensate for them to judge whether Autopilot is a net benefit.
All my line of argument is trying to adress is how long until self driving cars are 'ready' which directly relates to how capable they are today. The data I looked at suggested somewhere above 'drunk driver' but below 'school buss driver'. That seems like an extreme range, but they are each human levels of competence. Which is different than they are being portrayed and IMO a sign they are very close to ready.
While the points you bring up are meaningful, we let people drive without those safty features. So, in terms of policy it seems like an ever higher bar, as for example automated breaking systems and lane following raise the bar above human competence, but again are not being installed on older cars.
Which... doesn't really make sense. The safety features are part of the package. You can't get the dangerous part without them.
No cars are sold with a big blob of naked unprotected lithium ion cells. And no cars are sold with autopilot but not high-end crash protection.
A feature being dangerous in isolation is not enough to show that it lacks net benefit.
They stopped giving mile driven statistics after that.
Public transportation sucks.
When people trot out their arguments against public transportation they always refer to the inefficiencies and completely ignore the human element. No one wants to be crammed in a small space with a bunch of strangers every day.
No one wants to be stuck in bay area traffic every day either. Public transit is used for a higher percentage of journeys in Europe though because it's a better option relative to the available alternatives than it is in much of the US. If we had free teleportation I imagine you'd see very few people using any other form of transport.
There is probably a degree to which public transit usage is higher in Europe not because it is a better experience than it is in the US but because driving is a worse experience. Based on my own experience it's a bit of both but clearly usage levels vary around the world due in part to varying tradeoffs between transport options based on historical, geographical, political and economic factors. There may also be a cultural element as well where preferences vary but I don't think that's the biggest factor.
I take 15 minutes crammed in a small space with a bunch of strangers over one hour of solitude in a small metal box with windows any day.
Nobody has managed to explain me how self driving cars reduces congestion. It is space that is the limiting factor, and I do not understand how autonomous cars reduce the need for that.
Vice versa, rudimentary economic analysis says that if you do not need to concentrate on driving, you can spend more time in the car, thus sitting in the traffic is not that bad and more people are willing to do that -> more congestion.
I am reasonably sure economics will change that.
> Nobody has managed to explain me how self driving cars reduces congestion.
It doesn't, but it allows the work day to begin at commute time. Those that don't adapt will simply have fewer "work hours" to make up the lost productivity.
I suspect the bumpiness of roads will become a public nuisance once people start reading/computing in-car.
Why those durations? I would expect a typical car trip to be faster than transit, since it takes a direct route with no stops.
If transit is even as fast as driving, I'll take it 90% of the time. But that's quite an undertaking.
Compared to buses on the same roads (i.e. without a dedicated lane), sure, but not to subways/light rail.
Even if it works for just highway driving in good weather--which, in spite of overall skepticism about timeframes seems relatively close--it's hard to see how it doesn't put a lot more cars on the roads. I can't believe I'm unique in saying that I'd go into the city after work or head into the mountains for a hike more frequently if someone/something else were driving me.
Operating costs are still a constraint And those are higher than a lot of people assume if they don't really think about it. But easier driving will absolutely put more cars on the road.
I would prefer a bus or train to a private car any day, assuming it's reasonably clean and not overcrowded.
Being on a bus/tram/train with strangers never really bothered me, to be honest. When I walk down the street or onto a shop, I’m also surrounded by strangers; what’s the issue?
If your buses aren’t popular with old people, you have bad buses.
Waymo almost certainly will launch this year a fully self driving service, accessible to anyone, "within parts of the Phoenix metropolitan area, including Chandler, Tempe, Mesa and Gilbert" (https://waymo.com/apply/faq/) which is where their pilot program has been running for several months now.
See e.g. https://arstechnica.com/cars/2017/10/it-sure-looks-like-waym...
They already have at least 600 cars in that area and already announced plans to buy 80k cars.
So if by "mass" you mean "millions of cars available world wide" then yeah, maybe not 20 years but it'll take a decade to scale this world-wide.
But to me "self driving works in practice" will be validated within months of Waymo launching in Arizona and then they'll enter a phase of dazzling scale up to other cities.
For comparison it took Uber 7 years to outgrow taxis and I expect self driving to grow a bit faster.
Once the technology works it's just a matter of capital expenditure and physical limits of how fast you can scale car manufacturing.
It'll take many years to deploy this world-wide but I don't think this is the standard most people use when they determine if self driving technology works or not.
I think the difficulty of driving in Phoenix suburbs is maybe 1% of the difficulty of driving in urban areas, anywhere with snow and rain, etc.
"Phoenix is the capital and most populous city of the U.S. state of Arizona. With 1,626,078 people (as of 2017), Phoenix is the fifth most populous city nationwide"
Explain the criteria you used to classify it as "suburbs".
Phoenix is the first metro area to launch the service.
Waymo (and others) are testing in other places.
From https://waymo.com/ontheroad/: Kirkland (lots of rain), Bay Area (including San Francisco), Detroit (snow during winter), Atlanta, Austin.
I don't see how you arrived at the 1% and 1% of what?
If it took Waymo 10 years to drive in Phonix, Arizona, it'll take them 1000 years to improve enough to drive in Detroit?
Those things are pretty binary: either they can drive autonomously or they can't.
GP's assertion that driving in Phoenix is easier than most other places is absolutely correct.
I don't dispute that Phoenix is easier to drive than San Francisco or Detroit during winter.
It's perfectly rational to debut such service in the easiest possible environment.
I just don't think that it's a fundamentally different problem to make this work elsewhere.
This technology already improved by leaps and bounds. At first those cars couldn't finish a drive in the desert, with no other traffic.
I don't have an inside knowledge on this but the fact that they are already testing the cars in more difficult areas indicates they are working on more difficult problems.
It's anyone's guess how far they are on that front.
My guess is that after launching publicly in Phoenix, it'll take less than a year to launch publicly in San Francisco.
It is one thing to have broad streets, well marked, with multiple levels of marking, e.g., stripes, plus curbs, plus medians/sidewalks, plus trees/shrubbery. All of these well-designed and well-built modern road features cooperate to make a consistently recognizable environment. And the lack of serious weather is a big deal too.
Contrast that to a city like Detroit or Boston, where the streets are anything from modern to ancient, literally paved over the cow-paths, constantly changing with construction, lucky if the lane markings are still visible, pedestrians in all kinds of odd situations (legal and illegal) -- orders of magnitude more difficult to sort the environment. Now add snow in quantities enough to make it often difficult as an experienced human driver to figure out where you are in the lanes, and then snowbanks in odd places after it is cleared'.
Sure, it's still 4 wheels, power plant, steering wheel, roads, but two quite different games.
Going for a walk in Central Park at lunch and hiking up Denali in Alaska are also ostensibly similar activities, but in actual reality, are very different games.
>...I don't have an inside knowledge on this...
So, based on your complete lack of knowledge on the subject, you think that the current solutions will easily extend to more difficult applications? I work in a related field (image/signal processing and ML) and I can assure you that there is no such thing as a general solution here. It is absolutely reasonable to expect that your 70% solution may simpler never work for the remaining 30% and you have to go back to the drawing board.
Yes, but that same source for population indicates that Phoenix is only the 169th most congested city in the U.S. It's suburbs are presumably even less congested.
They've also done a lot of their testing in the SF Bay Area, which is a pretty difficult urban area to drive in.
They're obviously launching in Phoenix because it's the easiest major city to drive in, but they're also obviously going to expand to other places once they get successes in Phoenix.
Honestly, to me, the more interesting question is: As self-driving cars slowly expand to other cities, will other cities intentionally start making their lane markings more obvious and doing other things to make it easier for self-driving cars to drive in?
Sometimes, shuffling around blame can cause things to get fixed. Previously, with bad lane markings, you could blame bad drivers for accidents. But with self-driving cars, you can only blame the software. So if a self-driving car company is willing to serve one city with good lane markings but not another city with bad lane markings, suddenly the city government gets the blame for bad lane markings.
The more complex things that can happen in urban areas though, yeah that's hard. All kinds of weird shit can happen that takes social understanding to know how to deal with. Right now it looks like for those situations Waymo is relying on remote "coaches" that tell the cars what to do at a high level (guidance, but not directly control).
Imagine cars that refuse to tailgate, drive at excessive speed, drive with drowsy owners, drive on the right-hand side of the road in non-emergency situations, skip red traffic lights, etc...
If Tesla could develop a network of self driving buses, that would have far greater reach and impact than trying to solve how to make a private network for charging and all case-considered self-driving system for asshole-mobiles. Not that Tesla cars aren't amazing, but they seem like a giant waste of resources for self-centered people. Not to mention in situations like self-driving, buses have completely predictable paths and because of that, it's much easier to optimize for corner cases to work toward a general solution.
With the exception of the handicapped, people don't really need cars. Bikes/e-bikes work great for any type of commute <30 miles and hauling cargo <100 lbs with a trailer. Electric skateboards are also great if you have little to carry and <12 miles to go. Beyond that, self-driving delivery fleets in the vain of UPS and buses make sense. There's really no valid reason to use a car unless you're an invalid.
But guys like Elon Musk are happy to burn the earth to the ground to terraform Mars instead of working on how to terraform places like Arizona or Pakistan because it looks better to them on paper. Or so it would seem.
Only when they aren't. Winter means snow, ice, and below zero temperatures and my hands simply can't handle the extra cold from the wind. I'm not handicapped and do walk during the winter, but tend to take the bus on the very rainy and very cold days despite dressing for the weather. let alone the fact that I'm no where near skilled enough with a bicycle to feel safe riding on 2inches of ice - regardless of studded tires. Summers have really strong sun, making folks deceptively warm and sweaty. I dont' know about your job, but most of mine want me to be clean and not smell of sweat when I arrive. I always have the chance of cargo: I walk to the grocery store most times. (I am not legal to drive in my present country).
When I lived in Indiana, there were not only problems with actual weather, but then I ran into route problems. The easiest route for commutes was major roads, but unfortunately most of those roads aren't built for people to ride bikes or walk. In some places, doing either activity on those roads meant you got stopped by the cops and told to find another path to take. You were putting cars and yourself at risk using that road. Luckily, there are more paths for such things here.
Not only that, but e-bikes have a limited range in most circumstances. Higher-end ones go far enough, but some of the cheaper models risk losing power at the higher ends. By the time most e-bikes has power to spare on those commutes, here they might be legally a scooter.
I do agree with having a robust public transportation system, however. That actually soves the issues with the bikes, but bikes alone seem won't work.
And yeah, not having bike paths makes it a whole different story. Drivers can be very irresponsible regarding the safety of cyclists (especially during inclement weather). I would personally be very weary if I had to bike exclusively on the street. Where I live, I can get pretty much everywhere without touching a single street thanks to all the dedicated multi-use / bike paths. In fact, the reason I purchased a e-bike and decided to try to go car free was the installation of a bike path that parallels the major highway here. But all of the bike paths here are a response to how many people here are cyclists and the dedication of the community to take steps to fight climate change/pollution. Though places like Indiana may never have a good bike system without some sort of federal intervention if they are really that anti-biking, but there are much better places to live if you like to bike. Riding around the Denver/Boulder metro area is an absolute hoot and there are places that are even more bike friendly.
Living in Colorado I bike to commute in both freezing conditions and 100+ degree weather conditions without catching a sweat on my e-bike. I'm not some hyper fit guy, I've got a dad-bod and long hair (along with a beard at the moment) and I generally run warm. But after I settled on trying to use the e-bike I got as my primary means of transportation, I started solving some of the problems you discuss.
For instance during the winter, I wear a black down jacket (with a massive backpack that keeps my back extra toasty), a insulated black balaclava with ski goggles, ski gloves if it's extra cold (otherwise just some simple neoprene gloves), long johns and thick wool socks and to combat the wind I ride with my hands in my jacket pockets (though that's admittedly only possible because of the long open curves of the bike path and I never have to make any surprise emergency stops). You might think that I'd be sweating like crazy in all of that, but you have to remember, with an e-bike, my effort for pedaling is similar to that of riding a beach cruiser down a flat boardwalk with a tailwind regardless of wind, hills or cargo and my layering is generally designed to keep me comfortable. At most I will unzip my coat to ventilate my torso. And when I get to my destination, I usually just stop off at some restroom to slip off my long johns - sometimes I'll also switch my shirt.
Also on the note of snow, while I wouldn't recommend it as it's super sketchy and totally unsafe, I've found that getting around in even a few inches of snow on the ground is weirdly doable with an e-bike if you have it in throttle-only mode. The reason being that you can keep your body still and work on maintaining your balance while letting the motor do all the work. It works so well because you don't have to worry about unpredictable tire slippage causing you to throw your weight. An e-bike tire can slip all it wants and I'm still just going to be standing over my bike in a track stand while maneuvering the front tire. Plus with mountain bike tire, you get better traction on snow or mud so it works even better than my track bike.
During the summer, I wear white UPF rated compression tights under highly breathable hybrid shorts and a UPF t-shirt. I wear white nylon gloves, and a thin moisture wicking white balaclava with sunglasses. Do I look like a kook? Sure. I get a lot of weird looks. But I don't mind, the white outfit not only protects me from sun burns, it increases the evaporative cooling of my sweat as well as reflecting most of the light so my skin isn't really being heated by direct radiation. The result is I can get around quite comfortably regardless of the heat outside. And during the summer, it's even easier to carry an extra shirt or pair of short to switch to into a non-ninja-esque outfit. Plus, despite the weirdo factor, it's at least a little fun to go into a bathroom and come out wearing tights like Superman.
Regarding range, that's one of the cool thing about e-bikes, the one I bought for $1300 and most bikes I've seen over $1200 have a battery that can be unlocked and easily removed so I just bring the charger with me (not much larger than a power brick for a gaming laptop), stick the battery in my bag and then just plug it in where ever I'm stopping. The battery fully charges in 4 hours so assuming I don't need to go more than ~25 miles in a 5 hour window, the e-bike works perfect. (Though I will say that there definitely is a big power output drop off when the battery gets under 30% but I rarely get mine under 40%)
The big thing bikes have over buses is convenience and price. Much like a car, you can anywhere you want and because of the speeds (up to 28 mph for class 3 or 20 mph for class 2 motors) my commutes are generally faster than by bus, and I'd say on average, only 40-50% slower than taking a car which might sound high, but that's usually only an extra 10-15 minutes plus I get the fun of riding a bike around. Not to mention in areas where finding parking is difficult, riding a bike can actually end up being faster than driving.
Buses are definitely part of the equation, which was the point of my original comment, but that shouldn't negate bikes as a preferable means of transportation. Especially if you have a situation in which you can get away with commuting by traditional bike rather than an e-bike. They're the most energy efficient standard mode of transportation available to my knowledge.
https://truecostblog.com/2010/05/27/fuel-efficiency-modes-of...
I'm a big fan of the added basket on the back of the bike - so there is no need to put anything on my back.
What country do you live in? We don't all live in downtown SF. In fact, most of us don't. We live outside the city and drive 20 miles to get to work (you're nuts if you think most people are going to spend hours commuting 60 miles on a bicycle every day). And there are no bike Lanes, let alone paths And it snows.
>There's really no valid reason to use a car unless you're an invalid.
You're on another planet. It's interesting that you talk about finding practical soltutions to "real problems" yet seem to lack any notion of what the real problems are.
Yeah, I could get an e-bike, I just have to make 5x+ the trips to the store since I doubt it could hold what I can get in 1 trip now. How would I drive 20 miles one way to work in the winter when there is zero public transportation here? There's no bike lanes out here, and taking an e-bike on the road will eventually get you killed, that's if you can stand driving 40 miles round trip in the snow / rain 5 days of the week.
I guess I can move, (as been suggested before lol) but now there's a lot of new problems with that that an e-bike won't touch.
It just isn't feasible for most people yet. Hopefully sometime soon, but definitely not right now.
I've personally seen a few "self-driving cars" on the streets of SF and I suspect many of you reading this have as well (particularly if you live in the Mission or South Park).
Just last year I was talkng with engineers at the self-driving car company "Cruise" which had a free internal app to taxi employees around with a self-driving car. I personally saw the tech demoed. One of the engineers called one up to go drinking the weekend prior (sans drinking buddies - company policy). He claimed his coworkers come to work in them occasionally. The cars and engineers could be a grand charade but it seems like "self-driving cars" are already used by non-daredevils every weekday.
I learned to drive on backroads of rural Colorado; It may be these cars are safer than I in the sometimes "adverse" driving conditions of San Francisco.
Is this really how a distant technology looks?
That can be how a distant technology looks, if the stakes for failure are that people die.
You can get early adopters from one tail of the bell curve, hype, working prototypes, and people dying because prototypes fail. There may be a churn of companies... "We know that AlphaDrive and BetaDrive both killed people but we here at GammaDrive want to reassure you that we're totally committed to your safety!" "But that's what BetaDrive said." "Yeah but we REALLY MEAN it this time." "Wait, what to you mean, 'we'?" "Um, nothing..." "You hired BetaDrive's employees when they went out of business, didn't you?" "Uh... Oh would you look at the time!"
With that said I don't know Cruise and maybe they're doing better than Tesla.
And looking at cruise, their website is low grade marketing. Mostly concerned about hiring and not about pushing their actual technology. it's amateur hour honestly.
It sounds like most self-driving systems have several systems that relate to each other (which each, at least implicitly, have a world model). It's hard to know if they should instead have a single world model that all information is extracted from or whether they should improve the integration of their existing systems. I am think you show humans tend to have several world-models at different levels of exactness when dealing with the world (which doesn't prove a car should have that - complete consistency seems to have advantages here - seems is the word).
Perhaps one could say - "despite various kind of progress, it's not obvious what direction to take for full self-driving cars".
BTW, a bit of a plug. I'm running a nonprofit/public dataset project aimed at increasing safety of autonomous vehicles. If anyone here wants to contribute (with suggestions / pull requests / following it on twitter / etc) - you'd be most welcome. Its: https://www.safe-av.org/ . I'm just starting it/trying it out - simple feedback or following the twitter handle would already be helpful.
[1] https://arstechnica.com/cars/2018/04/tesla-autopilot-crisis-...
A sponsored article?
"WARNING
"When the adaptive cruise control is following another vehicle at speeds in excess ofca 30 km/h (20 mph) and the target is changed from a moving vehicle to a stationary vehicle, the adaptive cruise control will ignore the stationary vehicle and instead select the stored speed.
"The driver must then intervene him/herself and brake."
One of the most difficult things to do in a race, is to go fast/drive at your best the moment the flag goes green; and here it appears to me that we have people who don't like to drive, so therefore they likely aren't very good at it, and suddenly they are faced with a life and death crisis in which they NOW have to save their life by starting to drive the car when they are not even warmed up and they have very little room for error... I mean do the math.
Ultimately what you’re keying into is skill and interest - most people aren’t very skilled at driving and they don’t really care to be.
Not sure why you think so. If this is correct, there should be many other equivalent or better campaigns currently running at the same scale, right?
On the contrary, I imagine that data is a very significant limiting factor in this field.
Both data and computation are hard problems. AFAIK Tesla should still be leading in the former category by virtue of the number of its data collection devices. Crucially, they are collecting training data (real world human input) with which they will be able to train and test models.
Note, I am not saying that Tesla's current models are good, or that they will easily improve them. Only that you are too dismissive of the potential advantage that Tesla's data affords them.
edit yes, personally I'm very impressed by the size of the campaign. It is just that again, there is a spectrum of difficulty between running a campaign even of this size and running it right, in a way that is actually useful to building a product. And still, in my opinion, building an autonomous vehicle is much harder in comparison, to running a campaign of that size in a right way.
I’m pretty sure those outnumber Tesla.
They just ordered some 30,000-ish cars, so it would seem they're entering the mass-production phase.
Source: used to work there
[1] https://www.cnbc.com/2018/02/08/a-tesla-self-driving-blind-s...
Source: Watched a presentation on the computational aspects recently.
They built runways and carved wooden headphones like they saw the aviators wearing.
Here's a summary from Wikipedia:
With the end of the war, the military abandoned the airbases and stopped dropping cargo. In response, charismatic individuals developed cults among remote Melanesian populations that promised to bestow on their followers deliveries of food, arms, Jeeps, etc. The cult leaders explained that the cargo would be gifts from their own ancestors, or other sources, as had occurred with the outsider armies. In attempts to get cargo to fall by parachute or land in planes or ships again, islanders imitated the same practices they had seen the soldiers, sailors, and airmen use. Cult behaviors usually involved mimicking the day-to-day activities and dress styles of US soldiers, such as performing parade ground drills with wooden or salvaged rifles.[14] The islanders carved headphones from wood and wore them while sitting in fabricated control towers. They waved the landing signals while standing on the runways. They lit signal fires and torches to light up runways and lighthouses.
It should be noted that no AI system has anything like that. A model of the entire world is beyond current technological capabilities.
I think we humans are so good at just passively keeping track of everything in our environment we forget how difficult it is "keep a world model" from what one sees.
Sure it is. Whether it's an adequate model for some particular purpose is another question. But no model includes everything.
Objects and velocities would be a good start. I don't know enough about the present state of the technology to know whether we even have that. But from some of these accidents we're seeing, it seems like we don't.
I would agree that a better model would attempt to infer intentions: what is that driver/cyclist/pedestrian likely to do next, based on what they're doing now?
Current tech builds a (statistical) model of some aspects of the world that an agent must navigate- aspects that pertain to entity types, trajectories, velocities and so on.
But other bits are missing and they are important ones- for example, what is a human doing on top of a bicycle moving at the same speed as him or her? What is the relation between a driver and a car? And so on. We can still not represent stuff like that with any accuracy to speak of.
In short, what's missing for the most part is a representation of the relations between entities in the world- even where we can accurately represent the entities themselves, or their characteristics.
However, a "world model" should include the relations between identified objects, in some representation that can then be used for inference.
For example, a classifier can label an object in the path of a vehicle as, e.g. "tree", but a world-model would have to go further than that and provide some context about trees, why they don't normally grow in the middle of roads and why they should be steered around or otherwise avoided.
This kind of world model, that places entities in the world into a complex relational context, is, indeed, beyond our current capabilities (at least for the real world, as opposed to controlled artificial worlds).
I really don't think that's controversial. However, I got a few downvotes on my previous comment so I must have misunderstood the comment I was replying to.
There's a naturally impulse for a human being to look at a problem and say "of course you need a world for this and it will have X, Y and Z data and it will be give you all you need." That is until they figure out how it is to keeping X, Y and Z updated is.
Here I was thinking it was silly gimmick that I’d barely use. The self-driving Waymos of the future may all work perfectly because if it.
It's like planes which run on autopilot -- there is still a pilot. In addition to takeoffs and landings, the pilot is there to adjust. Self-driving cars seem to be similar -- hands off 95% of the time (and not 100% as some might expect), but that 5% that they are hands on are critical.
It would drift towards the next lane, and while not at the exact last moment - it would steer back slightly after what I would have done. Not confidence inspiring. If it had been an option, I would not have paid for it.
My guess here is that the lane assist and adaptive cruise systems somehow thought the car it was following switched one lane to the right, and so the Tesla assumed it was on the most left lane, without any moving car in front (hence accelerating back to 75mph). But what it thought was the most left lane was actually the space between an hov exit and the real most left lane.
What's amazing is that, prior to that article, I had no idea that collision avoiding system at high speed would ignore any stationary object, even if it's right in front of you... This changes dramatically how I'll perceive such assisted technology from now on.
This seems like how I would imagine a "dumb" system would be designed. One would imagine a "artificial intelligence" (of any sort at all) would find a way to smoothly integrate all these system. The problem is end-to-end neural nets or similar devices intended for such integration don't seem ready for prime time.
The problem the article points out is that there's no safe, dumb response to radar seeing something stopped right in front of a car going 70 mpg.
If anything, the systems the article describes are a lot like what you describe - a bunch of semi-separate systems each of which can do something if it can be sure that something is safe and which do nothing otherwise.
The problem seems to be that given enough no-easy-answer situations, you wind-up with a false sense of security.
Yeah, they all put it in the manual, but who reads those?
If you don't know how to e.g. turn on the hazard lights in the vehicle you're driving without searching extensively or digging out the user's manual, you've failed to be a responsible driver.
If I got in a loaner car with semi-autonomous features there's no way I'd drive away without first paging through a user's manual or online guide understanding those features first. especially given how many crashes have occurred due to apparent misuse of these vehicles.
Perhaps, but then almost everyone is negligent. The car designers (and the whole world) know that well, so to depend on people reading the manual is disingenuous and dangerous.
I actually find it something of an issue with rental cars these days. Obviously the foot pedals, turn signals, etc. are well enough standardized that it takes very little to accustomize yourself with them. But environmental/entertainment controls? And then there's the fact that things like backup sensors may or may not be available in a given car even though you tend to start depending on them a bit if you have them.
It surprised me to see it covered, let alone how thorough their guidelines were. Some things I recall from their pre-flight checklist:
- check general tire condition and pressures in a walk-around
- locate the spare tire and jack
- familiarize yourself with the location of pedals, and controls for turn signals, hazard lights, and the emergency brake
- adjust seat and mirrors
I didn't see the priority of locating the spare tire and jack before driving, but the others all seem like a good responsible bare minimum.
People don't seem to generally appreciate the gravity of operating heavy machinery capable of high speeds on shared public roads.
(Unless it's because they decided to print the same thing in 200 different languages. I've seen that with some other equipment --- surprisingly thick manual, but a tiny fraction of it is actually in English.)
So if current systems cannot or will not acknowledge objects not in motion even if they are in the path of travel why is the NTSB allowing them on the road? Why not force the issue and require it. It really seems that far too many bought into images of what marketers promised by interpreting them to be akin to what is seen in movies and TV shows (enhance!).
This isn't the only recent case where Autopilot steered
a Tesla vehicle directly into a stationary
object—though thankfully the others didn't get anyone
killed. Back in January, firefighters in Culver City,
California, said that a Tesla with Autopilot engaged
had plowed into the back of a fire truck at 65mph. In
an eerily similar incident last month, a Tesla Model S
with Autopilot active crashed into a fire truck at
60mph in the suburbs of Salt Lake City.
A natural reaction to these incidents is to assume that
there must be something seriously wrong with Tesla's
Autopilot system. After all, you might expect that
avoiding collisions with large, stationary objects like
fire engines and concrete lane dividers would be one of
the most basic functions of a car's automatic emergency
braking technology.
• Pedestrians shorter than approximately 3.2 ft. (1 m) or taller than approx- imately 6.5 ft. (2 m)
• Pedestrians wearing oversized clothing (a rain coat, long skirt, etc.), mak- ing their silhouette obscure
• Pedestrians who are carrying large baggage, holding an umbrella, etc., hiding part of their body
• Pedestrians who are bending forward or squatting
• Pedestrians who are pushing a stroller, wheelchair, bicycle or other vehi- cle
• Groups of pedestrians which are close together
• Pedestrians who are wearing white and look extremely bright
• Pedestrians in the dark, such as at night or while in a tunnel
• Pedestrians whose clothing appears to be nearly the same color or brightness as their surroundings
• Pedestrians near walls, fences, guardrails, or large objects
• Pedestrians who are on a metal object (manhole cover, steel plate, etc.) on the road
• Pedestrians who are walking fast
• Pedestrians who are changing speed abruptly
• Pedestrians running out from behind a vehicle or a large object
• Pedestrians who are extremely close to the side of the vehicle (outside rear view mirror, etc.)
This isn’t fluffing up phone features, it’s presenting a system in such a way that key, known deficiencies that maim or kill are buried.
The people impacted by this trusted the system based on manufacturer claims, which unfortunately were and remain horseshit.
As to whether this article is really a stealth advertisement for GM's Super Cruise technology, well... yeah, it probably is. But I still found it informative as to the current state of driver assistance systems. I was not aware that the cruise control, lane keeping, and automatic braking systems were generally completely independent of each other.
Well, that point is most likely lost on someone who dies in a crash of these cars. IE, the difference is big from some viewpoints but irrelevant from others.
I think we can see the point where objections to self-driving cars start to become strong. It's not some cooked-up version of the "trolley car problem" but a situation where a self-driving car does something a human observer naturally interprets as "wrong" and "dumb", does it in a way that kills someone and the designers give the answer "sorry, not a mistake, just a necessary design compromise. This is still, on-average safer than a human". I think this would produce a strong emotional effect regardless of any "generally safer" assertion.
A person is dead mainly due to the presence of a driver-assist system, this is unquestionable.
All I'm saying is that the distinction is important from the perspective of someone who wants to prevent this from happening again, and the perspective of someone who is in charge of making the laws and regulations to prevent this from happening again, and the perspective of the people voting for those lawmakers.
The real world is where all your independent worldviews are tested, simultaneously and integrated. And where metaphor shear may well prove fatal.
________________________________
Notes:
1. A not-necessarily accurate worldview itself, though I'll leave that to the cog-psych types to explore in depth.
But, hey. I'm not in the guy's head :)
I think that full autonomy (level 4 and 5) is an AI-complete problem and a true solution is many, many years away. On the other hand, I'm not an expert (I'm doing an AI PhD but on a different subject) so I may be overestimating the difficulties involved.
One problem with reporting on autonomous driving is that a lot of the technology is proprietary and the research takes place behind closed doors, so it's very hard to understand exactly where the state of the art is. We're left with the announcements from the companies who actually sell it, that will inevitably tend to be overinflated.
As the technology becomes more common, I guess we'll all end up adjusting our expectations, one way or another. I'm looking forward to your future articles :)
Of course, their stock price might crash and a whole bunch of customers who paid for self-driving features might be seriously pissed.
If they admit that radar + optical cameras won't be good enough for full self driving, there may be a lot of refunds to process.
From first principles, just optical cameras should be enough - humans can do it. But the software to make it work could be general AI level complex. Personally I think LIDAR self-driving will be available in less than 5 years, but optical may take 30 years. And the company who would be most capable of developing optical self-driving AI is Google, the fact that they are sticking to LIDAR is a good indication of the complexity of optical.
*autopilot
To be clear, when I say the software should be changed to not automatically steer, I don’t mean that it should more aggressively complain about hands off the wheel. I mean that the car should drive in a straight line with hands off the wheel, and it should also alert the driver if it thinks that the driver is leaving the lane.
Perhaps we need to regulate how driver assistance is sold so that people don't accidentally think they can just keep their eyes off the road?
[1]https://www.tesla.com/en_CA/videos/autopilot-self-driving-ha...
https://www.forbes.com/sites/bertelschmitt/2016/10/14/german...
Where did you pull those numbers from?
Somewhere in the middle of the page there's one "Every driver is responsible for remaining alert and active when using Autopilot, and must be prepared to take action at any time." But everything on the rest of the page is about convincing you that the car can drive itself better than you can.
Above 45mph hands free is allowed for 3 minutes [0] when following another car or 1 minute if not. There's no eye tracking unlike Super Cruise [1]. It only takes seconds to get distracted as the video of the Uber driver proved.
> There will of course be 1% drivers who will not.
It's the 1% that kill people.
[0]: https://en.wikipedia.org/wiki/Tesla_Autopilot#Alerts
[1]: http://www.cadillac.com/world-of-cadillac/innovation/super-c...
When Tesla was debuting Autopilot I got the sense that the system was more than your typical adaptive cruise control + lane assist that was solely designed for long straight roads. That was implied in the marketing, and messaging from Musk. Did the system even provide something as rudimentary as an audible warning if a driver climbed into their back seat like some dumbasses on YouTube?
>This message is repeated during onboarding, training videos, and manual etc. 99% of Tesla owners follow these instructions.
I bet it is now.
That sentence sounds strange to me. It is called 'autopilot', and in our society's mindshare - this means self-driving. Either way, the public believe these cars are self-driving, and as a result, people likely buy the car for this feature.
No one refuses to use GPS because it's possible to use a map or sextant like humans can. No one refuses to use radar because you might be able to point at a plane with your eyes.
Tesla should invest in improved lidar or imaging radar, rather than hoping they can come up with a neural network to solve all their problems from visual images.
My understanding was that self-driving systems are trained end-to-end to do simultaneous localization and mapping (a.k.a. SLAM [1]). In other words, the same model would control breaking, accelerating, lane keeping and everything else.
In fact, I thought this was why Uber had switched off its car's built-in breaking system- because their AI had taken over breaking and the AEB on the car would interfere with the self-driving.
Perhaps that is not the case for Tesla in particular, though?
_______________
[1] https://en.wikipedia.org/wiki/Simultaneous_localization_and_...
> When a car is moving at low speeds, slamming on the brakes isn't a big risk. A car traveling at 20mph can afford to wait until an object is quite close before slamming on the brakes, making unnecessary stops unlikely. Short stopping distances also mean that a car slamming on the brakes at 20mph is unlikely to get rear-ended.
But the calculation changes for a car traveling at 70mph. In this case, preventing a crash requires slamming on the brakes while the car is still far away from a potential obstacle. That makes it more likely that the car will misunderstand the situation—for example, wrongly interpreting an object that's merely near the road as being in the road. Sudden braking at high speed can startle the driver, leading to erratic driving behavior. And it also creates a danger that the car behind won't stop in time, leading to a rear-end collision.
When training new human drivers, they're taught to to steer around obstacles at freeway speeds, instead of braking for them. This is partly because it can take too long to brake.
The other half is these are often separate modules calling each other.
> And like adaptive cruise control, automatic emergency braking is often implemented as a separate system from the lane-keeping module. Most AEB systems lack the kind of sophisticated situational awareness a fully self-driving system would have. That means it may not be able to tell if an object 100 meters ahead is in the current travel lane or the next lane over—and whether it's a temporarily stopped car, a pedestrian, or a bag of garbage.
The auto-braking system could be a basic distance sensor calling the drive-by-wire API with "FULL STOP". This would definitely be non-ideal for freeway situations and speeds.
"If you're at lower speeds, at 30mph, and it detects a stationary object, these systems will generally respond and slow the car down and bring it to a stop," Abuelsamid told us. "When closing speed is above about 50mph, if it sees a stationary car, it's going to ignore that."
Indeed, automated braking that can only apply 100% braking is not ideal at freeway speeds. Collision avoidance at that speed must depend more on steering than stopping, but systems in cars aren't integrated in a way that would allow them to create this level of awareness.
But, yes, the i3 and Subaru have optical AEB that works great. Something people like to gloss over again and again in these discussions while talking about radar's limitations (and ignoring that the Model 3 has under-utilized forward facing optical cameras).
Of course, I'm not going to test it against stationary objects at high speeds, but at low speeds it definitely gives escalating warnings before you hit the door or wall of a garage. As someone who dented a garage door once, I value this feature.
> But while there's obviously room for improvement, the reality is that the behavior of Tesla's driver assistance technology here isn't that different from that of competing systems from other carmakers.
Is this actually saying that there's nothing "seriously wrong" with an autopilot that randomly decides to aim at a wall and accelerate, because other self-driving systems also do that?
This is some very weird apologetics.
But I think one of our blind spots as a culture is the assumption that nothing bad will happen if you drive a car correctly and follow all of the rules, and I think that assumption might prove to be wrong and that collisions are an inherent aspect of driving a car in an unpredictable world.
We as humans like to feel like we're in control of our lives, and I think that we have a prejudice toward looking at situations as if that were true, and that bad outcomes are the result of bad decisions instead of bad luck.
That assumption is clearly wrong. There are other drivers but even leaving that aside for the distant day when most vehicles are autonomous.
- Debris of various kinds
- Mechanical failures of vehicles
- Environmental factors, e.g. ice causing skids
- Road damage
- Animals
etc.
Inattentive tailgaters shouldn't be accommodated. Sure, their problem becomes your problem when you brake hard, but I'd much rather be rear-ended (with head rests and the whole rear of the car as a crumple zone) than plow head on into a stationary object at high speed.
Most of the time I can’t see past the car in front of me. It’s too tall, no windows (commercial truck), etc. so the ONLY way I know I’m in danger is how they’re acting.
You're supposed to keep a safe following distance where you avoid a collision if the vehicle in front of you stops suddenly.
Sure, slamming in the brakes is a bad idea. This doesn't mean the system can't start to slow down or change the direction of the car.
Still, you would think that there would be some threshold where the car decides, "Hey, this stationary obstacle is right in front of me. I should slow down"
You could imagine a next generation self-driving system that uses the combination of data from multiple sensors as well as maps to detect plausible obstacles. The mapping data could tell the vehicle when it should expect to turn. Maybe the vehicle could integrate imaging information from both radar and stereo cameras, to detect where the lane is, and which obstacles are in the lane.
Are there good existing techniques in the computer vision community for synthesizing data from multiple imaging sensors, like radar and stereo cameras and LIDAR? I'm imagining dumping all this data into an algorithm, getting back a 3D reconstruction of probable objects around me, along with metadata describing their velocity, confidence of the assessment, and all that.
From the article:
> When a car is moving at low speeds, slamming on the brakes isn't a big risk. A car traveling at 20mph can afford to wait until an object is quite close before slamming on the brakes, making unnecessary stops unlikely. Short stopping distances also mean that a car slamming on the brakes at 20mph is unlikely to get rear-ended.
But the calculation changes for a car traveling at 70mph. In this case, preventing a crash requires slamming on the brakes while the car is still far away from a potential obstacle. That makes it more likely that the car will misunderstand the situation—for example, wrongly interpreting an object that's merely near the road as being in the road. Sudden braking at high speed can startle the driver, leading to erratic driving behavior. And it also creates a danger that the car behind won't stop in time, leading to a rear-end collision.
If a human driver started hallucinating behind the wheel, we would expect them to do the same, not maintain speed (or accelerate!) through the supposed object in their path.
This tech was supposed to be safer and more convenient than human driving, not a simulation of the decision-making abilities of a 12-year-old playing Grand Theft Auto.
This requires the automated brake computers to be connected to the autonomous cruise computers. According to the article, in most cars, these computers are separate systems.
It's like if one person operated the brake pedal, another operated the gas, and a third steering the car. When one hallucinates, the others might not realize right away.
I think there is, it's just that by the time it reaches that threshold, the distance to obstacle is less than the stopping distance for the car at 60+ mph. A human making that judgment might steer around the obstacle, but AEB systems don't have that option.
They worked well for the narrow domains they were programmed for but they couldn't deal with novel situations and had no way to generalize. It is not clear we can engineer our way out of this by just adding new rules.
Proponents of early symbolic AI systems (expert systems) said we just need to be patient and add more rules and at some critical point the system would reach the singularity. One such project has been going on for 34 years! https://en.wikipedia.org/wiki/Cyc
This dilemma is only really bad for a passive system. For a system that's actively steering, it can make a committed decision to curve, so that even if it misjudged the obstacle it still won't crash.
It doesn't hurt to slow down for a predicted near-miss either, as you get close to it.
> Still, you would think that there would be some threshold where the car decides, "Hey, this stationary obstacle is right in front of me. I should slow down"
Yeah. There's a big difference between "obstacle might be in the way" and "obstacle is definitely in the way".
The big difference is that people who drive those cars do not generally expect that their cars are self-driving. Tesla owners do seem to expect that, and consequently the decisions made about trade-offs for, say, automatic braking in a Honda Accord are not the right decisions for a Tesla Model S.
The bottom line is this: if people think their cars can drive themselves, the cars really need to be able to do that or there are going to be crashes.
The 2014 Audi I used to have would do different things depending on the calculation it made on how close you were to crashing:
1. Pre-tension the seat belts
2. Increase the braking power (IE needs lighter push)
3. Close the side windows and roof
4. Turn on hazard lights
5. Give you a slight jerk through braking to try to warn you. etc
The first one alone helps a lot. https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
(13% fatality reduction)
Now, most cars will do this these days, and do it using the same sensors they need to trigger air bag deployment even without ACC.
I don't know of any study that tries to estimate the effects of doing it earlier like the Audi with ACC does (IE no study tries to evaluate whether the ones without ACC sensors have achieved the optimal amount of pre-tension in the time they have, which would make pre-tensioning them earlier like the ACC systems do pointless)
So in other words, Autopilot works reliably only in lab conditions. In the real world, not so much.
This car left the road entirely. There's no excuse for that.
Also of note the impact attenuator was completely missing due to bad highway maintenance. If the tesla had struck an engineered set of barrels instead of side striking a jersey barrier the driver would have walked away.
Tesla explicitly states that a driver using their Autopilot must keep their hands on the wheel at all times. They place the driver in the role of supervising the machine. It seems like other driver-assistance technologies require the same, which is a fundamental misunderstanding of what drivers expect these systems to do. Drivers would expect these systems to ease their workload or take over some of the tedium of long highway drives. But these systems require the driver to be alert and constantly monitoring the system for abnormal behaviour. As far as I can tell, this is more work than simply driving manually. On a long drive, I can slip into my own sort of autopilot where I'm paying full attention to the road, able to react to changing conditions ahead, but am also entertaining a long train of thought in my head.
Six seconds sounds like a long time to react to a changing situation when driving (drivers know that accidents happen in split seconds), but if things are working normally, six seconds is about the length of time you might spend changing the climate controls or selecting a different playlist. If Tesla are saying their cars can't be trusted for a single-digit number of seconds without human supervision lest they total themselves, since it's so often toted that the driver's hands were off the wheel and thus he couldn't retake manual control and prevent the crash, then these companies are approaching driver assistance technology in completely the wrong way. More than that, these systems are outright dangerous because they're implemented so completely wrong. Taking your hands off the wheel in an unassisted car for six seconds won't result in it driving itself into the barrier beside you unless you're extremely unlucky at that exact moment and your front wheel hits a pothole. For these machines to be this unpredictable is becoming a serious safety hazard.
I'm going to stick with a car with non-adaptive cruise control and nothing else. And be extra vigilant around Teslas on the roads.
You won't be competitive with normal human drivers that way. Humans (usually) have to ability to combine many diverse, and potentially conflicting, pieces of info into a coherent story. AI and AI-like automation will need to similarly synthesize diverse clues.
It did make me wonder if firetrucks were accidentally left out of the datasets.
Yeah, my Model S did this once on the Interstate, for no reason that I could tell. I nearly had to engage a dry cleaner.