Hacker News Reader

Comment by Animats | Hacker News Reader

Watch this at 0.25x speed or slower to see what's going on. This is a carefully chosen environment. Every place it drives has very clear highway centerline markings. It seems to be highly dependent on those for guidance. Sometimes it can't quite identify the road edge, but the centerline provides a position reference.

The inputs seem to be road line recognition, optical flow for the road, and solid object recognition, all vision-driven. Object recognition is limited. It doesn't recognize traffic cones as obstacles, either on the road centerline or on the road edge. Nor does it seem to be aware of guard rails or bridge railings just outside the road edge. It probably can't drive around an obstacle; we never see it do that in the video.

This looks like lane following plus smart cruise control plus GPS-based route guidance. That's nice, but it's not good enough that you can go to sleep while it's driving.

View on HN

It terrifies me to think of relying on image recognition software to correctly determine an upcoming crossing road (with cars zipping across) so it can properly slow down, rather than get broadsided at full speed. Or a number of other life-and-death situations (which are common during driving). It just seems fragile (what if the 'vision' is somehow impaired, f.ex. blinded by sunlight, or the road markings are wrong or obscured) when the failure mode is so catastrophic.

I suppose once statistics start to prove that these cars are safer than human-driven ones, we can chalk it up to an irrational fear, but for now it seems crazy to me to put my life in the hands of an AI, when a mistake means that I die or kill someone, rather than play the wrong song on Spotify.

I don't fully understand why more effort is not put into a hardware solution, where roads are simply marked up for self-driving vehicles, e.g. magnets lining the lanes or something like that. Of course a more expensive solution, but seems like it would make the vehicles themselves a whole lot simpler and safer. Begin with inner cities, where the area is limited and traffic is most complex.

> It terrifies me to think of relying on image recognition software to correctly determine (... snip)

Do you not drive using only your eyes? If you are not terrified of the sensors, then software? Turing's central belief was that the human brain was 'just' a computer.

Regards doing things like embedding reflectors in roads and other ways to simplify lane holding, completely agree. But we can't forego the cameras etc that deal with situations that don't contain reflectors.

The thing is, human "machine vision" has graceful failure modes. You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy. The same cannot be said about current ML demonstrations - because they operate on just vision. Humans continuously reconcile visual input with their model of the world and with other inputs, to the point of overriding visual data if needed.

I suppose you could make ML avoid sharp changes in recognition, but I don't trust the current neural-network models to do so reliably.

Another issue is detecting issues and fixing them. For example, if dirt gets into my eyes while driving, I immediately know and do what is necessary to clean my eyes before continuing on my drive. On a car, what if there's a small spec of dust on a camera that blocks some part of the view. Is the car smart enough to figure out something is wrong and refuse to go?

The AP2 feature in the Model S currently alerts the user when vision is obstructed and refuses to go into autopilot mode. If you're in autopilot already it makes noise and flashes warnings to tell the driver to take over. I assume if the driver doesn't it will slow down and come to a stop, but the one time this happened to me I didn't give it a chance to do so!

You've never been temporary blinded by a low-sun? Never sneezed and had your eyes closed at the wrong moment?

This is the same thing..

Humans know when they can't see properly. The problem with adversarial examples is that the neural net thinks it's doing great. You have to manually specify every possible failure mode for the neural net before hand -- if you miss one, you might have an accident.

Motorcyclists have a word for this phenomenon. It's called SMIDSY - "sorry mate, I didn't see you." Clear road, good vision, bike traveling in a straight line within the speed limit, car still somehow pulls out and hits the bike. Even worse, the times when the bike is stationary at a red light and the car just drives through it.

And it's not limited to cyclists. I think the general term used in cognitive science is "Looked but failed to see".

The real world is not filled with adversarial examples. Further, NN often does not think it's doing great when providing off the wall examples. Further this is a car, with GPS, where simply stopping is a completely viable option 99% of the time.

The truth is people really really stuck a driving, but it's easy enough that the vast majority of mistakes don't result in any problems.

>simply stopping is a completely viable option 99% of the time.

For a naive definition of "vaiable" that does not take into account other people that need to use the roadways. A car stopping or even slowing drastically in a lane will have the same negative affect on the transportation system as a minor accident.

You mean like we have every day already? Human drivers constantly make mistakes, and in many cases they aren't even aware of it or their affect on the drivers around them. Most don't cause an accident, but the behavior doesn't get corrected and that is what it eventually leads to.

Slowing for a turn then deciding to continue on, merging without checking blindspots, running red lights, speeding next to the short on-ramp lane, not following a yield to right turn sign, ignoring people in crosswalks, etc. All of these things happen all the time, I see at least one example every time I go out somewhere.

These cars are unlikely to stay stopped for long as there are going to be passengers in them.

Now, we could get into hypothetical situations, but the reality is minor accidents are already really common like with so many other things machines can be better than people without being perfect.

I really want to see how cars deal with the massive code explosion as they have to account for things like tak tak gangs.

I posit now: "as the problems with automated cars become evident, solutions provided will tend towards Skynet as t increases."

> Humans know when they can't see properly

There's plenty of conditions that are more difficult to humans: cars that are painted with the same color of asphalt seem to disappear in highways. Also, fog.

Usually. Drunk and sleepy drivers blur this distinction, but you still make a good point.

Aren't general adversarial networks designed to solve this?

It's not. When you're being blinded by sun, you don't suddenly see elephants or spaceships or comic books. You also don't think the world just disappeared - you know to just ignore the visual input.

Also, I'm more worried about temporary, unexpected configuration of things that suddenly make a NN see something weird.

That doesn't sound as an insurmountable problem.

Humans don't see elephants suddenly because a prior is telling them that an elephant is improbable in that context.

Something like that in a computer system shouldn't be above the current capabilities.

I agree it shouldn't be above current capabilities. But I'm yet to see anyone doing something like this. And I'm definitely wary about taking the results of toying with "deep learning" and saying we've can put that on a car and expect good results - which is an impression I sometimes get in articles about those topics.

Also, this will stop working the day an elephant escapes from a ZOO and manages to get onto a highway. All the autopilots will declare the elephant's presence as impossible, and will happily crash into it.

Not likely. Computers can do a double-take, just like humans.

I'm not sure the teslas are going to be trained to see elephants either!

You don't necessarily have to train it to see elephants, but Neural Networks fail in pretty interesting ways. This that are very obvious to you and I have the ability to confuse a CNN[1].

[1] - http://www.evolvingai.org/fooling

These look like a digital Rorschach test. With the minor difference that a human "knows" the image is not really an object and is being asked to interpret in some way; where the machine has no "knowledge" nor "understanding" but has an imperative to match this input to something in its repertoire of knowledge.

As a human, if you asked me to match these synthetic images to a most-likely real-world match, then my responses will also look "confused."

Julia Evans: How to trick a neural network into thinking a panda is a vulture

https://codewords.recurse.com/issues/five/why-do-neural-netw...

The thing is that computers also rate them with 99% certainty.

Those aren't too bad (out of context of the Tesla article), I can see how the ai might misinterpret those images. The bagel one, for example, is pretty on point.

Those examples show that ML algorithms "see" things much differently from us. It also shows they're very vulnerable to nonsense data. How can I be reasonably sure that some random play of shadows and lights, that would never fool a human, will cause a failure mode in a neural network in a vision-only self-driving car?

Or worse - paint jobs created specifically to muck with image recognition.

Or when the graffiti becomes intentionally malicious (thinking of a variation of Wiley E. Coyote's tricks with paint and solid walls)?

Would these be the same graffiti artists that are currently maliciously painting over road warning and direction signs to fool human drivers? Say, whiting out a 'low bridge ahead' sign, so that trucks and double-decker buses crash into it...? Because, nobody is doing that now, and if they tried it, they would be prosecuted, probably for murder or attempted murder.

I don't understand where some people get the belief that automated vehicles will suddenly become targets of murderous or sociopathic pranksters?

> I don't understand where some people get the belief that automated vehicles will suddenly become targets

There is precedent when it comes to the automation of people's jobs (truckers, cab drivers, courier services) and fanatical condemnation of technological advancement; see everything from the Luddite's infamous wooden shoes to the spiking of trees to stop logging.

In the forests near my hometown, some environmental fanatics will string up wire across motorcycle and ATV tracks, or create blockages and attack riders with crowbars and pipes. I wish I were kidding.

As for current graffiti artists - they're taking down stop signs and road markers, defacing handicapped parking signs with stickers, tagging any and all road signs, and so forth. They're minor nuisances for human drivers, who can usually easily identify compensate for such changes. Computers, even those running neural networks, are not able to compensate so easily.

Last year, there was a popular hoax about a graffiti of a Road Runner tunnel: http://www.snopes.com/road-runner-tunnel-crash-rumor/

I guess the CNN was trained only with real images. They must add some [random noise], [crap], [glitch] and [collage] images to the training set. This may reduce the efficiency of the NN in the real world, but it will make more difficult to find funny examples of misclassification.

They will need to be if they are to be able to recognise all the potential real world phenomena they might encounter!

That's just Bayesian priors. Easy peasy for ML.

Haven't seen much Bayes in current deep learning trends though.

> You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy

yeah, instead you get hallucinations, muscle twitches, seizures, mental breaks and many more ways a brain can fail to do it's job.

Sure, but a) we have thousands of years of experience with those, and b) none of those are caused by little random noise in the visual input. Brains are incredibly robust against noisy data.

Our vision and persistent world model are good, but all humans are overconfident in their ability to drive in poor conditions (or even normal conditions, for many people.)

A computer isn't going to look down at a text message on their phone, fall asleep at the wheel, or get in a car while intoxicated. All which occur literally every day. People take driving for granted, because (at least in the US) we have to. Our sprawling suburbs couldn't function without it, so the barrier to entry is simply being a teenager.

Would I completely trust Tesla's auto-driving? No, not at all. But I at least trust them as much as I do an average driver, which is still just barely. The difference with Tesla's system is that one failure leads to improvement of the entire fleet, whereas one human failure maybe improves that one human driver. That means self-driving cars will continuously improve, while humans will just continue to be terrible drivers.

Yes! Tesla will take any death due to a failure of self-driving quite seriously indeed and work to fix the failure mode.

Whereas humans get the same kinds of fatal crashes over, and over, and over...

The Australian government has a "Black Spot" [1] program to provide federal funds (often millions) to fix up locations with a history of crashes. Bet it's cheaper to make those fixes to software.

[1] http://investment.infrastructure.gov.au/funding/blackspots/

Robustness is a key property of the methods and models used in self driving cars.

For a human, the noise isn't bad recognition, it's corrupted sensor data via distraction. It's one of the most common causes of accidents. For the AI agent an elephant will never appear unless it's been trained to be recognized. More likely it's recognized as an obstacle, which is ok. But keep in mind that years of Google's autonomous vehicle success make this pretty improbable, certainly less than in the average driver.

Much of the fear here is irrational and natural when humans lose control.

I'm not trying to express a fear that comes from humans losing control. I don't fear that - in fact, I can't wait until the day comes no humans are allowed to drive on public roads.

I was just answering about the concerns of self-driving systems relying solely (or primarily) on visual data from cameras. It's true this is how humans drive, but human visual system is much more complicated than what the current (published) state of the art in image processing seems to be. I do not trust visual-only systems today (especially if they employ deep learning shenanigans).

I know those problems aren't insurmountable, but I'd feel much safer if they threw in a LIDAR there too.

While I'm sure more sensors gives a better chance at consensus I don't think relying solely on visual data is a problem if done well (low false positive false negative rate, comprehensive trained, fault tolerant)

a) well, no. we only have about 100 years dealing with these problems in context of driving cars and we're really bad at it; b) that seems like shifting goalposts, what we care about is whether driver's picture of the world is correct and actions are sensible with respect to that picture, it just so happens that driverless cars' picture is visual (or close to visual), but if you're judging them on entirety of input and side-effects it's only fair to judge our brains by the same standard.

Some adversarial methods work on black-box networks - has any work been done on applying those to human vision? We know there are a wide range of simple inputs that produce unexpected results (eg. optical illusions).

> You don't switch from seeing cars to seeing nothing to seeing an elephant just because the input got a little noisy.

You'd be surprised. All kinds of conditions can do really weird things to your vision. Migraine auras probably being the most common.

We don't drive with just our eyes. We fuse vision, sound, touch and proprioception. We self-locate easily. Most important though, is we do sophisticated visual, theory of mind and physics based inference.

If you take the example of Chess, humans and machines play very different styles. Computers replace predictive understanding with excellent memory and fast search.

Similarly for cars, we want to replace the lack of a powerful predictive model with some compensatory mechanism, such as sophisticated sensors. The problem is that, even with faster reaction times, better sensors and fuller awareness, these cars' compensatory abilities against a lack of powerful predictive models are still far from sufficient.

At this point in time, a car with more sensors and failsafes to augment the human against collision is safer than a self-driving car that depends on a lack of lapses of attention. That depends on proper behavior in the customer. Even dividing tasks, with the human controlling the steering (but allowing the car to takeover when highly certain of danger) and the car controlling acceleration would be safer. This set up acknowledges the human propensity to mind wander when attention is not engaged.

Until self-driving cars no longer require human rescue in difficult situations, one can't rightly say that self-driving software are better drivers than humans.

If the brain is just a computer, its machine learning algorithms for image recognition at a busy intersection are believably currently more accurate than any image processing software run on any computer that Tesla puts in its cars.

There are ways in which current machine learning is better than your brain -- tracking many objects at the same time is one of them.

Sure, it's worse in many other ways.

The ability to track multiple objects doesn't necessarily mean it's any good at integrating a solution. My camera can track multiple faces and smiles - that doesn't mean it can take better photos.

We are far away from hard AI (computer actually can think) some people start to believe we might not be able to reach it. What we currently succeeding is developing soft AI (solving specific problems) the problem is that such AI might do fine when things are clearly labeled, but might not know what to do in unexpected situations. With car stopping might often be correct, but in other situations actually not stopping is best etc.

Another thing is that your eyes can last you 100+ years, technology is not that durable and breaks more often, the objectives in cameras get dirty etc.

The cameras typically aren't as good as human eye, for example there are reports where Tesla wanted to jam into a truck in front only because its back in the camera couldn't be differentiated from the sky.

Yes, relying on cameras only is perfectly fine when you play PS4 game, but when life is on the line you want extra safety checks. Kind of like cars have brakes on every wheel and then also a handbrake. How every car has minimum of 3 stop lights (how often you see that one or more lights are bad in other cars?)

Your belief that we have hard AI (and this is kind of indirectly created by companies like Tesla, Google, Amazon, IBM etc) gives you conclusion that just two cameras should be enough in reality we did not made much progress in that area and instead just concentrated on specialized AI that is capable solving specialized problems, that kind of approach absolutely works better with more sensors.

Is the "computer vision" on Tesla good enough to ignore this false "lane marker" and use the actual yellow lane marker, not the "fake" one caused by the sun/shadow/pavement interaction?

http://i.imgur.com/fk3OvEW.jpg

This situation generated constant lane departure warnings for the driver.

I can't speak for Tesla. But I have a newest generation VW with lane assist. And lots of road construction sites in my area, where the normal white lanes markers are "overridden" with some yellow lane markers, which are narrower and often also take different pathes. Once I'm inside a yellow lane and drive straight it halfway works and the lane assist keeps that lane. However on the first position where the lanes depart (e.g. normal white lane goes straight, but yellow lane turns left to change over to other side of the road) the car has not really an idea what to do and mostly follows the wrong lane --> Without manual intervention a crash in these conditions would be guaranteed.

Here in the southeast, tons of roads have leftover sets of lane markings from when lanes were shifted over. Or no lane markings because of a lag between relaxing and marking lanes.

Not sure but that's pretty confusing to me too tbh without close inspection. The car in front seems to be treating it as a fog line too.

I really want to see one of these put on the road in India and see how it works. That's the real litmus test for me.

The human visual cortex is far more powerful than the computers we can put in cars.

"I don't fully understand why more effort is not put into a hardware solution, where roads are simply marked up for self-driving vehicles, e.g. magnets lining the lanes or something like that."

How would "magnets" be any improvement over visual markings? Is there some sensor that can track magnets at greater distance, or with greater reliability than visual markings?

Besides, how many millions of miles of road would need to be equipped with these magnets around the world? What would that cost? Would it be acceptable that autonomous cars could only drive on roads that have been properly equipped?

It's worth remembering that self-driving AI does not solely rely on lane markings to navigate. It also can see the road edge, follow other vehicles, and cross-reference with a (learned/crowd-sourced) navigation model of the laneway. Additional sensor types, like LIDAR and radar, are often used in addition to cameras.

You can see a lot of this at work, in a complex London traffic environment, in this demo:

https://www.youtube.com/watch?v=cfRqNAhAe6c

"I don't fully understand why more effort is not put into a hardware solution, where roads are simply marked up for self-driving vehicles, e.g. magnets lining the lanes or something like that."

Volvo is plugging for magnets in roads in Sweden, so they can find lanes in the snow. It might happen, because it can also be used for snowplow guidance. In heavy-snow areas, posts, poles, and even over-road arrows (Japan uses this in Hokkaido) are placed to provide guidance. It's not intended to replace vision and LIDAR, just as additional guidance hints for bad conditions.

A simple RF transmitter in new versions of the existing reflector discs would do it.

It wouldn't need to do anything but "ping", the battery could last for years. You could even have a mechanism to disable it when the lines are repainted, though normally roads that use the existing version of these reflectors would have them removed when moving lines.

The transmitter would have to be powerful enough to go through a small amount of snow in some states.

The question is can we get the car companies to agree on a standard for this? Would it make sense to put it in the frequency range allocated for vehicle radar systems?

What we get instead is an attempt to define the whole system de jure, including full V2V protocols and specifications.

But what is the actual advantage of this RF system? Why is this any better than just using a camera and image processing?

OK, RF would mean the lane geometry can be "seen" through all weather conditions (fog, snow). But so does precise-enough mapping/fleet learning. Neither would help in detecting traffic and transitory obstacles that might be hidden by that fog - you still need other sensors for that!

A "safe" autonomous vehicle should perhaps simply slow down in adverse weather conditions just as a human driver would. Even without additional sensors, their ability to see through snow and fog is certainly no worse than a humans.

We need transmitters in licence plates and on street signs too.

Why? So they can be "seen" through rain, fog and snow?

Reading road signs optically is a solved problem. Mobileye and similar systems have been doing this for a decade or more.

Well you answered your question.

Further, it would help learning in limited visibility, although they can already do it by remembering locations of signs (people do that, do autonomous cars store city/state/worldwide road data?)

The fear is irrational because even now you can always be hit by other drivers who are not as careful as you. In my home town, certain people are driving like crazy.. Ignoring traffic lights, going to fast, call or text during driving, etcetera. With autonomous driving, these people can game, work, drink etc instead. The main benefit of AI is that it drives more predictable and follows the traffic rules. Furthermore, cars themselves have become safer. If you look to traffic fatalities per country. You can observe that modern countries have less fatalities. Because of airbags, cripple zones, safer knocking of pedestrians.. And also traffic rules. To conclude, even though sensors might fail, they are better than human sensors.

Seriously? Do you want to spend an incredibly huge amount of money to put magnets to delimit lanes all around the world?When there are so many roads without any clear painted lane marking, when the paint itself fades away in an handful of years and the magnets will last a couple of years at best ONLY if embedded in the asphalt, that means spending 100 times the amount of money above. And when in 2-3 years time we very likely will have self-driving cars far safer than humans. IMHO is pure nonsense. It will be a clusterfuck order of magnitudes worse than the British building thousands of km of canals when after 20-30 years the rails were dominating transportation. But at least they have the excuse that they didn't know what was coming, and 20-30 years is an order of magnitude longer than 2-3 years. Furthermore they spent an infinitesimal fraction of what you propose to spend.

"worse than the British building thousands of km of canals when after 20-30 years the rails were dominating transportation."

I'm not sure that's a fair analogy. Without the canal network to cheaply transport goods and resources in bulk, it's hard to imagine how Britain could ever have industrialised to the point where building a railway network became possible in the first place!

Besides, railways did not immediately obsolete the canals as you seem to suggest. Some canals were still being built and extended until the early 1900s (70+ years after the railway boom), and the last commercial freight services lasted until as late as 1970 - suggesting it was really motorways that "killed off" commercial canal traffic.

One key point of difference is that (like road freight) pretty much anyone could operate a freight service using canals, where as the railways were in the hands of a few huge corporations. That meant a lot of niche, point-to-point freight was more viable to transport over water than rail.

They tried the magnet thing but it isn't flexible enough. For instance during roadworks you can't easily move the lanes, it only works when everything is perfect. There was a test track in NL with this technology a few years ago.

Start with highways - they are grade separated, there are no pedestrians, no traffic lights, no intersections (besides merging), driving is often long and boring.

I agree that 'properly' marking roads for self driving vehicles is better. But this is a chicken and egg problem. There is no incentive to create such markings until self driving vehicles are common. And limiting self driving vehicles to properly marked roads will make them useless for most people, resulting in less funding for further development.

These would also apply to Lane Departure Warnings present on more and more cars each model year.

It doesn't have to rely on vision.

They probably use GPS+map for a low resolution location and trajectory planning. Vision should only be used for fine positioning and obstacle detection.

When did all algorithms become AI?

Whether an algorithm is AI essentially depends on whether the objective is 1) human level difficult and/or 2) heuristic.

It should probably be both to avoid moving goal posts. That would make even chess algorithms still AI.

Self driving cars and image recognition are both definitely still AI.

"AI" does not typically mean strong or general AI.

This video is from 7 months ago and a concept demo. They would be quite a lot further along now. Read further down: "Your Tesla will figure out the optimal route, navigate urban streets (even without lane markings), manage complex intersections with traffic lights, stop signs and roundabouts, and handle densely packed freeways with cars moving at high speed."

They _might_ be further along. Everything you've quoted is vapor until it is demonstrated with their current hardware.

> This is a carefully chosen environment.

This is Los Altos - Los Altos Hills - Palo Alto route to Tesla's HQ on Deer Creek Rd. A very expensive area to live and therefore well maintained.

https://goo.gl/s9RJxw

> .25x speed?

That's a bit of an overkill.

Regardless, there was two points in the video ~48 sec, and ~1.40 min/sec mark. The car stopped AFTER turning... I thought that was really strange behavior. Then when it passes the pedestrians, it almost went to a complete stop, instead of going slightly to the left and mildly slowly down.

Overall, it's a great case study. I think your criticism is technical snobbery instead of what this means for the average person. The future is here, get ready. When techies complain that someone is doing something with 'x tool' and not 'y tool'...I think those people are just throwing mud around and not actually contributing to the conversation. Build something better if you're going to nitpik over the technical details. It would serve you better and all of humanity.

The future is here, just not from Tesla. Waymo and Volvo have systems which actually work. Waymo has real situational awareness, watching other road users and predicting what they might do next. Tesla is mostly following the lines on the road and stopping for interfering traffic.

Here's Tesla's "Autopilot" killing someone.[1] This is the Mobileye "does not recognize obstacles protruding into left edge of lane" bug.

Here's another crash from that defect.[2]

Here's Tesla's "Autopilot" hitting a traffic barrier.[3] Again, it's an obstacle at the left edge of a lane.

This is what happens when you take Level 2 lane-keeping and automatic cruise control and hype it into automatic driving. Other carmakers have offered comparable systems, but with much stronger driver-must-have-hands-on-wheel enforcement. Tesla didn't do that, encouraging drivers to relax and let the computers drive.

Waymo and Volvo take the position that when the automatic system is in control, the manufacturer is responsible. Tesla tries to blame the driver.

(Been there, done that. Ran a DARPA Grand Challenge team 2003-2005. Too old now to work on this.)

[1] https://www.youtube.com/watch?v=fc0yYJ8-Dyo [2] https://www.youtube.com/watch?v=qQkx-4pFjus [3] http://video.dailymail.co.uk/video/mol/2017/03/02/5177969943...

It's also fun to watch the guys hands. Little tells that he's suppressing the desire to grab the wheel (e.g. 0:44, 0:54). He seems to be fine at stops and intersections, but a bit more attentive when things get tight and curvy.

Also, what is the car doing at 1:33. It takes a right turn, then just stops in the lane, then proceeds.

My guess is that at 1:33 the car makes the turn a bit too wide and brakes because of the oncoming cars that come just a bit too close for comfort.

Speaking of - what happened exactly at 0:54? It seemed like an abrupt slowing down. A human driver would not slow down this much due to a couple of pedestrians walking next to the road. The correct behavior is slight slowing down and making sure the pedestrians will remain safe and then going past them.

Isn't Tesla's official recommendation to keep your hands on the wheel at all times?

One day some joker with two pots of paint (one white, one gray) is going to have a field day causing cars to run off the road.

Something like that happened to a Tesla, here.[1] The Tesla is the car in front in the left lane, not the one taking the pictures with a dashcam. There's a construction barrier that's inconsistent with the lane lines. The Tesla plowed into the barrier.

This shows why the Tesla approach isn't good enough. The vehicle is on a freeway, an supported environment for the old "Autopilot". Everything is going just fine. Until there's something unexpected the system can't handle, appearing fast enough that the driver couldn't take over in time.

Tesla is somewhere between Level 2 and Level 3. Good enough that the driver is tempted to tune out, not good enough that they can.

[1] http://video.dailymail.co.uk/video/mol/2017/03/02/5177969943...

Holy crap that looks like a very basic failure. This is exactly the sort of thing that keeps me up, getting into an accident because of some stupid software bug. Fortunately the car didn't spin or he'd have side swiped the vehicle in the next lane.

If you look across the roof of the Tesla it totally failed to imitate the car in front of it too, that's a double fault, it had more than one indication the lane was changing direction.

Here is an article with a bit more info about that crash:

https://www.digitaltrends.com/cars/tesla-driver-error-autopi...

There's more like these:

https://www.theguardian.com/technology/2015/oct/21/tesla-aut...

I drive through a construction zone once a week where the old lane shift lines and the new ones are close enough in quality to make following the lines instead of the traffic around you a very bad idea.

My Passat (2016) with lane assist falls into these traps all the time. Same with offramps. It can be a fight to keep it from jeering off. It is really nice on long drives though.

https://creators.vice.com/en_us/article/meet-the-artist-usin...

That's funny! One day I'll have other original thought... one day. But not today apparently.

a'la Wiley E. Coyote? Yup. Going to be interesting.

well all one had to do before to understand how staged previous demonstrations were was to go you youtube and type in tesla autopilot failures. some were downright spectacular and at those at night, well just forget about it.

I would love to see some work made towards configuring HOV lanes that many interstates have (toll or otherwise) to have not only clear visible markings but in road indicators to facilitate easier autonomous driving in a more controlled environment.

It would have so much safer to Tesla to have trademarked (not sure of proper legal term) Autopilot for later use and call it CoPilot until its safe on its own

You had to reverse early cars up hills due to their gravity fed fuel systems.

Yet here we are, driving forwards up hills.

Compare Chris Urmson's SXSW talk from 2016 about Google's system.[1] Look carefully at the graphics which show what the sensors are detecting, and compare with Tesla's. Watch Urmson discuss hard cases, problems, and failures. Not seeing that with Tesla. Or Uber. Or Cruise. Or Otto.

[1] https://www.youtube.com/watch?v=Uj-rK8V-rik

Tesla want a system that works all the time, even if that means it kills some pedestrians once in a while.

Waymo want a system that has a zero accident rate, even if that means it can't operate in the rain.

Otto is Uber now ;)

Cool, when is it being released?

Waymo's Early Rider Program is now running in Phoenix AZ.[1] They have soccer moms and their families in self-driving minivans.

[1] https://www.youtube.com/watch?v=6hbd8N3j7bg

self-driving minivans with human safety drivers

And yet, its still an objectively better driver than you.

Not a better driver than "me +active driver assistance like collision detection, Lane keeping etc".

Indeed, and not only that, Elon Musk's example of Autopilot supposedly saving lives that he famously tweeted a while back was manual driving + active driver assistance in the form of automatic emergency braking (which is supposed to become mandatory on new cars at some point). It happened in conditions far outside what Autopilot was capable of actually driving in.

I think that we'll only turn over driving to a computer (for reals) when safety systems are good enough to prevent most accidents. I think we'll keep the safety system separate from the drive system.

Perfecting safety on human drivers first is the way to go and will most of the risk of lettings computers drive.

Is that what they're claiming here? It looks like they're just saying that the hardware is capable of it, and that it will beat human driving at some point in the future.

Correct. There are probably primates that have all the hardware required to drive a car too. But more is required.

Humans are still better at interpreting many things and probably dealing with weather and construction and when there is no data.

"All Tesla vehicles produced in our factory, including Model 3, have the hardware needed for full self-driving capability at a safety level substantially greater than that of a human driver."

"Have the hardware needed". Note: not the capability; just the hardware. It's an unnatural way to write promo copy for a feature, and is written that way for a reason.

I think that with tesla's wireless software updating capabilities they'll continue to improve the software over time after you buy the car.

That is Tesla's existing track record, yes. How far they can improve on a given sensor platform is the unknown part.

So we're just waiting for Sufficiently Smart Car Software, is what you're saying?

That sounds speculative.

it certainly is, but that's my read on the situation.

It's not objectively better if I've caused 0 accidents to the autopilot's several, even if I've driven several million fewer miles.

Reply on news.ycombinator.com