Autopilot Full Self-Driving Demonstration [video] | Hacker News Reader

Autopilot Full Self-Driving Demonstration [video]

tesla.com

386 pointsmegakid243 comments

A longer Tesla demo video with no cuts: [1] This one is at a consistent speed, about 3x normal, and it's worth playing at 1/3 speed to see what's happening. No freeway driving; it's driving around the Page Mill Road / Los Altos Hills area. This video shows an ahead view and three of the vehicle's cameras with their annotations. The system puts rectangular 2D boxes around things it recognizes.

Road centerline recognition seems good. Every road seems to have really clear centerline markings, though. Road edge recognition isn't always successful. Roadside obstacle recognition seems very dependent on recognizing cars and people. The system doesn't recognize a large trash can in the roadway until very close. Recognition of roadside traffic cones and barriers seems to be about 50%. Guard rails aren't recognized at all.

This has the look of something using deep neural networks trained on common obstacles. It doesn't look like something that profiles terrain.

[1] https://vimeo.com/192261894

Is there an easy way to play a Vimeo video at 1/3 speed?

There are bookmarklets for Vimeo, and a Firefox add-on for controlling all HTML5 video speed.

Press F12 go to the console and type `document.querySelector('video').playbackRate=0.33;`

I use the html5 video speed control extension in Chrome: https://chrome.google.com/webstore/detail/html5-video-speed-...

Mainly to play Vimeo videos at 2x speed since Vimeo refuses to put the feature in the player.

There seem to be some issues. E.g. it recognizes the joggers (0:55) as obstacles even though they're obviously (to a human) on the sidewalk. And when taking a right turn (1:01) it mistakes a hydrant and a parked car in a parking bay on the other side for an obstacle and stops for a moment before noticing that they're not actually in its lane.

I think a polite driver may have slowed down gradually to not startle the joggers, but not stopped if it were avoidable, especially since there was a car behind the tesla.

In the latter case i guess the field of view of the cameras is not wide enough to assess those things during a turn in advance?

It passes a single jogger without an issue beforehand, but they were two when the car stopped and left one was relatively close to the lane when the car was approaching. Maybe it wouldn't stop if it was a single jogger.

If you pause it at 0:54 it clearly made a mistake. The foot of the left person is outside the green box, the car thought the person was further forward and to the left of where they really were. Good progress but not ready for prime time.

you're looking at it from the wrong perspective. the car just wanted to slow down to give the male driver time to properly look at the female joggers.

I don't want my car to be like the creepy telepathic dog from "A boy and his dog" that is always looking for partners for his owner to mate with.

i don't know much about self-driving, but i would not over-interpret the green boxes. Also i don't know if your assessment about the range is true. Seems like a bug.

The car might have been playing on the safe side. If you look at the joggers carefully, only the left one is marked as an "in-path object", the other one has just an "object" box. They could probably tune down the collision prediction algorithm just a little bit, but that has to be done very carefully.

Why bother, err on the side of caution with pedestrians. The driver is already benefiting from auto pilot, a minor delay due to safety of other road users would be acceptable to most.

It's inconveniencing other drivers on the road and much more importantly this erratic stopping from a human perspective might make it more likely to cause an accident. Granted the accident would legally be the meat driven car's fault who bumps into the suddenly stopped Tesla. It would still be nice to play well with human drivers who don't have the luxury of relaxing in a self driving car.

Well no one think of the coffee?

> Granted the accident would legally be the meat driven car's fault who bumps into the suddenly stopped Tesla.

Exactly. If you bump into the car in front, because they stopped for a percieved hazard, it is your fault for tailgaiting, not their fault for percieving a hazard.

That's all very nice in theory but, in the actual real world, if you do a lot of unpredictable braking or other unexpected behaviors, you will basically get hit all the time even if those accidents aren't technically your fault.

Soon people will either adapt and end up following the law, or they will be unable to pay for insurance, because of the amount of claims that they're taking out against it.

I generally not a fan of social darwinism, but in this specific case, I think that something of the sort will have a positive effect. How many people have been injured because the car they were hit by was being forced to go faster, because of tailgaiting?

I agree, legal or not, I very much rely on other drivers "signals" when driving. Over the years you learn to predict what the driver in front of you would do in a normal situation and adjust your driving in advance. I'm still able to break on emergency but when you can clearly see way ahead of the car in front there is no obstacle and yet they still brake it's extremely risky and frustrating. Young drivers here usually mark the car some way to warn you to be super aware of their unexpected behavior, I'd expect self driving cars would to the same.

It means that you would have been surprised if the jogger was making a jump to her left (avoiding hole in her path for example), forcing any driver to hit the brake.

Of course you are correct. However, we don't need to make those incidents more frequent than then already are.

Instead of relying on other drivers' signals, you could instead use them as extra warning, and instead of driving so close to them you're putting yourself and them and potentially others in danger if they brake when you didn't expect, you could back off to a safe distance.

Well, not sure where you read I drive close to anyone. What I'm trying to say is that erratic drivers are dangerous because you learn to predict what a normal driver would do in a typical situation (e.g. jogger too close to the lane => slow down to a safe speed and navigate towards the center).

If a self driving car doesn't reproduce the most common driving style the other drivers should be extra careful around it, some kind of mark on the outside could help.

I wish there were some external visual cue like a light or something to indicate a car is using autopilot. That way you could give them a wider buffer.

Maybe a pulsing red light, moving back and forth horizontally.

when you park it

"by your command"

The problem is that pedestrians on the sidewalk could potentially become an unavoidable obstacle at any time if they walk into traffic. For this reason there can never be a 100 percent safe system until vehicles and pedestrians are physically separated by barriers or separate structures which may happen eventually in some new urban designs.

Interestingly, it behaved just like a young learner-driver would - play it safe, be cautious, and give extra space to the joggers because a new driver is a little uncertain about controlling a 6,000 lbs hunk of moving steel so close to delicate humans.

It's astounding to realize the AI will learn and change it's habits, just like a young human driver learning.

the jogger is a problem because the legs are so flimsy :)

If you don't see the legs and don't have very accurate 3d vision, the object might just as well stand further away in the middle of the road: http://malea.lacerta.uberspace.de/up/e92e6492-ce56-468b-8e09...

I think the hydrant is the same problem: head on you don't have any parallax effect to do know the distance, so if you don't detect where the thing is rooted to the ground you might guess it's in the way.

and it completely fails do detect the right end of the lane (the moments before I was amazed how it still drove so fluid)

I think 0.25x is the right speed if you want to watch it in realtime

Does this new Telsa autopilot use stereo cameras? Would stereo camera help to have a better 3D vision? Or is a video stream enough to get the 3D data.

From the video it looks like lane detection and 2D object recognition.

It might use a structure-from-motion (SfM) running on CUDA to get a 3D point cloud out of the 8 cams. Does someone know more about how their new auto pilot works?

I'd imagine they use stereo cameras. AFAIK though, their main sensor is radar.

They do not. They have 3 forward cameras (1), but they all point directly forward and are for separate use cases. I've always thought it would make more sense to have two cameras at either side of the windshield rather than one in the middle. The downsides are you need a new method to clear water and debris since the wipers won't hit that area, and there might be limited FOV.

Stereo might require some level of calibration to account for camera alignment, maybe they want to avoid that? I would think that implementing stereo would be fairly cheap both computationally and in hardware- if they can get by this well without it, they could pare the stereo computations down to a few frames per second, on specific objects in the scene. Maybe a future update will add binocular cameras in the dash, but they are pretty confident about their current hardware.

1: https://www.tesla.com/en_EU/autopilot

I would guess that the car couldn't predict the direction the joggers were moving in. They could be jogging towards the road. We as humans would know that they were running along the road because we see their backs but not necessarily from extrapolating their movement

Very cool demonstration (despite curious choice of music!).

I did notice that the lane markings are suspiciously bright and well-painted throughout the video. Almost as if they'd been touched up by Tesla! Or maybe this area just has particularly diligent road-painting crews.

In many parts of the real world you'd be lucky to get such clear lane markings throughout the journey, certainly not here in the UK or in places that get winter ice & snow.

In my autopilot gen 1 Tesla, when the software was first released, lane markings had to be really clear, but as the software has matured, it now does less than perfect lane marking in pretty heavy rain without issue.

I think the current-gen autopilot vision code is a completely different codebase from a different supplier running on different hardware than the gen 1, though, so all those improvements have probably been thrown away.

Curious, why do you think so? Wouldn't make much sense to throw all that stuff away.

For the same reason it makes no sense to keep your Sega Saturn games when you now have a Playstation.

There are some testing scenarios which Tesla should be able to use, ut none of the software from MonilEye has relevance to Nvidia's platform.

I know I'm probably gonna get downvoted, but I cannot not bite. I'm super happy I kept my Saturn games. I still play them occasionally and some of them are very valuable now. I also hope you meant PS3 or PS4 when you said "PlayStation" and were not trying to revive a long gone console war that was very painful for some of us.

Actually, i think in AP 1 they used MobileEye and Tesla has their custom software to compliment it and with the switch to AP 2 "It runs a Tesla-developed neural net for vision, sonar, and radar processing." https://cleantechnica.com/2016/10/25/inside-nvidias-new-self...

I also read another quote from the Tesla engineering team that "nearly all of it" (the data collected from AP 1) can be used for the new AP system https://electrek.co/2016/11/07/tesla-bought-1-million-data-s...

So it seems like the analogy is more akin to Xbox 360 and Xbox One, hardware got an upgrade but they were able to run Xbox 360 games on it with some work.

Despite the different codebases, they still have the datasets that they've collected over the years, and that's really what matters (for things like lane detection.)

It would be quite weird if they didn't retrain their new models with all that data.

What does this mean? So they send all 8 video cams data to Telsa? This would mean ould mean very high upload traffic and costs - sounds unrealistic. I assume Tesa sends the GPS data, and sensor data and maybe occasional photos back to HQ, but not full videos of all 8 cameras - or do you have a source/reference?

Of course they don't send all that back, but they didn't train the original on that data either.

What does it do on roads without any lane-markings? My country has lot's of them outside the cities.

Or heavy rain will do the same. When it's dark and rainy, we often have to drive almost blind here.

It does two things.

Holistic path planning uses the entire scene to identify the lane, not just the lane markings.[1] This is exactly how a human drives on a rural road.

Once enough fleet learning data is gathered, high resolution maps will mark out the road surface to high accuracy (cm).

Of course that's no good unless the car knows where it is within cm! For this Tesla uses sensor fusion with GPS, an onboard IMU (similar to a smartphone), four wheel wheel odometry (via the ABS sensors), steering angle, and torque delivery. Owners report that the Autopilot 1.0 cars know their position in a parking space to within about 10 cm. And it has demonstrated high accuracy and low drift in no-GPS situations[2].

This type of sensor fusion (Kalman filtering) was originally developed for the Apollo spacecraft, and was one of the responsibilities of the famed Apollo Guidance Computer.

[1] http://wccftech.com/tesla-autopilot-story-in-depth-technolog...

[2] http://motherboard.vice.com/read/how-teslas-model-s-tracks-y...

I wonder how it works in situations like we have in NZ where the recent earthquakes have moved parts of the country six metres.

> despite curious choice of music!

I really cannot wait for next video, featuring Jean Jacques Perrey's "Little Ships".

(Benny Hill music, I guess signaling humorous mayhem?)

> (despite curious choice of music!).

The first self-driving video came out a few days after Episode 1 of Westworld, where an a cover of this song featured prominently (I'm trying not to spoil it).

Elon confirmed on Twitter it was inspired by this. [1]

e: Added twitter thread link.

[1] https://twitter.com/elonmusk/status/789020178969276416

There's also the longer (3:28) video with the Paint It Black theme: https://www.tesla.com/videos/autopilot-self-driving-hardware...

https://player.vimeo.com/video/192179727 for sharing just the video

That's really weird choice, who are picking these songs? :) I can't imagine they've licensed the music either... I guess next up is the "Highway to Hell" video

My guess? Elon.

Haha, I find the image of him doing that, and no one daring to try change his mind, hilarious for some reason. I can see it escalating:

"So for our next video let's set up smoke machines on the road here, and then we'll have a camera team in front capturing it coming out of the smoke in slow-motion with "Smoke On the Water" as soundtrack. It's going to be amazeballs!"

"Umm, ok Elon..."

"Put in a lion roar as well"

"I quit."

Elon would have picked Amish Paradise.

It feels like they edited the video and then picked a song that exactly matched the video's length.

Like "The Wizard of Oz" and "The Dark Side of the Moon"?

I think the musical choices are awesome. It's so much better than choices that seem like all the marketing people debated in endless meetings who it might offend and so on. The company has some personality (cult of personality, maybe, but whatever :) )

It's related to Westworld, as I mention in another comment here[1].

[1] https://news.ycombinator.com/item?id=12995428

...so...going on basic understanding of IP rights management...

If I was an investor I'd be super fucking pissed off Tesla is squandering capital on NOT CHEAP sync rights for Rolling Stones songs to be used in their PR materials.

If I was an investor I'd be super fucking pissed off Tesla is ignorant enough about Intellectual Property that they'd use a Rolling Stones song in their PR materials without paying the NOT CHEAP sync rights.

No credits at the end? Yeah, I get the feeling they didn't pay. Before you want to dog pile me for doubting if Tesla did things right, keep in mind this is a company that blows its own deadlines and wants to lock in its IP...well, I've seen a thing or two over the past 20 years of online life and I remember when Microsoft paid out the nose to get the Rolling Stones to license "Start Me Up" for a Windows campaign. The rights holder to some Rolling Stones material broke up The Verve following "Bittersweet Symphony" case.

Yes, I'm an IP asshole, but I'm a lot better at it than most.

I tweeted the Vimeo link to the Rolling Stones so hopefully we'll get to the bottom of whether Tesla paid or not. Why is this of any importance? Because if it was my song they were using I'd pursue the $250,000 per infringement max penalty for being a billion dollar company and shamelessly ripping off an artist. Just because it's the Rolling Stones, who I wish would retire, doesn't change the rules on the books, as much as Elon & some of the culture might wish was the case...

The $250k penalty doesn't apply, because sound recordings made prior to 1972 (like Paint It Black) aren't subject federal copyright law, but to earlier state copyright laws. In this case, that's supposedly California Civil Code, specifically articles §980 and subsequent. At least that's what it seems to me - IANAL.

From your post it sounds like you should short TSLA since you seem to have some sort of pent up pessimistic anger against them.

Impressive to watch. The Tesla drives quite politely, but not annoyingly so (read that as impeding traffic). It actually stopped for the joggers who wasn't actually in the road, but very close. It also waited for a bicyclist on the right to turn left in front of the car.

Looks pretty promising, not bad, especially since the trend in software (and sensors) is to get better over time.

On perfectly marked roads, in fair weather, with no bad behaviour seen from other drivers in the video. Impressive, but the technology has a long way to go yet before it can replace human drivers.

I do not understand why the focus on self driving is so much on the whole trip.

90 - 95% of the time I am driving long stretches, I spend on highways. No problem if only that part would be done selfdriving. Those few minutes in busy town centers I do not mind to do myself. No problem of driving short stretches (10 - 15 min) either.

maybe your driving behavior is different from other people. I live in a city and do about half time on city streets and half time on highways. Anyway, consistent performance on all parts of the trip is needed to convince buyers.

> needed to convince buyers

Buyers were convinced by "cruise control", which just (barely) keeps a set speed.

s/convince buyers/avoid mass accidents caused by drivers becoming even less attentive than they're now/. Cruise control doesn't cause that, but a "90% there" self-driving solution will.

Certainly. That's a completely unrelated debate though.

It shows that the car would have been able to drive the route completely without a driver. This is handy if you want to pickup somebody, want to avoid high parking fees, need to charge your car without actually doing it.

Doing those things right now in a Tesla is probably not safe just now, but it shows that level 5 autonomy is not so far fetched and 2020 is a realistic date.

On that particular day, under favorable conditions. There's a lot of work between 80% and 100%. Showing the 80% doesn't show anything about how long the last 20% will take.

There's virtually zero chance we'all have level 5 autonomy within 3 years.

The tech is impressive but it has to be absolutely bulletproof. I like Tesla but I find it annoying that Musk doesn't deliver anything on time and it seems like Tesla delivers way more demos than real products.

Autodriving 90% of the time would be a fantastic feature.

Autodriving 100% of the time is where it gets revolutionary (mass-layoffs of truck drivers, taxi drivers, summon features that park cheaply outside of downtown).

The former is great, but there's a lot of hype going around right now so it's not strange for people to be cynical.

I think anywhere between 80% and 99.999% would cause a lot of accidents... "good enough to not pay attention and be inattentive when it needs your help, not good enough to not need your help."

It depends on where the 80% is. If it's "80% of the roads in the country" that's one thing, if it's "at random points, at any time and on average for 20% of your journey you'll be taking control" that's another.

It's all about predictability. Even just full motorway / major road driving would be pretty awesome, and the car would know when it's coming up to a section you need to drive and you can have plenty of warning. Not "HELP DRIVER I SAW EITHER A LEAF OR A CHILD QUICK TAKE OVER"

If I have to pay attention to the road while the car is on autopilot then it's basically useless -- I might as well drive the car just for the stimulation.

If/When it's good enough that I can sleep during a long highway trip (and pull into the last rest stop before my exit if I don't wake up) then I'll be happy.

Forget parking, the cars would be driving, ferrying people and goods around, as much as possible when not charging. No need to own one, just hail one when you need a ride.

It will be interesting to see how people take to not owning cars. People like customizing their spaces, and I can see people with young children not wanting to deal with all that setup multiple times a day...

That's practical, and for people like you and me who aren't bothered by the idea of not owning our own vehicle, but it's very culturally desirable in the US to own a personal vehicle. Not an unsolvable problem, but it will definitely be a bump in the road.

i don't think its an issue. live in the city? dont own a car. just like today. uber is way cheaper than owing a car.

live in the gonnies? own a car. hailing one would take forever. having cars near you in the gonnies,that you dont own wouldn't be economical.

basically, there would be less vehicule owners but not that many less.

I do wonder how many people living in cities would still choose to own a car if they could just get a self-driving one on demand.

For example, friends of mine recently bought a summer cottage. It's a little over an hour's drive. They don't have a car, so they currently rent one or try to borrow one when they want to go there. Once there, the car will probably sit still for a week.

Currently, there's no really good solution to this. Renting is expensive if you just need to drive a couple of hours a week. Buying a car might make more economic sense in the long term, but it's still pretty expensive for the purpose. Taxi rides like that cost an arm and a leg too. But if the car could drive itself out of the woods...

Of course it's a question of pricing, and I'm guessing we're still some way off from having self-driving cars navigate dirt paths that pass through people's yards as is the case here.

It is so important to do the whole trip because switching costs for the driver from "totally inattentive" to "grasping the wheel and driving" are measured in seconds.

Anything less than 4 is asking for trouble.

Do you assume that the car, after recognizing that human intervention is required, continues to move forward into the path of danger while it waits for the human to take over?

I assume that there are a huge number of situations where "quietly pull over and let the human drive" is impossible.

I'm not talking about "oh hey, here's a situation I can't handle" and more "take over now".

Volvo takes the position for their 2017 that they cannot demand that the driver take over. The car can ask, but must be prepared to pull over and stop if the driver doesn't push the "Manual Drive" button.

Here's the first of 100 self-driving cars Volvo will put in customer hands in 2017.[1] This shows close-ups of all the sensors - high radar and camera behind the windscreen, low LIDAR and camera in the nose, and side and down-looking cameras on the door mirrors. No high-mounted LIDAR, though. These vehicles are intended for self-driving in all weather conditions in Gothenburg, Sweden.

[1] https://www.youtube.com/watch?v=nwcSeRE-XEY

Sure. But didn't the Volvo CEO push "Level 4 only"?

They certainly are saying they'll only release a Level 4 vehicle - the Tesla is still expecting a human to take over.

NHTSA levels: [1] Level 3: "Vehicles at this level of automation enable the driver to cede full control of all safety-critical functions under certain traffic or environmental conditions and in those conditions to rely heavily on the vehicle to monitor for changes in those conditions requiring transition back to driver control. The driver is expected to be available for occasional control, but with sufficiently comfortable transition time. The Google car is an example of limited self-driving automation."

Tesla's old system was Level 2. Google and Volvo have level 3 in test. Tesla's new system is supposed to be Level 3. There's a bright line between level 2 and 3 - NHTSA says that at Level 3 and above, the manufacturer is responsible for accidents and cannot blame the driver.

[1] http://www.nhtsa.gov/About-NHTSA/Press-Releases/U.S.-Departm...

You don't mind.

However statistics say that most of the accidents happen at the beginning or towards the end of the trip. There are environmental reasons (city vs. sub-urbs vs. cross-country) and there are human reasons.

A real convincing promo video would be in stormy weather with angry, unpredictable drivers and random jaywalking kids.

Agreed. Throw in some fog, high-speed stop-and-go interstate fun, and moving from the interstate into a crowded city full of people jaywalking and running stoplights trying to get to where they're going.

And inevitably the AI will still be/become faster and safer than humans.

Absolutely yes. I can't wait for that day. We'll end up saving more lives per year than if we somehow banished war, at least in industrialized countries.

What's weird is the period we're entering, where the technology is becoming more and more capable, but we're still a very long ways away from full auto. Yet the hype machine says it's just around the corner. I'm betting we're still 10-15 years out. So how do we handle situations where the driver has to pay a little attention, but not that much? In airplanes, this has been a very dangerous area -- and things in airplanes usually happen much, much slower than they do in traffic.

> What's weird is the period we're entering, where the technology is becoming more and more capable, but we're still a very long ways away from full auto. Yet the hype machine says it's just around the corner.

I think it would be worthwhile to figure out how to shut down the hype machine, or at least tune it down a few notches. Dreams and visions are fine, but every time I switch the current tab from HN to Facebook, I get to see overhyped bullshit predictions about AI, nanotech, self-driving cars, etc. being just about the corner, as evidenced by whatever misinterpreted paper or video news sites could find that day. And the non-tech crowd is totally buying that. Not just regular folks, but also investors and politicians and all the other people who then end up causing resources to be wasted because they buy into the bullshit.

It's interesting, though, to see rise of the robots talk breach tech circles and enter into mainstream economic and political discussions. Between the arrival of imagenet in 2011 and now the drums have been steadily getting louder.

In 2013 an oft-cited and alarming study out of oxford suggesting most jobs will be automated over the next decade was released. It was followed by a series of influential books whose authors ran the lecture circuit: 'The Second Machine Age', 'The Zero Marginal Cost Society', 'Superintelligence', and 'The Rise of the Robots'.

There were some bold bold and well publicized statements from respected luminaries such as Bill Gates, Elon Musk and Stephen Hawking.

Aggressive maneuvering to hoover up machine learning talent, and bold investments from automakers pursuing autonomous driving has only added gasoline to the fire.

Automation became the talk of the town at Davos in Switzerland. There's been a rising chorus from Basic Income supporters.

Now the hype is out of control. Nobody is actually looking at the technology. It's okay, though. Hype is self-correcting.

Can we use a game environment to artificially synthesize scenarios? I don't know if there are any that can generate realistic video.

A few months ago I red about researchers using GTA V to train some self driving algorithms. I googled it and it looks good

Throw in some electric scooters diving in and out, bicycles, crazy couriers, cars parked on the road, someone pushing a hand-cart, an ambulance, and at least one of each of these kinds of vehicles going the wrong way and disregarding all traffic laws.

Or, as we call it in China, "driving".

yes. If Tesla really wants a good test, try autopilot in Morocco!

all at once: moped taking over from the right, in a round about, truck entering the roundabout without slowing down. donkey & cart on the side. woo hoo!

Well, give them time :). Maybe they can use this video to leverage the generated PR into getting themselves allowed to shoot the next video at more difficult conditions :).

Tallahassee, FL in the summer would be perfect: heavy rain storms daily, college students everywhere, and everyone drives with a supreme sense of arrogance!

This is pretty impressive. Still, at 00:54, getting a closer look at the female runners? Someone following could have gotten caught out if driving too close / playing Pokemon.

A human driver (well, at least myself) would simply have lifted the foot off the accelerator somewhat and drifted closer to the centre line when passing - seeing as there was no oncoming traffic.

Perhaps the car knows it's own capabilities better than we humans do.

Maybe it knows that if the joggers were to suddenly jump out, doing what you or I would do isn't enough, so it slowed down to a near stop just in case.

Also, it's a bit of a "blind corner", so drifting out toward the middle of the lane might not be the best idea (again, even though it's probably something I as a person in that scenario would have done).

I've wondered about this. Totally defensive driving would reduce a car to 5mph - somebody might jump out from behind every blind corner or mailbox.

Truth is, we all drive fairly irresponsibly, trusting in the rules of the road and human self-preservation to keep most stuff out of our way. And if we do hit somebody who dashes out on the freeway, well that's an accident or 'their fault' or 'nobody's fault' so no foul.

Will we allow automated drivers that privilege?

I think it's going to take some societal change, but we will have to.

Computers are just an extension of ourselves. The people making these systems will make mistakes, these systems will kill people (and arguably already have), but hopefully they will kill less than we currently do by not having them.

There's this stigma around "computers" where people treat them differently than everything else. Yes, you're getting into a car whose programming could cause you to die, but you are always relying on this kind of thing. Getting in a regular car, you are relying on the engineer who designed the braking system to have been correct, you are relying on the factory worker to have been paying attention that day, and the QC robot to have been programmed correctly, etc...

We need to get away from that, we need to treat the computer in your car as you would treat it's braking system. Accidents will happen, people will get hurt, bugs will occur. If they are systemic, there will need to be changes, recalls, and fixes. But the car will never operate in 100% of conditions flawlessly, and neither will you.

The big difference is, if I make a mistake and hurt myself, my family, or others, I have myself to blame. If my car makes a mistake though, I have someone else to blame.

Even though the statistics may point to self-driving cars being safer in the aggregate, would an individual be willing to give up control to software which might make a mistake and kill them? Even if, statistically, they might be safer?

Will people make the rational choice, or the emotional choice?

just film and show a complete flight from an airliner from the cockpit. flights are fully autonomous except take off and landing, which still are computer assisted.

all crashes occur when the pilot overrides the flight system or the runway has no support for computer aided landing (ils).

thats the only reason flying is statistically safer than driving.

Funny: since the 80's they've been able to take off and land too. Which could avoid most of those crashes. But the pilots (and FAA) won't allow it.

The issue is really about capitalism.

A video where a car manages to stop because a drunk falls into your lane will make the stock soar.

A video where a car hits an old lady who jumps in front of your car will make the stock crash.

There are people on either side who want this technology to succeed or fail (not self-driving, i mean electric cars)

seems like a dumb statement. why in hell would you think its ok to hit the lady when the car can avoid it?

getting there 30s later on average is worth killing grandmas? with automated cars you dont even have to pay attention or care. get in get out.

Yeah. We do (or should) typically drive "defensively." But in practice that doesn't mean that we defend against everyone potentially pulling some utterly crazy stunt at any moment. For example, we watch for pedestrians but we don't actually worry too much that there might be someone hiding behind a tree who is going to jump into traffic. And we may be somewhat cautious at cross-streets but we still have a general expectation that a car isn't going to just randomly blow through a red light in the middle of its cycle.

Perhaps the AI also needs to be able to perceive the motivations of the humans ie read body language. If the joggers suddenly turned their heads to the left and twisted their bodies etc, then that would to a greater degree warrant the extreme caution. There's a lot of work cut out for the machine learning engineers.

I agree, but until then I'm guessing they are going to play it safe.

That being said, looking at that portion of the video again it looks like this may have just been the system having trouble figuring out if the people were in the road until it was closer.

I was wondering if stopping for the joggers was more of a bug than a feature. I don't drive much, so I'm definitely not an expert about what is best in that situation, but I would have drifted over the middle line to give them a nice berth and kept going.

As a cyclist that was nice to see.

I believe Tesla's AP 2.0 is built on NVIDIA's platform (with probably a lot of custom stuff, of course). If you want to learn more about how that kind of system works, that CES 2016 demo is very instructive, especially from 42:50 onwards: https://www.youtube.com/watch?time_continue=2573&v=KkpxA5rXj...

Reading through Tesla blogs and watching this video, my understanding is the self driving hardware is cameras and ultrasonic sensors.

Does the lack of lidar not scare anyone else?

Also, Out of curiosity, does anyone know how a Tesla behaves if you point ultrasonic transducers in the same frequency at it? Do they have special modulation to avoid tampering in this manner? I imagine you could confuse the car into thinking there are barriers it can't see.

> Does the lack of lidar not scare anyone else?

I don't think lacking lidar should be inherently scary, humans don't have lidar sensors either and they do well enough. I do hope that cheap solid state lidars do come to market to improve low visibility driving nonetheless.

The human eye has an incredible dynamic range (>20 stops). It also have very high resolution at the point of interest. I seriously doubt that Tesla's cameras are anywhere close to the performance of the human eye. LIDAR can also make up for deficiencies in the software. The human CPU is also probably a little more advanced than the one driving this car so anything helps.

At this year's DEFCON there was a lecture [1] about the sensors on a Tesla and yes - you can confuse both the ultrasonic and the millimeter wave radar and hide an obstacle or put one that is not there.

[1] https://www.youtube.com/watch?v=Fvasj4qQ7Lk

Finally got around to watching this. Super interesting. Exactly what I was thinking about.

Thanks!

I would absolutely love to see how it handles roundabouts in the UK. Some of the ones I use daily are pretty hairy even for a competent driver.

And then there's the magic roundabout in Hemel Hempstead! https://www.youtube.com/watch?v=ih223jffmek

Much better music video of real self-driving: [1] This is what traffic looks like when you totally eliminate human drivers.

[1] https://www.youtube.com/watch?v=zm_rlLyelQo

It's so amazing, I was wondering if it's a computer simulation or for real. Great job! http://www.ect.nl/en/content/euromax-terminal-rotterdam

To be completely fair, you can show me almost anything with Benny Hill music in the background and I will enjoy it ;)

Anybody knows if these 3 cameras on the right are the only ones used by the auto pilot?

No. The system uses 8 cameras. Three forward facing (narrow, wide, standard), two rear facing sides (the two showed here), one rear, and two side cameras.

Thanks. makes sense, and I guess makes easier to implement some kind of depth perception.

I think watching this in real-time would be more informative. At 1:32, it pull out on what looks like could be a busy road, then stops before starting again. It doesn't look so bad sped up in the video, but not great in reality.

I agree about real-time. I get that it would be more boring but I find it hard to judge how the performance differs from a typical human driver in the sped-up version.

Here's one someone did, from the teslamotors subreddit. https://www.youtube.com/watch?v=hLaEV72elj0

What's the most amazing is that Elon mentioned on twitter that it's only using data gathered in the past month since they announced the hardware/first video.

But why wouldn't they use old training data to make it even better ?

Could be because there'd be assertions made under the old system, that were later found to be unhelpful or unwise?

A simple one, the old system didn't have 8 cameras so you don't have the full training data you need.

That's a little bit of a misdirection. They certainly learned a lot about how to build their models and what data is important, and this results in more effective training.

Fastest speed clocked in at 35mph during the entire trip with an average of 15-25mph for remainder of the time. Does this suffice a realistic use case? Specifically in US?

In town, with the level of traffic seen? Yeah, it's keeping to the posted speed limits. Realistically, I'd be doing 5 to 10 over except in explicit residential zones, when there are cops (limit to 5 over), and when traffic doesn't allow it.

Couple spots where I think an inattentive driver could have easily rear-ended the car. ~0:55 mark is a prime example.

I understand that whether the driver was paying attention or not, he or she would be at fault, I'm just pointing it out.

So it appears it's reading the road signs? I see it stops on the intersection with a stop sign where's no traffic lights. But what if the sign is damaged or removed? The car will go straight without stopping?

Maybe it's relying on maps & GPS data.

What if the GPS data is outdated?

What would you do if you'd remember there used to be a stop sign there, and now it's gone?

I'm still having a hard time not stopping at the intersection by my old high school and they removed the stop sign there 3 years ago!

No tesla expert but I think they have an extension package that would let the car adjust itself based on previous "experience".

IIRC it would automatically make a note of your location when you pass speed bumps or bad patches of road and adjust the suspension accordingly.

I hope they are using GPS/Map data for existing signs along with visual identification of the signs. Even when the gps is not accurate enough, the system should amplify the accuracy of the location based on permanent objects and the topography.

Either on the first video or some news, i think i heard the ability to drive where the lanes are not clearly defined. Things such as snow would absolutely requires such reconstruction.

What would you do if a stop sign was removed?

If a stop sign was removed, I probably would've driven straight through without stopping too. How would I know there used to be a stop sign?

Yes, i mean what happens more often- if sign is not fully removed, but bent by collision or turned sideways by hooligans or something. Human would recognize that. Car will not.

Most algorithms can handle slightly turned signs. However signs that are gone could be dangerous.

Sounds like the cars should share a centralized database of critical sign locations, and cross reference their GPS location with the database. When they approach a location that the database says should have a sign, and one is not detected, it should behave as if there was a sign, and report the incident to somebody.

> But what if the sign is damaged or removed? The car will go straight without stopping?

Probably the same thing a new-to-the-area human would do if the sign was removed. Where I live there are many intersections where only 1 road has stop signs. I'd imagine there are other areas were the norm is a 4 way stop.

As for damaged, I would imagine it can still recognize the sign as often in image recognition you do not need the whole picture to confirm a match.

If there is no STOP sign, I don't see how it would know to stop. However, what's interesting is if it goes through an intersection with a STOP sign and the next day the sign has been knocked over by strong wind - will it stop? What if the sign is missing, but there's a STOP sign on the opposite end of the intersection - a human driver will infer that his own side also has a STOP sign, but will the car?

Normally if there's no priority sign, the principle of priority to the right[0] should be employed.

[0] https://en.wikipedia.org/wiki/Priority_to_the_right

A stop sign is paired with a stop line in the road which can be seen. It will also see other cars and react accordingly if they are not following their own stop signs. An SDC doesn't really need a stop sign.

I'm disappointed that radar images aren't included in the video. They ought to be very interesting, a whole new perspective on the world around us.

What the car sees with radar does not translate directly to an "image" (it's not 2D radar, it's 1D with time of flight for depth).

Still it can be visualized.

I'm curious about the implications of this technology (which is amazing) on courteous driving habits.

I regularly stop to let people cross the road, I cross the center line when safe to give people and cyclists (if I plan to pass) extra space etc. I suppose, in this Tesla at least, you can just take control, but some other proposed implementations seem to have a less traditional UI.

Well it was certainly fun to watch. Aren't there more detection methods than just cameras? This is certainly the best one I have seen from them but I still want to see it at night, in rain, and such, times when people need the most help with driving. This was still too nice of weather to get a good impression of how far its come.

Also, if the driver wants to exceed the speed limit of a road will autopilot allow them? If so, who becomes liable since the system will obviously know the speed.

If the road had not been curved when it came upon the joggers would it have passed without stopping? Is that something that needs work on? I understand its erring on the side of safety.

So next drives, a night run, a rain run, and if they are really ballsy then both at one time. Would be cool to have someone throw something in front of it while its going through the parking lot, beach ball, etc.

It seems like the right-bound of the lane (the right red line in the midview camera) was not calculated, so it thinks the joggers were inside a big lane. that'd be my guess as to why they would think the joggers were not "outside" of the lane for the system to breeze by. At 1:23-1:24, there's a person walking their pet and it looks like they were really close, but the camera was able to detect the right boundary.

I'm surprised by the number of Stop signs everywhere. Is that normal in US? Also, there are moments in this video where the car was turning left and a driver coming from the opposite direction was turning right - yet Tesla turned first? How?

I really wish they provided a non-sped up video.

I agree, the roads are perfectly marked. It's early days but incorrectly hardly complex edge cases, like graffiti on the signs, could be a matter of life or death.

I wonder if the AI just detects an intersection and if it doesn't know what to do, disengage?

Why would a missing sign cause life or death? It has radar and complete around-the-car vision. Should do a lot better than a human.

I'm almost certain that I can't see around corners though.

I'd love to see this in $heavilyPopulatedIndianCity. ahha, found this : http://www.theunrealtimes.com/2016/09/17/top-secret-uber-tes...

these guys are doing it right (teach your car to drive on the other side of the road) : http://www.deccanchronicle.com/technology/in-other-news/0103...

Is a realtime version available?

I notice at 0:24 in the short version (0:41 in longer version [0]) it seems to stop early for a stop sign and then resumes driving to the stop sign. It looks like the vehicle in the oncoming lane may have caused the stop. I only noticed because in the longer version the person in the driver's seat lifts his hands when the stop occurs (not as noticeable in the short version).

[0] https://www.tesla.com/videos/autopilot-self-driving-hardware...

I think the most interesting part of this video is that there's a lot of fog.

Is it doing a weird maneuver on 1:32 or is it just me? Something like stopping right after turning. I'm not full aware of the USA transit laws so this might be required but it doesn't seem likely.

That's really impressive, but I really want to see it driving through rush hour in a densely populated old Italian city. Let's continue pushing the envelope.

I did it. Once. Friday evening in Milan.

Something you wouldn't wish on anyone who is even remotely your friend.

I think the choice of music is quite clever, makes it seem jolly and approachable, instead of the connotations of solemn futurism that self-driving usually has.

The car seems to have a bad habit of turning its steering wheel at stop signs before making its turn. This isn't really a problem at stop signs, but would definitely be dangerous if turning left on a green while waiting for oncoming traffic.

The only left on green we get to see in the video is at 15s, but it didn't have to wait for any oncoming traffic. Hopefully it knows to treat those situations differently.

I would love to see how autopilot works in different cities (e.g Mexico City, Rome, Buenos Aires). I Just found "I tested Tesla Autopilot in Manhattan traffic — and lived to tell about it" http://mashable.com/2015/11/06/tesla-autopilot-new-york/

I wonder how these self-driving systems are going to deal with wildlife or even domestic pets running wild. It's fairly common on my commute to see deer grazing along the roadside just a few feet from the pavement. I typically slow way, way down and creep by as they'll walk right onto the road if the mood strikes them. Getting enough test-cases for the software could be difficult.

I really hope Tesla would do a demo on the roads of India. If a car can self drive there I would have no trouble purchasing one. Indian drivers are some of the most resourceful drivers I have seen on my visits there, yes its complete chaos, but how they navigate through all that, while utilizing every bit of the road is, just amazing.

p.s I really enjoyed the choice of music on that video.

As I watch all the views from the cameras, and watch how it is making decisions based on that information, I have to ask: who says a machine can't be conscious? Who says it can't have free will? How do you define those terms so that includes people and animals, but doesn't include this vehicle?

</deep philosophical mode>

Daimler done that 3 years ago even in high density city traffic. Daimler could even pass vehicles that covered half the lane and didn't need to stop for them. https://youtu.be/SUOC8tE4bdM

But the video does not have Benny Hill ;-) (Just kidding)

Are you implying timelapse format is intentional?

That's more a bunch of edits than an end-to-end demo.

I did not invested the time to search for that version and I doubt it exists.

And it has a proper emergency stop button!

I'm curious what that button does? Does it go full "emergency brake" mode and stop the car ASAP, or does it give control back to the driver?

I could see the former being useful during development, but I wouldn't want a "slam on the brakes" button in my car. And the latter could be much more easily worked out by just switching under any driver input on the steering wheel.

I work in the automotive industry for a manufacturer. All of our prototypes have big emergency stop buttons in the center console using 3D printed adapters to slot into the cup holders.

They just kill the engine and disconnect the battery. It's a legal requirement. Nothing fancy.

So if you are at speed, the car starts engine-braking (with no proper brake force distribution between front and rear wheels), and the power steering isn't working, right?

And a horse could do it centuries before that, but that's not a fair comparison either.

The car in that video had so much extra custom stuff it's not even funny. And while it's still a technical feat, I don't see every mercedes being sold with that equipment in it already.

The reason this is amazing is because Tesla is planning on switching this on for all of their new cars in the very near future. It might not be the most technically advanced, but it's here.

It's just like how Mercedes (and others) have shown some fantastic "Tesla killer" electric vehicle concepts. That's nice, but it's not really something that competes with Tesla until it's available to buy.

While the others make promises, Tesla's making cars.

The equipment is available in the current S-Class and the most current E-Class, when we talk about the sensors and the actuators. It is different with actual ECU who does the computing stuff.

BTW I doubt that the Tesla is ASIL D complient in this sense. AFAIK their is no ASIL D implemented component right now. Everybody avoids that. But this is required to be fail safe.

Where is all that extra custom stuff? Because the car looks perfectly normal, no Google-esque radar dome on top.

Very cool video. Said Daimler was using 8 radars and three cameras, so sort of the inverse of the Tesla approach.

The Tesla video is a bit more exciting, because it is using a car that consumers are already driving, whereas the Mercedes has "aftermarket" sensors and a computing platform built in. Mercedes is impressive on the other hand, because that was three years ago.

A DLR/TU Braunschweig cooperation was able to drive through light traffic 6 years ago:

https://youtu.be/BrmorE5W1tM

Having a working implementation in a consumer grade car is impressive, but the ability itself is not novel.

Actually, you can buy the car too. I have been driven in a S-class about 3 years ago that had the same capabilties the Tesla here has.

It's been known for a while that many of the other semi-autonomous platforms are out probably a little early compared to Tesla's and feel a bit dangerous. It's likely their fully autonomous platforms are similarly not as ready to go.

http://www.caranddriver.com/features/semi-autonomous-cars-co...

Anyone understand how it's color coding the boxes on the right? Seems like green, blue, and purple all likely stand for different categories of objects.

there's a legend at the bottom of the video

I wonder if it can detect when it doesn't know what do it. Like when it can't find the edge of the lane.

Camera based... so if you drive through mud or it's raining hard does this still work?

Do your eyes still work when you drive through the mud or it's raining hard?

Do the cameras have windshield wipers? At times if the window gets too dirty or it's raining too hard I need to pull over to clear it, even with my high tech eyes. Will the car go into a fail safe mode? Just stop in the middle of the lane because it thinks the bug-strike on the lens is a person?

Ok... I'm officially old now. I've been keen on the idea of self driving up until now. Watching this just makes me want to grab the wheel... too much traffic, too many people on the side of the road... nope I don't think would I be comfortable.

Oh well... I guess I'll have to wait for the mini-van style with blacked out windows :)

That's impressive, but still an easy course compared to what I see daily in my (large) city. I'll get mine in February and am curious to see what it'll do in those narrow roads where 2 vehicles going in opposite directions can't pass each other sometimes.

Wouldn't the answer be "same as an average human?"

I hope not; the average human can't pass an obstacle at 1-2cm distance safely, a machine could.

Can someone explain the left turn on 1:41? Seems a bit aggressive.

Watch out at regular speed. Ashley consider the acceleration power a Tesla has. It can get into a tight spot quickly without altering traffic.

what happens when a few autopilot cars come to different roads on a 4 way stop?

What about driving through a parking lot (lots of cars around).

Seems dang near impossible to deal with all the edge cases.

the design possibilities and field day designers will have when steering wheel and forward-facing seats aren't retrictions will be amazing

best part at end: you can save time by exiting before the car parks itself.

I thought the best part was the car doesn't need to park anywhere near. Large parts of cities will be freed for us to use.

Large parts of cities will be turned into roads for driverless cars to ferry themselves to and from the office 2x as many times to save the owner $10 on parking. Traffic will go slowly because driverless cars don't care about speed when no one is inside. And then all the plebs still driving their own cars will be stuffed.

This is kind of true, as much as this tech is fascinating, I can't help but feel trains and better planned cities are the real solution to the car problem.

Even in parts of Europe I noticed they put big trucks on trains to automate a large part of the journey and a driver finishes the last leg of the trip.

Trains are also much easier to automate, I also imagine a train track costs less to maintain than a road.

Trains are complementary but there will always be need for personal transportation.

I never said there wasn't, however; What is obvious is that if taxis and trucks (privately owned or not) are now so cheap to run because there are no labor cost involved in having them on the road, then the next problems we will face are increased congestion, increased pollution (electric cars still consume a lot of power), increased road degradation. The only way to control this might be through increased taxes or car registration fees.

This still means the car problem needs solving.

I don't know about other countries, but at least in the Netherlands the Benny Hill song is used mainly in videos to ridicule the subject. I remember when funny 'Home Videos' were a thing on TV (before cats took over on the interweb) this song was used in about 90% of them. So to me, using this song in something as serious as a self driving car demonstration is pretty weird.

This song is also often used in a recurring absurd segment in cartoons when some people are running after each other and they go into / come out of random doors in a hallway.

Which is an homage to Benny Hill.

Probably to humble brag how Tesla made it so simple it's almost something to laugh at.

Technically the tune is called the "Yackity Sax" but it may well be more widely known as the Benny Hill theme tune.

It's just asking for someone to redo the movie with Kevin MacLeod for the soundtrack.

Come to think of it, I just feel like making a montage.

If you look at it from the perspective of the sane and rational (Tesla autopilot/a cartoon viewer) viewing down to the ridicule (Human drivers and their emotions/absurd cartoon scenario) I think it is well placed :)

On the Internet, specifically on the Something Awful forums, it was was infamously paired with footage from 9/11 and posted either the day of, or the day after, the attacks. The video is called "tribute.wmv": https://youtu.be/2AYujWCCHRk

I suspect it has to do with the video being sped up the same as the segments on the Benny Hill were, which masks one of the challenges of the self driving car. From my experiences, at least the northeastern us, driving the speed limit, in the traffic density shown in the demo, almost instantly attracts tailgating drivers who want to go faster.

The Benny Hill theme tune was a pretty unfortunate choice of background music, not sure if that's the impression they wanted to make (maybe it was, not sure).

Geez, indeed poor choice. I muted it first and kept listening to my deephouse podcast :)

What? Come on. The TV show was ridiculous but the theme still brings a smile to my face.

Appropriate in bringing light heartedness to a typically sped up video.

Can everyone chill the fox out about a theme tune?

When the last video came out, there were quite a few people saying "wait, that video's sped up, right? It doesn't seem to be taking turns at a safe speed."

I expect Tesla got feedback indicating that some people didn't notice that the last video was sped up, so they decided to make this video a lot more obvious by speeding it up even more and adding the music. I agree, it's not really a good choice, but I can see how they might have come to that decision.

Indeed; for me this music has two primary connotations - "funny" and "sped up".

I like it.

The tech community tends to be a bit dour and pompous. We tend to overblow every tiny announcement into some world-changing event. Look at Apple's recent announcement video for their $300 book for a particularly gratuitous example.

It's healthy for Tesla to retain a sense of self-deprecating humour. Self-driving cars are massively important, but they're also a bit silly and strange. There's no harm in acknowledging the absurdity of a car that drives itself. There's no harm in bursting your own bubble.

Wow so many comments about the Benny Hill music. I for one love that Elon Musk's quirky sense of humour. So many people take themselves far too seriously.

Do you think that Elon Musk was the one to put this out (or even design the video)?

I'd have a hard time believing that, to be honest.

Probably not. But if he doesn't like it, I wouldn't want to be the person who did put this out.

Elon is known for being closely with involved with his companies' PR. Here he acknowledges the music choice explicitly: https://twitter.com/elonmusk/status/799908000689336320

I think Elon Musk had a lot to do with the video, I imagine the music was his choice. Very few other people could suggest this tune due to its previous comedic use.

the barge names also show this

We are talking about a car that decapitated its driver this year.

Why should an unfortunate death involving something mean that their marketing can't be fun or lighthearted.

While watching Harry Potter. Perhaps the Harry Potter theme would have been more appropriate, but I'm sure that licensing would have been prohibitive. I can hear some similarities, so maybe it was an NN based suggestion

That's still just an allegation from the driver of the semi (he said he heard harry potter playing, and there was a portable DVD player found, nothing more was released).

But what is confirmed is that he was speeding at the time as well, and after that incident Tesla put out an update preventing autopilot from being set for more than 5mph over the speed limit.

>>after that incident Tesla put out an update preventing autopilot from being set for more than 5mph over the speed limit.

That's not correct. I routinely drive with autopilot set to +10mph over the speed limit.

Ah, looks like it's only in effect on "Residential roads or roads without a center divider".

His fingers, so twitchy.

What I've learned from this is how boring demonstration videos of self-driving are, regardless of whether Yakety Sax plays in the background

Reply on news.ycombinator.com