Tesla's “full self driving” beta is potentially dangerous
roadandtrack.com
roadandtrack.com
I think it's fair to say that there are definitely some sketchy moments in there. The one that surprised me the most was marked "Near Collision (2)" in the video (12:25). A car really close to the Tesla doesn't seem to be picked up at all until it's too late: https://i.imgur.com/BG0gN2R.jpeg
You can see in the first frame there that on the screen there's nothing in front of the Tesla, it's just the usual white background. But right out the windshield we can see the car is already very close. At the bottom frame which is a split second later, the car is finally detected and we can see the screen now shows the car. But at that point the driver manually intervened with a sharp swerve right as they were about to hit the car. Going frame by frame on Youtube the green box indicating the car was only there for 4 frames before the driver disengaged.
I was aware that self driving is far from ready but I didn't know that even basic detection stuff like this is buggy. Double parked cars, people walking in the street, etc, I would expect to be problematic. But this is not really an edge case, it's just a regular car at close range with no obstructions in between...
Very hard to see because the brightness of color yellow is pretty close to brightness of the background, almost no contrast. They should have picked some other color instead.
I would encourage anyone to watch the videos and make up their own mind, bc I do feel that people over-correct in their Tesla skepticism sometimes. But in this case, it seems warranted.
At the very least you should be required to put a visual sign that your vehicle is in "FSD" mode. Sort of like how we require "oversize load" signs so that others can take extra precautions around those vehicles.
These indicate speed. I would guess FSD should have something similar.
[0] https://www.quora.com/What-is-the-usage-of-three-green-light...
The vast majority of Teslas aren't using this FSD beta but the older and more limited Autopilot software.
You'll only have to start worrying about this in the next few months, or last year, or the year before, etc (depending on which Musk tweet you go by).
One should add that in this case it's tested by consumers, not certified professionals.
Conspiracy theory: Tesla wants to show how bad the system is early on to raise public awareness and make people drive more cautiously around teslas.
https://news.ycombinator.com/item?id=26519495
It's failing to safely do a very basic turn across traffic on a clear flat road with wide open view to oncoming traffic, on a clear sunny day. There's not even any weird concrete barriers, shrubbery, signage, traffic light control boxes or such blocking the view of the oncoming cars.
Even if Tesla owners opting into this beta test are willing to put their lives on the, uh, line, I'm not sure everyone else on the road is as keen as they are to be volunteered into this test.
I'm not sure I understand how other posters are watching this video, and are concluding that the FSD system is being 'overly cautious'. To me, it looks like it's doing its best to get T-boned.
As a baseline, this should not pose a problem.
For contrast, here's Tesla's promotional video[1] for this feature and its caption:
> The person in the driver’s seat is only there for legal reasons. He is not doing anything. The car is driving itself.
It's been up on Tesla's site for almost 5 years now.
[1] https://www.tesla.com/videos/autopilot-self-driving-hardware...
He's the legal crumple zone.
My thoughts as a driver of a car: wow, this is trash and should not be on the road.
My god, and this is on the roads?
FTFY
I believe safe autonomous cars should behave a bit strange to human sensibilites: They would drive a LOT more defensive in many situations since they cannot be 100% sure that a child is not going to pop out the bushes and onto the street. But on a freeway, maybe if they negotiated with all surrounding cars to drive as a "fleet", it would be able to drive a LOT faster than the safe speed limit for a human driver. The comfort of the passengers would become the limiting factor in many situations.
I assume from your comment that you haven't actually spent a lot of time in a car with these features, but the best way to describe is that it's like driving in grooves. You're driving, but you don't have to actively work to keep your distance to the car in front of you, stay below the speed limit, or to keep your car centered in your lane. Just like with an automatic gearbox you don't have to actively work to keep your car in the right gear, the car does it for you. Same principle here.
January 2016: "In ~2 years, summon should work anywhere connected by land & not blocked by borders, eg you're in LA and the car is in NY"
June 2016: "I really consider autonomous driving a solved problem, I think we are less than two years away from complete autonomy, safer than humans, but regulations should take at least another year," Musk said.
March 2017: "I think that [you will be able to fall asleep in a tesla] is about two years"
March 2018: "I think probably by end of next year [end of 2019] self-driving will encompass essentially all modes of driving and be at least 100% to 200% safer than a person."
Nov 15, 2018: "Probably technically be able to [self deliver Teslas to customers doors] in about a year then its up to the regulators"
Feb 19 2019: "We will be feature complete full self driving this year. The car will be able to find you in a parking lot, pick you up, take you all the way to your destination without an intervention this year. I'm certain of that. That is not a question mark. It will be essentially safe to fall asleep and wake up at their destination towards the end of next year"
April 12th 2019: "I'd be shocked if not next year, at the latest that having the person, having human intervene will decrease safety. DECREASE! (in response to human supervision and adding driver monitoring system)"
April 22nd 2019: "We expect to be feature complete in self driving this year, and we expect to be confident enough from our standpoint to say that we think people do not need to touch the wheel and can look out the window sometime probably around the second quarter of next year."
April 22nd 2019: “We will have more than one million robotaxis on the road,” Musk said. “A year from now, we’ll have over a million cars with full self-driving, software... everything."
May 9th 2019: "We could have gamed an LA/NY Autopilot journey last year, but when we do it this year, everyone with Tesla Full Self-Driving will be able to do it too"
Dec 1, 2020: “I am extremely confident of achieving full autonomy and releasing it to the Tesla customer base next year. But I think at least some jurisdictions are going to allow full self-driving next year.”
-
Elon’s just been repeating the same promise for the over half a decade now. Oldest trick in the book.
Disclaimer: I drive a Model 3
Elon operates by setting impossible goals and under delivering on them but still going faster than all the other players. Most FSD buyers are reasonably aware of the gamble they are taking.
If Tesla turns out to not be capable of deliver FSD, it's a straightforward class action to refund everyone. Meanwhile they are trying hard to build the damn thing and my kudos for that.
The reality distortion field in action. "Hey no big deal they're straight up lying to consumers and putting their lives in danger!"
They are nowhere near Waymo, who is doing actually driverless rides for the public. Tesla's confidence level is still at "you need to keep your hands on the wheel at all times".
For example, you come up to a 4-way intersection and a traffic cop there signals you to stop and says you can't go through because all the man-hole covers are off, but since you want to turn left you can go through the parking lot of the corner gas station to get onto the other road.
For another example, right after the st-patrick's day parade is over or right after the college town's team has just won the NCAA championship, the street is full of people and the car has to inch forward at a rate of half of a mile per hour and the pedestrians don't get out of the way until the car is about to touch them.
The examples you gave sound like scenarios where remote operator would instruct "don't go there, go here" and the car does it by itself. But specifically about the traffic cop, it can actually detect hand signals from them (can't find the video though).
In my state this is explicitly illegal. You may not use a parking lot as a thoroughfare, regardless of convenience.
Waymo also has the sum total of CAPTCHA learning for them.
Tesla is trying to solve a much more difficult problem. It’s unclear if they will.
Of all the self-driving companies it is likely that Tesla is dead last, behind Waymo, Cruise, Aurora, and Zoox.
And possibly a few deaths, yeah.
Having multiple orthogonal subsets of the electromagnetic spectrum at your disposal provides redundancy and diversity - two features that simply CANNOT be accommodated with a single class of sensor, not matter how advanced it may be.
Meanwhile, Tesla autopilot accelerates towards the walls or parked vehicles. Tell me again how Tesla is "at the forefront in the field".
Loads of people called Elon a charlatan, but for them criticism came at virtually no cost, unless you account for the cost of not betting on him.
When Spacex formed in '02 and Tesla in '03 many observers would have estimated that privately funded moon rockets, luxury electric cars, and mass produced solar panels would have been a bigger challenge.
I have news for you, the most common operations in AlphaGo, GPT-3 or any state of the art AI are: multiplication, addition, max, log, exp, sin, cos and random - all functions known for centuries.
It's the architecture, data and the compute that are making the difference, not the math, and all three are recent accomplishments.
So maybe they’re developing an “ASI.” But because there are already plenty of those and we call them “AI,” we might just say Elon’s trying for a harder, higher scale, and more commercialized version of a set of technologies that already exist. Kind of like all the other “impossible” things people mocked him for missing deadlines on in the past. And now, we’ve arrived at the point.
I do wonder if some of these same things could, or would have been accomplished in the absence of Musk, but with the same amount of capital and under the leadership of Gwynne Shotwell. I think they would have. Musk has a big dream, and is great at hyping stuff, but it's not like he's personally engineering the Falcon 9 and its recovery system. Shotwell hired the right people to implement the grand vision.
What I'm concerned about is the people who think that Tesla can do no wrong, and it's the most amazing thing ever. When the reality between the sales/marketing pitch, as you've documented above, and what actually exists in the real world on a certain date is so divergent.
It does seem like his is heavily involved in the engineering, and he does say he's the chief engineer of SpaceX.
https://www.reddit.com/r/SpaceXLounge/comments/k1e0ta/eviden...
That assumes the same objectives would have been pursued without Musk, which I find unlikely, given the skepticism and criticism Musk had to endure, and dispel, in order to keep his companies on-target, without a scale-back of ambitions, all those years.
The issue isn't the predictions. It's the time frames he makes the predictions about. We will have everything he's said one day, but in the mean time we need to call him out for what he's done: lying, and why he's done it: to get money.
The reason why he's so motivated is because his compensation package is so ridiculous.
If he is so uninterested by money he can of course show that by giving it to charity.
18 hour days are the "walking barefoot in the snow for three days" of our time: https://en.wikipedia.org/wiki/Road_to_Canossa
Amazing how neither sleep deprivation psychosis nor frost bite are a thing if you're rich enough and talking about how virtuous you are.
It's an unfortunate flipside to their willingness to work so hard-- they almost need to believe in the impossible to keep up the pace. Ideally there is someone between them and the public face of the company, but all too often they can't help spouting off stuff they should be keeping under wraps in front of TV cameras or in tweets.
It's like there is this enormous gulf that most others can see, but to Elon it's invisible.
So don't let anyone tell you that philosophy and theology don't matter. We are witnesses a multi-billion dollar bet being made on the basis of some extreme views about the nature of man, and because this appears to be a core belief of Elon, I doubt he will ever update his priors to make more effective investment decisions based on feedback from real-world tests. He will always view this as a project that is almost-ready with just a few technical glitches to overcome.
In which way did he lead the program technically?
NASA didn't stop innovating per se. In the 1980s and 1990s they worked out their own version of vertical take-off and landing and actually test flew it:
https://www.youtube.com/watch?v=wv9n9Casp1o
Linear aerospike engines are pretty neat too:
https://www.youtube.com/watch?v=FcW9kUUTfxY
Problem is absolutely none of this stuff ever got close to operational. It kept getting cancelled in the prototype stage.
If the DC-X had been developed into a fully reusable first stage, we'd have had something like Falcon 9 by the year 2000 and might be on Mars by now.
The difference between SpaceX and NASA is mainly that SpaceX cares about cost.
Or we can say that NASA has incentives to absurdly increase cost, like any bureaucracy.
There is no way the DC-X could have been developed without increasing enormously the budget (and taxes from Americans). That was the reason it was not done.
And innovation when you have many, many billions is one thing, if you have to operate in a commercial market but still find a way to innovate at that level its quite amazing.
I have issues with how Musk communicates, but fundamentally I think Tesla is innovating as much as SpaceX. Many here and me have lots of issues in how Self-Driving is developed, but fundamentally its the right concept.
What Tesla is doing with batteries is quite amazing. I have over the last 2 years spent a lot of time understanding the battery industry and what Tesla is doing is actually incredibly impressive. People mostly don't understand that because battery and battery production is far more obscure as topic.
However whatever else you can say about Elon, he is committed to the projects that he is committed too.
SpaceX will not give up on Starship, or at most replace it with a changed design that tries to solve the same problem. Tesla will not stop trying to push down the price of battery, they are comited to it and are willing to go all the way to vertically integrate mining if that what it takes. Tesla will not stop developing Self-Driving, they will push forward and invest as much as they need to invest.
I do wish the Communication around the Self-Driving technology would change. They should have just called it 'Co-Pilot' and sold a 'Advanced Co-Pilot' and say they are working on 'Self-Driving' but it not a product (yet). Stop promising it will come so soon, and I not sure about how I feel about letting Beta testers put up videos.
When it comes to automated driving though you're really requiring solving entirely novel never-before-solved problems. If there is a spectrum between "found a company on doing some existing engineering twice as good" and "found a company on solving the Goldbach conjecture" then driverless cars are a bit more towards the latter than hoverslamming rockets is.
Problems that are conceptually simple can still be very complex once all of the details are added. Theoretically landing a rocket is a simple matter of turning it into a giant model aircraft and doing the math on how much fuel you need to reserve. Re-lighting an engine is conceptually easy too. In practice we know this is a huge accomplishment.
Self driving is a problem that starts out hard. Think about how you get a computer to recognize other objects, especially other cars, based on it's fairly limited set of sensor inputs. Once you add in the details, like dealing with unexpected road conditions, detecting vehicles that are partially invisible to your sensors, etc... and it's hard to see a future where the technology is viable, at least in the near term. All computer vision stuff currently has a sizeable false positive/negative error rate that you just have to accept. But on the road a false negative or false positive can be fatal.
In theory, if the approach to self-driving that Tesla is pursuing in any given year actually worked... then the release would be about two years away. In reality it hasn't been working well enough, and every year a new plan is drawn up to reach full autonomy in 2 years.
This is also coincidentally slightly longer than the average tenure for an engineer/scientist, and as such the champions of a given strategy/approach will have departed the company before someone observes the strategy not panning out.
Exploratory AI should be thought of as "potentially kills your company if it doesn't work and you gamble on it working."
The ultimate truth is that you're outsourcing your thinking to ML researchers, much of the time. And as someone on the ground floor, let me tell you that we often don't have a clue how to achieve X any more than you do. We have ideas, testable theories, plans that may or may not work, and have a higher chance of success than random people. But it's still only a chance.
I don't think a lot of companies have fully internalized that. If your company is based around the premise of exploratory AI, you are literally gambling. And it's not the type of gambling you can win in the long run with. It's closer to "put it all on black" than "let's take a principled approach to poker."
There are lots of ML algorithms that definitely work but none of involve claims of intelligence. So it's not just "exploratory AI" that you're talking about. It's "any AI", anything claiming to be "artificially intelligent" and fully replace a human in a key decision making position (but not "any ML" since ML encompasses much more boring stuff).
I try to be an optimist, mostly because of how many counterexamples you see from history. Didn't some newspaper claim that human flight "may be invented in the next several thousand years" shortly before Kitty Hawk?
But for some reason, rich people keep coming to me with what they're trying to do. My advice is the same: you're going to lose your money unless you bet on proven techniques that exist today.
Take AI dungeon as the prime example. That is what I would call a grade-A idea. It was incredibly obvious that it would become a company (now Latitude) and that it would be successful, if someone was willing to put in the effort to make it so (which Nick Walton did). Once AI dungeon worked, and it had a "minimum viable working thing," the rest was a matter of execution.
But a lot of the ideas seem to fall into the category of... Well, for example, someone came to me saying they wanted to build a "virtual universe, filled with virtual people that you can talk to."
It sounds lovely on paper. But what are you selling, really? There has to be some specific value proposition. So strike one is that it's an unproven market. You yourself want a virtual universe. But is that virtual universe going to lead to something that will solve a lot of people's problems? And we haven't even begun to discuss how you're going to get there. What do you mean exactly by "virtual person"?
It's easy to pick on some of the outliers. But unfortunately, the problem runs much deeper. There are people who genuinely believe that AGI is within reach within our lifetimes, or perhaps within one generation. But whenever I try to corner them into giving specific details on how precisely to get there, the handwaving begins.
This is now a complete tangent, but, I found myself excited and enthusiastic to pursue AGI after a long conversation with a certain someone. They had "fresh eyes" -- a new way of viewing the situation, unlike anything that people were currently trying.
Unfortunately, after throwing myself into that mindset for several weeks, I had no choice but to conclude that their chance was closer to zero than 1%. And I was really trying hard to find that 1%, with all my mental effort (such as it is).
So what choice do we have but to let people pursue impossible dreams, and return to the work that we feel we can make an impact on? Live and let live. And of course, there's the likely outcome: our predictions will be incorrect, and we'll be talking with AGI in a virtual universe sooner than we think. But I wouldn't fall in love with their dream.
(I also think you were unfairly jumped on, and that you had a fine point, for what it's worth. Thanks for the prompt.)
Self-driving is absolutely the key problem, I'd say. I sympathize with you optimism. I'm optimistic about what "computers can do" but I'm pessimistic about DNNs + standard control becoming able to navigate the human-machine border.
Basically, I think a lot of problem are "AGI-complete", especially problem around human-computer interaction, more problems than people like to admit. And Remember, for things just in the NP-complete class, an "average" problem can be easy, it's the few examples that prove troublesome. It seems to me that AGI-complete problems are similar (and yes, I realize the term is a neolism, defined by only extension etc but I'd still say it's valid).
I also think you were unfairly jumped on, and that you had a fine point, for what it's worth. Thanks for the prompt.
Theoretically having lots of "karma" should make me not care about it. Theoretically...
Having recently become a parent, I have a newfound appreciation for how complex spatial navigation tasks are. Children learn to recognize faces and objects within the first 6 weeks to 4 months of their lives. They learn to navigate environments over the next 1-5 years language is fully understood for non-gibberish use cases over the next 5-20 years. It's not a fair comparison, but it provides for roughly the only yard stick we know of.
Not to further rain on the parade (without good reason) but I should mention...
"Recently we’ve learned that the server costs incurred for both the Griffin and Dragon models are higher than our total revenue. Nobody has ever deployed AI models this powerful before, and it requires extremely expensive hardware. If we don’t make changes to cover the costs of that hardware soon, AI Dungeon will not be financially viable."
I don't think there are companies with products based on AI where the AI has to work for the company. Google uses AI for search but search can screw up and search returns a lot of just indexed results. There's no "real world application" where AI works reliably (ie, gives a result that you can count on). That doesn't stop deep networks from an improvement on applications that were previous a combination of database queries. But this same only-relative-usefulness can be problematic when companies and institutions delegate AI to make decisions where it doesn't hurt them being wrong but it can mightly screw some random person (from credit to parole to whatever).
When confronted with these uncertainties internal stakeholder will often swing from "we just need more scientists working on this problem" to "it works fine, why would we spend time on this?" attitudes. The former almost always leads to over-investment where 3 teams of people are working on what should be one individuals project. The latter can sometimes be right, but I've also seen Fortune 500 Search rankings that have never been tuned let alone leverage an ML model.
This kind of marketing is a standard release of information in the form of opinion of the future, which shareholders can decide for themselves whether to believe it or not.
When the SEC slapped him down previously it was because he conducted a release of information about "facts" - not opinion - that were supposedly true now (that he had a private buyer at $x price). This was deemed a market manipulation because it was presented as 100% true. Whereas self-driving cars in 2 years might be true and it's up to individual shareholders to do their own analysis.
just to write it all out for clarity (not because I think you misunderstood): the might makes right that I was pointing at is an attitude of "winning is everything. winners are superior, no matter how arbitrary or unfair the game."
the might makes right that you're pointing at is the fact that in the struggle to define laws and norms, there are winners and losers.
I think it's just a question of values. there is no "neutrality" - there is always a fight between competing values, and everyone picks a side. the value that says "winning is everything, winners deserve everything, losers can die in a ditch" is a bad value to me.
Not sure what this phrase means to you exactly, but it may not be the worst idea-- the only way I see a mass rollout of automated vehicles any time soon at this point is on dedicated roads/lanes with roadside sensors to assist. Anything else is gambling with human lives.
The folks at Waymo/Google, some of the smartest people in the industry, admit to how hard of a problem this is and set realistic expectations. For instance, Waymo (and every SDC company except Tesla) say Level 5 autonomy is impossible and they are strictly targeting Level 4. But here is Musk who says "Level 5 will be coming next year" every year. I feel like there is a serious lack of humility at Tesla (or maybe it's just Musk).
Part of Tesla's business model is selling $10k FSD packages based on a promise that they will eventually deliver full self-driving. If, at any point in the future, any evidence surfaces showing Musk or other senior leadership at Tesla did not have confidence in their ability to actually deliver FSD, this would amount to large-scale fraud and may result in a class-action lawsuit.
On the other hand, if in all communication, internal and external, Tesla leadership projects confidence in their ability to deliver FSD, but then "unforeseeable circumstances" prevent them from actually delivering it, then it's not a scam, but just an unlucky turn of events. At the very worst, Telsa might have to partly reimburse their customers, but most likely they can keep the money.
So it does not really matter what Musk believes deep down. It's probably easier to do his job if he consciously makes himself believe that he can deliver.
Tesla recently said that "Full Self Driving" is not capable of autonomous driving. They tell the regulator one thing and tell their customers another:
https://www.thedrive.com/tech/39647/tesla-admits-current-ful...
https://www.caranddriver.com/news/a35785277/tesla-fsd-califo...
Ed Niedermeyer’s “Ludicrous” chronicles the story without the breathless media coverage that usually follows Musk. It’s incredible what the Musk empire has become, but there’s a lot more to it than meets the eye. https://www.amazon.com/Ludicrous-Unvarnished-Story-Tesla-Mot...
Marketers are usually overly happy, especially when in the act of estimating. This makes it hard for them to resist being overly generous with their predictions.
I do agree though that there’s so much that can be done right now without waiting for some magical future tech. Inner cities without cars can be solved with public transportation. De-urbanisation can be achieved with a combination of remote work, cashier-less shops and a well-run postal system. We don’t need to wait to do these things, they’re achievable now.
At 5:01 in the video it clearly has plenty of time to go and just doesn't move.
Also, far from the worst thing in that video, but I notice the car sits waiting to turn with its wheels turned into the oncoming lane. You're not supposed to do that, because if you get rear-ended, you're going into oncoming traffic; best to keep the wheel straight until you're ready to turn.
Lane shifting, totally wrong placement, everything moving in an arc.
That is a system not fit for purpose.
Then try making the left turn decision solely based on that screen. Good luck. Things don’t appear until they’re way closer than what is plainly obvious using vision.
Tesla should upload this video to their page where the misleading video was "driver would not have to be here"
Not saying other manufacturers are worse, but this borders fraudulent advertisement.
The beta is clearly designed to err on the side of caution. The majority of disengagement's are due to this behavior. You could actually argue that an overly cautious beta is safer than one which is 99% percent reliable because it keeps the driver from becoming complacent.
Until there are actual fsd beta accidents I say let them test their software, the upside of self driving for society is worth it.
You frequently have to enter the other land, cross a yellow line, or turn into a different lane because of double parked cars or construction.
Perhaps this does the opposite, it's so unpredictable that drivers much pay more attention than they normally would and therefore they don't crash :D
Us non-Tesla drivers have not signed up to be part of your volunteer risk-taking beta-testing team, go do this shit on private roads.
It's basically driving like a drunk guy.
What's striking is watching the system's screen vs the real world, and seeing Tesla not detecting important things. Tesla's big problem is simply that they are not very good at building a 3D model of the world around their vehicles. They try to do it on vision alone, and, after over five years, they still can't do that reliably.
I don't know what you're referring to. Any links?
Not only is FSD not worth $10k; it wouldn't be worth any amount of money if Tesla paid me to use it.
FSD is going to get somebody killed. Tesla should stick to what they're good at and quit hyping something they're very bad at.
The last two minutes were really shocking. The car would have crashed several times had the driver not been extremely vigilant. Tesla should not be testing this on the general public.
Unless the self driving saga has a different goal: transportation as a service. This is where the money are and this is where full auto driving is a necessity.
What's so difficult about making an app? You mean other than having to have some understanding of 20 years of tech its built on top of?
It could even project this onto the wind screen. Some new cars do a neat trick with perspective adjustment so the projection appears 'flat' even on a tilted wind screen.
Cars don't randomly appear and disappear in the middle of the open road. If their model cannot interpret these events as "a big fuckin' problem, I'm out of my operational envelope, I need to stop", then that model is indeed dangerous.
Sure, this is easier said than done.
Sure it might make travel by car suck less but not in any significant way that would change humanity in any meaningful way. At least not outside of a very narrow bubble we might be a part of.
It’s fairly obvious that the only thing people will achieve is killing themselves, their loved ones, or innocent bystanders.
And Tesla will bring us nowhere closer.
Anyways, hasn’t Tesla been far more successful as a Bitcoin marketing company in a week than as a manufacturer of cars for over a decade?
If you do want to risk your lives over advancing humanity why not risk it with Waymo which is actually making progress in this arena.
Heh:
>Because horseless carriages pose a risk to people who did not throw money at Ford.
We normally require a minimum age before a prospective driver may obtain a learner's permit to drive within a city, unlike this FSD, which in the linked video drove like a drunk driver.
Driving in downtown Oakland was incredibly complex with all sorts of unusual features, driving the course the car did in the video would have stressed me out a little, and I've been driving for 24 years. Full self driving does need to handle those situations to be trustworthy, but for highway driving and sedate suburban settings it's probably just fine.
Driving test takes 40min, through city (including all possible variations of parking) and highway on unpredictable route. I think less than half manage to pass it on first attempt.
If you passed, you get "new driving license" with a bunch of limitations (no passengers, supervision of experienced driver for 1 year, etc). If you get a violation of anything, license is taken and back to driving test.
After few years there is mandatory refresh course of (advanced) theory with exam. If you fail exam, back to driving school you go.
Edit: add into the mix, crazy, Mediterranean style driving on streets where exam takes place
The test must not screen people well enough if there are that many "crazy, Mediterranean style drivers" on the streets who already passed.
but, from my vague memories, when you sign up to theory lessons you need to bring note from doctor that you are healthy and have no history of mental disorders (serious stuff) and such
Driving tests are indeed way too easy, however, the level of driving seen in these Tesla videos would get any human driver a fail on the test. Just the constant indecisive jerking of the wheel would be a fail, let alone all the near crashes.
In europe, they are far more strict.
We are not the brightest, nor safest species.
Watching the live 3d view shown in these videos - it's a very erratic representation of the world around it and should give a pause to anyone that thinks this is "FSD".
Driving sucks and is dangerous as hell. Anything that 1) makes me need to drive less often, or 2) makes driving safer is a worthwhile financial & time investment for me.
I also noticed a few sudden stops from the way the guys lurched forwards at stops. Getting rear-ended due to pointless sudden stops is a fail as well.
Its not ready for use. From what I've just seen from that video I'd be de-rating that "driver" from full license back to probationary or more realistically, right back to learner.
I'm glad there was a human at the wheel.
garbage collection?
Why would anyone risk blowing that whistle when Musk and Tesla have tried to get employees murdered by the police by accusing them of being mass shooters[1]?
[1] https://www.bloomberg.com/news/features/2019-03-13/when-elon...
I also wonder how good the training examples are, if they're just using average drivers without any quality control. Like when I'm teaching someone to drive, I'm following traffic laws to the tee. But when I'm doing my day to day driving, it's mostly about optimizing time. There's probably a reason why Waymo hired and trained drivers to teach their algo in a standardized way.
Mountain road: https://www.youtube.com/watch?v=6nr9_wBPAHs
https://mobile.twitter.com/MSPFirstDist/status/1372152867612...
Innovation is the new Moloch.
This is an extreme example of "socialize losses, capitalize profits".
What on earth is with "double parked cars"? Is that an American thing? I've never seen it in my life. How can it be legal to just park in the middle of the road and block traffic?
The actual parking spots are full of residents' cars which haven't moved since the last street sweeping or snow ban.
To top it off, many/most streets are one way, meaning, cars will be "double-parked" in the left lane too. Fortunately our streets are 3-4 lanes wide so there's still room for actual traffic.
I used to try to bike on these streets (which are also full of potholes). I'm surprised I'm still alive.
Here's a great example on Boylston Street in 2016: https://goo.gl/maps/PQgtntVNYzgq98wcA Note the bicyclist up ahead who just had to navigate that. Spin around to see the Dunbar truck double-parked on the left side.
Same spot, 2019: https://goo.gl/maps/fb6k6i6HQUizpoKc8 This time the car's on the opposite side of the street. But spin around to see both a city bus and a huge SUV double-parked at the same spot.
(City buses often have to double-park either due to construction or just because some cars parked in the bus area.)
Of course when we are lucky to get bike lanes they are often treated as "double parking" lanes.
Tickets are handed out but they are treated as the cost of doing business. It's rare to see a delivery truck driving around without a ticket stashed under windshield wiper.
Having driven in streets like this most of my life is the reason I feel it is the ultimate hubris for Elon or anyone to think they can solve full self driving without also solving AGI.
Driving is often surprisingly social. I had many instances where I observed somebody pulling out of a driveway in front of me and something about how their head was turned or whatnot meant I subconsciously “knew” in a split second that they were about to go full speed without looking, so I had to slow down. Stop sign tie-breaking gets complicated. Pedestrian/bike rider eye contact is a thing. “Do I know that they know that I know what their intentions are?” And so on.
It’s not as complex as fully solving human communication, but it seems like a huge challenge when mixing human and automated driving on the same roads.
But I also don’t know if there were similar transition concerns when horse buggies were being replaced with cars and people had wrong expectations?
Maybe we should paint these cars in bright orange and have flashing lights on all the time so that people learn to expect the different behaviour. (The huge Lidar rig on top of some self driving cars works to that purpose)
At this point, many teams (Waymo, Tesla, Uber, etc.) have been working on this for years. And there must have been times when the engineers thought, this would work so much better if only... Maybe it's time to gather these thoughts and sort them by low effort/high reward. For example, if all cars had a cheap sticker indicating forward vs back, would that improve self-driving performance by a lot? It wouldn't be that hard to get car manufacturers to add a sticker somewhere. Or what about traffic lights? Maybe adding a cheap light or sensor to it would vastly improve self-driving performance. Or those road reflectors, can they get some minor changes to help the self-driving software?
It doesn't seem likely that level 5 or even level 4 will happen when self-driving is literally defined as replacing what a human driver would do. But what if the computer had some help from both infrastructure and other vehicles? Why not start thinking about cheap changes that can be implemented to fix the blind spots and flaws in the self-driving software? If these changes are cheap and easy to implement and would vastly improve the software performance, isn't that worth it for true self-driving to work?
It just seems like a problem that's multiple orders of magnitude easier to solve from a technical standpoint.
You can always decrease the odds of something bad happening with better road design (and better viability), better lane markings, better signs, etc. This would work for both FSD systems and humans. Yet, terribly unmaintained roads are the norm for a lot of people who are just “used to it.” If you create an FSD zone and outfit it with tons of sensors, why not just fix the road design problems that would benefit humans too?
Obviously it's not the same in every city in the world, but roads are pretty decently maintained where I live. And it's not like we're talking about high tech stuff, like other people said, it's just painting distinctive patterns on things to give sensors more information than they otherwise would have. We already paint distinctive markings for humans, it's not much of a big step further. Probably just some QR-code-looking stuff sprinkled on signs, lane markings and the front/back of other cars etc.
Honestly to me the hard bit seems to be getting everyone together and deciding on the standards, especially with how technophobic some cities/countries governments are. Painting a few markings once every few years to keep them fresh is a piece of cake.
Tesla's FSD scares the absolute shit out of me.
I've read that "intelligent" people are often easy marks for con artists, because they're often so overconfident about their judgements and opinions.
We don't need trains, just proper painting of the roads and separate autonomous vehicles from the rest.
Trying to use the current infrastructure without any adaptation seems to me a dead road.
Also, man, people in oakland seem to honk pretty quickly.
And they've been having streams of data for _years_ with autopilot. and yet they let this shit out. Look at how incorrectly they've placed the objects, traffic and other things. It has _no_ situational awareness, because it can't see 20% of the objects.
Considering that https://wayve.ai/ can drive around london unaided, tesla is utterly rubbish
Props to Tesla's engineering team, hopefully it'll be a bit better as years go by.
Edit: I guess I was wrong, the accidents are now so common we don’t care anymore. In the beginning they were hyped up a lot.
That said, I have a lot of faith in Papa Musk and his team. I think this nut will be cracked eventually, but its just going to take a lot of time to do so - far more than the timeline he initially proposed, as detailed in your excellent post earlier.
Elon said he was going to build a private space launch company. He did.
Elon said he was going to build an electric car company. He did.
Elon said he was going to build a $35,000 electric car for the masses. He did.
Elon said he was going to build a worldwide satellite constellation for Internet access. He is. Rapidly.
Everything Elon has said he was going to do, he has. He hasn't always done it at the speed at which he said he would, but he does always get it done.
He (meaning his team / company) will eventually develop a fully self-driving car. No, it may not occur at the pace at which he proclaimed, but I have confidence it'll occur.
His achievements are remarkable but that doesn't mean that he should be treated like an infallible demi-god. Yes he's done some really cool stuff but that doesn't mean he should not be the subject of scrutiny.
Hyperloop was an idea and nothing more. He never said he was personally committed to developing that, hence third-parties working on it.
Interplanetary travel? Starship is actively going through testing and looks like it should be ready for moon launches around 2023, which is in two more years.
I don't consider him infallible, or a demi-god, but its hard not to admire the sheer work ethic of this man. He's legitimately trying, and by all accounts of people who know him, he's actively working on all these projects and also making considerable stride towards these goals. These are not easily achieved tasks.
He promised Earth point to point travel, which would require rocket safety to improve by a factor of a million.
He has been promising ridiculous things from Neuralink but with literally nothing to show for it.
Look at this tweet: https://mobile.twitter.com/elonmusk/status/88805317515594957...
Does that really indicate no personal commitment? He touts that he's received government permission to build a Hyperloop, NY to DC in 29 mins.
We didn't get to the safety levels that airliners currently have by sitting down with books and, we got there by having the NTSB thoroughly analyze each incident.
Humans are terrible drivers, so each day we delay the transition to self-driving also costs lives, it's just that human traffic fatalities are less visible since society generally accepts that level of death.
It's not even his idea, Robert Goddard came up with it decades ago.
He also threw money into it, and then backed out, saying he was too busy.
And now he's tweeting about applying for and obtaining permission to build on the East Coast.
So, there's several issues there. Not to mention the more general issue - that it offers the same benefit as Maglev with massively inflated costs and far less comfortable passenger experience.
All of these are still in progress/development in various ways. I don't remember him ever saying they were complete (or even over with).
Thank uncle sam for putting in the effort to create the markets that Elon plays in.
The Federal Government has consistently funded some of our most important projects. The Human Genome Project, for instance.
He's not unlike Steve Jobs. They both possessed enough knowledge and technical acumen to understand what they're being told about disparate subjects, but they may not have the in-depth knowledge that a Jim Keller would. And despite my enormous respect for Keller, he's not an Elon Musk or a Steve Jobs. Jim's a craftsman at heart, and that comes through in interviews with him. Musk and Jobs are true visionaries.
This is not genius-level but is the blending of two skills, a charismatic Administator, who can temporarily suspend disbelief, and a Fundraiser. Each of those takes a ton of work. I'm not minimizing Musk's dilligence in any way, but there are lots of people who can do those activities at high levels. The difference is that most people can't survive their tweeting SEC violations while stoned.
True, as long as this excludes everything he still says he is going to do but hasn't yet done. But of course, he will.
So if he ever doesn’t accomplish something we’ll never know because maybe it just hasn’t happened yet.
He also said they would have a solar roof as cheap as a normal roof and that the new roadster would hover with rocket engines. Let’s see if that happens.
That is true for absolutely everyone in the world.
Tesla will never achieve safe, certified level 5 autonomous self driving with the current generation of vehicles, without either a fuckton more CPU power or better sensors(as in some proper high res depth sensors).
On this I will bet. Come back at me in 5 years. He'll either have to bite his pride and put some sort of re-branded lidar on, or some decent stereo camera rigs.
When Tesla is 2x better, we'd be reckless not to use it immediately.
So depends on your measure. If you mean "on average" then yes. But it seems Tesla has a long way to go in other safety measures first.
If you watch the videos in the thread, the answer is it is far worse than the worst human driver you've ever seen.
Anyway, "accidents" are not a good measure of safety for FSD Beta because just in this video it would've caused about 4-5 accidents and all of them were averted only because the driver intervened.
To the extent that people who were using the feature on their way to work are working from home, that's a lot less miles driven and less potential collisions.
To the extend that many people are working from home, that's a lot less traffic on the road to interact with, and drivers may be more attentive because of the novelty.
Statistics would be nice, of course. Tesla doesn't report on total miles driven.
Here's a recent Tesla Autopilot into parked emergency vehicle report though. https://www.cnbc.com/2021/03/17/tesla-in-autopilot-hits-poli...
As soon as it's going to be legal somewhere to remove your hands from the wheel, there will be incidents for sure. And then there will be responses to those incidents. At some point, that drops to acceptable levels and more will follow. Tesla will simply be able to pull out the numbers and say: look we're doing it here and it's fine. Solving the chicken egg problem of having the data to say that is key to ever getting to that point.
Tesla's strategy is not to lock itself up in an ivory tower for a few decades and to then release the perfect system. Companies following that strategy will fail. Instead, they are releasing imperfect software now and they are improving it rapidly and incrementally. As long as they have hands on the steering wheel (level 2) it does not actually have to be perfect. It will give them a steady stream of issues to fix. Lots of real world video footage to analyze. And a track record of slowly improving safety relative to human drivers in terms of numbers of incidents per million miles. Rolling this out to millions of drivers is going to get them a lot of data to work with very quickly. Data is the key thing to have in this space. Without data you can't make progress. Tesla is setting themselves up to get a lot of that. The reason they are getting cocky is that they already have good metrics.
It would be interesting to repeat the same journey with the same car in about a year. I bet it will still have issues but probably less of them. It's going to take more than two years for sure. But ultimately leveling up to 3 and 4 is going to have to be evidence based. And Tesla is building that case by gathering data from real drivers intervening when the software messes up and fixing things as that happens. That's something you can measure. The better that metric gets, the closer they get to their goal. Who else is even close to doing that? I think some of the Chinese manufacturers have shamelessly copied Tesla's strategy. And in some cases the actual software apparently. Regardless, China is the market to watch when it comes to full autonomy. A bit of wild west but they are moving fast there. IMHO if Tesla gets there, those manufacturers won't be very far behind. Unlike some traditional manufacturers who seem to be a bit hands off (bad pun) when it comes to moving forward on this front.
Don't forget about that Tesla on autopilot which crashed into a parked police car __two days ago__ in Michigan...
Full self driving doesn't mean perfect driving, it will always be continuous improvement.
The standard should be: "Safer than human driving" shouldn't it?
And when you realize that humans are already terrible drivers it doesn't seem so reckless to hold AI driving to a realistic, achievable standard instead of an ideology.
Another way to look at it: imagine two different human drivers, one is clearly "better" and faster than the other, but wrecks occasionally, but the other is a "bad" driver, but never does.
Which is a more desirable model for an AI to emulate?
The only bar to clear, and I mean the only bar is, "Do autonomous vehicles kill fewer human beings than humans on the road do?"
That's it. When it decreases by even 1 human being below what humans cause, its time to switch over, because like every single mechanical, electronic, and computerized tool, it will only continue to reduce the number of human deaths as we progress with its development.
I get that autonomous vehicle deaths freak people out because they do things a human would never do. But? So what? If 53,000 people die in automobile accidents where they drive into each other because they stop paying attention to the road to reach back and slap their 10 year old who won't leave his sister alone, it makes absolutely zero difference than if the car drives into the ocean and drowns its driver. Same result. Dead person. We only think it matters because we like to think we have control over our lives. You don't and you never will. You could have a massive aneurism the next time you're driving and slam your vehicle right into an oak tree. The Universe is a random uncaring system.
Your life is just a series of mostly random events that you shape into a coherent story because accepting that shit just happens and you have to deal with it is, like, a real bummer, man.
Not even that, I would totally accept if autonomous vehicles kill only slightly more than human on the road do.
Presumably the risk of an accident is relatively evenly distributed between all AI drivers (they are using the same AI after all). But is the risk of a car accident evenly distributed between all people? Not even close. It’s perfectly possible to simultaneously reduce the overall risk for everyone while at the same time increasing the risk for a given individual by an order of magnitude.
Would you be willing to assume a greater risk of accidental death _personally_ to decrease overall risk of death? Not a question I imagine reaches broad consensus...
And what about the soft problems? Like responsibility. A self-driving car runs off the road and kills your daughter. Now what? Tesla is certainly not going to accept responsibility. So... you just “chalk it up” as bad luck? At least the current paradigm has the _ability_ to offer closure after a tragedy.
Reducing “self-driving cars” to a single metric is not only mathematically dubious, it’s ethically abhorrent and just plain stupid. I expect better.
Come on folks... Elon has been claiming for years and years that you'll be able to read a book, or take a nap while in FSD mode, and FSD is "just a year away...". He also claimed all the hardware existed in current-model vehicles and would just be "unlocked" later. The lies pile up, yet people clamor over each other to defend Tesla and Musk.
Here's an article from 2016[1] where Musk makes exactly that claim.
From the video embedded in the article, I'd wager we're a long ways away from any Tesla having anything remotely similar to FSD... yet Musk still claims Next Year™[2]
I'm at a loss why Tesla even decided to wade into the FSD waters. What was wrong with making a really good EV that people wanted to buy? FSD seems to have become a major distraction for Tesla... now plagued with build quality issues and FSD issues.
[1] https://evannex.com/blogs/news/elon-musk-self-driving-hardwa...
[2] https://www.cnet.com/roadshow/news/elon-musk-full-self-drivi...
Musk is a deceiver, he is not accidentally optimistic with his "forecasts".
I will believe fsd or autopilot works properly when musk and his kids do a coast to coast trip in winter from NYC, via Chicago driving through every city at rush hour. Blindfolded with no driver intervention. Then it would deserve a fsd label. There should be no room for double speak on this matter.
People say that humans are bad drivers as well. True, if distracted, tired, DUI, inexperienced. Still, no human with sound eyes and a drivers licence would drive like in that video. Unless on the run from police maybe.
None of us are perfect, we’ve all driven drunk. How many times you got caught? It’s the fact people don’t get caught or crashed before that they risk it.
So really these vehicles are driving worse than humans at their worst. That’s probably why we are underestimating the problem.
No we haven't. Some people are responsible drivers and don't drive after drinking alcohol. Others don't drink at all. I've driven while tired, but never drunk.
Apart from that, I agree with you.
So there’s something going on in the brain that we’re not getting.
The distraction issue could be minimized as well, albeit with rather invasive methods.
All the above plus taxi apps becoming abundant and the common sense to assess when to use a taxi and the fsd software will take a long time to be better than humans.
So far, summon does not even work and i remember driving in a Mercedes with lane assist and automatic braking if the distance to vehicle ahead is too small in 2010 or something. Did Tesla even exist at that time or where they at the stage where they used Lotus cars and replaced the drive train and the stickers? The features are not much better than what I have seen in a 2010 Mercer.
The full part is where it also include: following line, navigate, parking, line change, stop and go, recognize traffic sign, etc
Its not FSD because its beta of the software that will enable that. That's the point, your training the network.
The hardware does not seem to be the constraint here. CommaAI does a great job on a processor much, much slower. If their prediction is wrong, they put their own money on the line to upgrade the hardware if need be.
I don't think they were lies, Elon usually just says what he believes, often he is right but optimistic, sometimes he is wrong. Clearly he was most wrong about AI but he is right often enough and always speaks his mind.
This is what basically everybody that interacts with Elon says, the guy is candid and says what he believes at that moment. This has been reported by people that worked with him and by people like journalist who have interviewed him (recently just in Liftoff book).
Tesla has to put most of the money made from the FSD sale into a holding account, as they have not delivered the feature.
Generally more generally, I don't think its smart marketing or communication that they are doing.
Musk is "the first to say" a lot of things.
If I were participating in a beta test and my phone kept crashing, I could complain about that, but not much, right? beta software does that.
Of course a car crashing is a much bigger deal, so, like: pay attention.
I'm pessimistic about it ever getting out of beta, personally. But it is what it says on the label. I've seen other, equally cherry-picked videos, where the FSD system has done a competent job of getting from A to B with zero driver intervention.
> I've seen other, equally cherry-picked videos
Cherry picked? Did you watch the video? It's a complete drive through Oakland - not cherry picked snippets.
I've seen video which is also unedited, and which shows Tesla FSD behaving just fine. It was promoted onto my Twitter feed by people who want to believe FSD is going well.
Whereas this one was promoted to the top of HN by people who want to sell clicks promoting the idea that FSD is going poorly.
Either one is cherry picking. What wouldn't be cherry picking is watching as much video of FSD driving as you can, collecting statistics on it, trying to make sure your samples are representative (good luck if you aren't Tesla), and writing up a report on what you find.
This is, of course, a ton of work.
The guy that made the video and posted it to youtube is a Tesla "fanboy". I doubt his goal was to paint Tesla's FSD software in a bad light.
Yet, in one video, the Tesla nearly hits pedestrians, nearly side swipes other vehicles, drives in the wrong lane, comes to a complete stop in the middle of a busy road, and tried to drive straight through a iron fence. That doesn't look very good to me...
No, sue Musk & Tesla for fraud. & false advertisement. Unless they are willing to put their responsibility on the line, it will remain a piece of garbage.
1. Success isn't perfect driving. It's when it drives statistically better than most people. Humans themselves are often terrible drivers ourselves, so why would we hold a robot to a standard we can't even achieve?
2. Ford's customers were right all along: What we want is faster horses.
I'm not asserting it even, just spitballing because this is fascinating to me.
The counter-intuitive advice people are given for things like an animal suddenly leaping into the road is to just hit it, because slamming on the breaks can and has lead to the car's occupants dying for the sake of a squirrel.
What do we expect a self driving car to do in that situation? Just imagine the outrage and headlines if a Tesla does what humans are told to do, and kills a dog or cat!
That said, FSD is driving so bad that I would demand an actual driver that drives this way get their license revoked.
A lawyer would say Tesla, because $, but at least at this point I would still claim the driver is. Especially with 'beta' in the software title, where they're explicitly told to pay attention and take control.
I don't envy car manufactures working on this. They're facing exactly the same battle that Ford did when the 'horseless carriage' was becoming popular. We knew then that humans are honestly speaking, terrible drivers who frequently get distracted, or lose control of the vehicle.
Just look at the newspaper articles from those days: https://pessimists.co/horseless-carriage-archive/
Stanford banned them!
I expect self-driving to follow exactly the same course that horseless carriages did, to have heavy early resistance and lots of bad incidents, then acceptance, then be utterly commonplace. With the same track record of decreasing accidents and increasingly safety measures over time and eventually cease to even be a controversy.
And to clarify again, I think Full Self Driving and Autopilot are bad names for driver assist that does not fully self drive or autopilot in the sense that the average layperson would think those terms mean. I dont know if Tesla will ever get to full self driving or autopilot in that respect. I also think that the driver assist as it exists today will prevent entire categories of accidents even if it causes another class of accidents. I think the net difference will be that we are safer on the road because of it. While we are so quick to call out Tesla for the quality of autopilot we forget that at least 50% of the people on the road are distracted and have no such safety net. At least once per drive I sit behind someone who decided that the red light was the perfect time to catch up on their digital life and misses the green light. I see people weaving like a drunk driver and going 20 mph under the speed limit so they can have a buffer zone in front of them while they phone and drive. We need a solution to this as people arent going to stay off their phones...
There is a big difference between "bad marketing" and shipping wannabe self-driving code.
Now they've released a "full self driving" beta that can not actually fully self drive without killing its passengers, while Waymo has been operating a publicly accessible, fully autonomous taxi service for years now. Last I checked they don't even have safety drivers anymore.
And somehow, there are still people who seem to believe that Tesla is pushing the envelope.
I don't even own a Tesla (I do have an Audi E-tron though) or own Tesla stock directly so it's not like I'm a fanboy or anything. I'm just tired of the total failure of journalism in this country.
The video is genuine and real but it shows a completely different experience from pretty much almost every other drive-through FSD video I've seen (which have overall been mostly flawless, with a couple other ones sprinkled in that have also been pretty bad at times).
It's important to report on outliers and worst-case scenarios though, especially when the outcome could be death. Obviously this beta has a lot more work to be done before it can graduate into widespread use.
I watched the video. If I had to pick an uncharitable description for just how bad it drives, and how jerky and unpredictable it is, I would say that it drives like a drunk 17 year old.
ARK Invest's success and openness made people realize that they can put their money in a disruptive company if they are able to handle the risk/volatility. There are amazing channels that go into the technical details (Limiting Factor is an examlple for Tesla's battery tech).
Shooting at the messenger fallacy.