Rivian's gambit for full autonomy
spectrum.ieee.org
spectrum.ieee.org
At first I was skeptical, but reading the article, they do track object across video frames which for me it's the minimum a system like this should do. All previous FSD systems are just Fake self-driving.
"That proprietary AI driver identifies features in images and point clouds, groups them into objects, and tracks them across frames, time-stamped to the millisecond to account for differing frame rates. The AI thus builds confidence over time, acting on object detections that persist across several frames, rather than, say, slamming the brakes due to a camera blip on a single frame."
Ironically, I'm sure most humans have done the eyeball equivalent a handful of times, despite our knowledge of object permeance.
Nice in theory but in practice they still need a ton of training data. The bitter lesson is very bitter. Tesla's end to end FSD uses occupancy networks and brake stabbing still hasn't been fully solved.
I'm pretty sure Waymo does this. It's hard to imagine not using this information, it's basically 50% of what I do when driving (e.g. that kid that was running on the sidewalk suddenly disappeared from view in an unexpected way; I wonder if he's about to cut across the street, etc).
Oups a ball crossed the street 15m ahead. Let's slowdown to a crawl and watch for the kid is what a human would do.
I suppose they may have special cased the ball case.
What Rivian is going to release at the end of 2026 is what Tesla had 3 years ago. All that time has gone to dealing with edge conditions: bike lanes, emergency vehicles, construction zones, and all the crazy one-off situations that their fleet has seen.
Tesla's fleet is an order of magnitude larger, so events that Rivian has seen only once, Tesla already has ten examples of. And rare events that Tesla has trained on, Rivian has never yet seen.
I will say this, though: Rivian is the closest competitor to Tesla (not counting Chinese cars). Waymo FSD is obviously more advanced than Tesla, but they will lose out to both Tesla and Rivian when it comes to manufacturing. And the robotaxi game is going to be all about manufacturing.
That depends on manufacturing (e.g., low unit costs, and high throughput).
Are you sure about that? Google is probably one Trump bribe away from being tariff exempt for these.
Maybe I misunderstand what you mean but the car itself (chassis, bodywork, drivetrain) is a commodity today. If you can turn any car into an autonomous car then the car isn't worth that much in the self-driving game. Tesla's stock is propped by the promise of becoming the first to achieve autonomy not by its ability to build cars, or else VW and Toyota would be worth multiples of Tesla. They aren't because cars are a commodity. Autonomous car brains are the hot deal.
If manufacturing ever became Google's Achilles heel, they can take a majority stake in something like VinFast and manufacture a Google car.
It's like saying Linux will lose out to Microsoft when it comes to manufacturing. Yes, they do lose out by not manufacturing anything and yet their product is slowly taking over the market. Local loss (no profit from manufacturing), global win (core product is increasing market share).
They won't even have to (partially) buy any manufacturer - they can utilize contract manufacturers like Magna Steyr (which made the Jaguar I-Pace) or Foxconn, or lean on/expand partnerships like the one they have with Toyota.
I agree that Waymo doesn't need to ramp up manufacturing themselves until they really start hitting walls with vehicle makers, or want tighter integration with the vehicle.
They also spend $1 billion a year on R&D. while they are probably at least break even on opex, they are continuing to invest.
There is no shortcut. The reason they have all of those sensors, and the tiny windshield wipers that keep them clean, is that they need them to make it work. They are on version 6 of their hardware. If it were possible to remove some of the sensors and retain the same level of safety, they would.
I still find it insane that the entire established automotive industry saw what Tesla was doing with cameras and incrementally working toward FSD, and still to this day has not even started to collect data (or even install enough instruments to collect the data!) to get onto the same trajectory. Meanwhile the Model Y is the best selling car in the world multiple years in a row.
I'd rather these guys just focus on building a Euro spec model of the R3 and getting it over here.
Have you ever tried reading a book while being driven around? You get car sick really fast, because your vestibular system does not like constantly getting jostled around with mismatching visual inputs. Eating dinner while being driven around is a similar story: anyone around you does something unexpected, and your dinner ends up being spilled all over the car.
Self-driving cars will never have the same smoothness as planes or trains, simply because they have to share the roads with the general public. You might be able to sleep, but your commute isn't going to neatly match up with your sleeping schedule, so that's only really useful for longer road trips.
> this halfway autonomy where you have to watch everything is kind of useless
No, it is actively dangerous. Humans are extremely bad at staying fully-focused and being ready to act in the blink of an eye, while at the same time having literally nothing to do 99.99% of the time. If you don't let your attention wander, it is significantly less exhaustive to drive yourself. Halfway-autonomous cars exist primarily so that car manufacturers can demand extra money without having to accept any of the reliability.
I agree with the proposition that people are horrible at halfway autonomy.
But I am horrible at driving, and I kind of think almost everybody is. I have a car that just does lane keeping and distance keeping cruise control. The number of times it is notified me that I’m sliding off the side of the road is large. Similarly it’s let off the gas a vast number of times when I was creeping up on the car in front of me and I didn’t notice.
Maybe it’s having just one long, largely empty commute for five years. But I suspect it’s that you can drive any modern car without 100% concentration at which point people are prone to getting distracted.
So I’m thinking the comparison should be halfway autonomy vs a modern “dumb” car and I think halfway autonomy is safer overall.
I don't suffer from this all, and judging by watching people in public transport I'm not alone.
When would I want autonomous driving? For longer trips. Where do those happen? On the motorway where driving is mostly straight and simple.
When on the motorway, my Tesla's lane assist gives me basically all the benefits on the motorway I'd get from full self driving. FSD doesn't let me stop watching traffic or release the wheel, and I'd still have to take over to adjust to traffic conditions.. i.e. the same exact thing I'm doing with lane assist.
I'm not interested until I can literally sit in the back and take a nap.
well, perfectly is a bit much, it doesn't know how to dodge potholes yet and it sometimes misses exits/turns, but it's never done anything i would call dangerous
And since I can't look at my phone (or sleep or read) I just sit there and think. It's surprisingly relaxing and I didn't realize how much I would appreciate it.
Up until about a decade ago, I insisted on getting cars with a manual transmission as I wanted more control over my car. I definitely don't think that way anymore with all the advances in transmission technologies...
I took a fully-autonomyous Cybercab to a coffee shop yesterday; it doesn't even have a steering wheel. It was just a short drive, but it handled various complications (a garbage truck, double-parked cars, construction, unpainted speed bumps) with aplomb. The driving felt extremely smooth and human-like (better, in my experience, than Waymo, which can still be a bit jerky).
Those operate in constrained areas of a few cities though - with a control room, that's a long way from telling your car to drive across state unattended.
(In Austin it covers what looks to me like the whole city but not the surrounding smaller towns.)
I'm not sure how you would tell, but that is a critical question for scaling and true autonomy.
The obvious level is the monitor just takes over driving like a remote FPV driver/pilot.
But I wonder if the software can also flag alerts early when it starts to identify ambiguity, and the monitor can see the camera+lidar outputs and make a guidance-level input like [get in the left lane here] or [pause/go], and the direct driver inputs (steering/brake/accelerator) are never interrupted, but they have better "tactical" guidance from the central office?
If and when FSD systems become a reality and are safer than human drivers, I would expect that regulators will require them on all cars, just as they did reversing cameras, airbags, automated emergency braking, and other once-optional safety features.
That assumes that suppliers are available to affordably add it.
So far Tesla is the only company remotely close to unsupervised self-driving that doesn't add $1000s to the BoM cost of the car.
What makes you think it doesn't? All those cameras, the expensive chips and cooling for them, and redundancy for all the systems don't come free.
(FWIW, the business story for Waymo has never been about its hardware making its way to consumers. It's always been about whether Waymo continues to run its own taxi service or leases out hardware and operations to other taxi businesses).
It's maybe $1.5K of BoM total for the chip, memory, and cameras.
Where are you seeing this? The estimates I've seen all land around $1K.
The 5 millimeter-wave radars and 12 ultrasonic sensors are $100-150. LiDAR is $150-200. The 12 cameras, control unit, and 300 TOPs of computing is $150-200. This computing is comparable to what Tesla puts on their consumer cars, and the sensors suite is much more extensive. (The current robotaxis have about the same computing power as the consumer cars but a lot more RAM).
It's very obvious to me that FSD is a reality already, usually from parking spot to parking spot. And it at least feels much safer than I am driving, especially since I can get tired, etc
But still, even v12 can handle things like manual traffic signalers, construction zones, etc. The first time my car followed instructions of a cop waving us through a stop sign, I was amazed.
Which would then offset these monthly subscriptions by car companies.
It presents interesting questions, though. Right now, insurance companies are incentivized to fight for their customer in court against the other driver because they don't want to pay out.
But if the FSD provider is also the insurer, they aren't a neutral party if the dispute is between whether FSD is to blame or if the driver is to blame. It could become adversarial, let's say if the data in the car shows the driver wasn't looking at the road at the moment of the crash and therefore wasn't ready to take over for self-driving.
There might be ways around this, but it's a challenge to overcome. I'd want my insurance company to protect me against my FSD provider, too.
(My point is complicated by the fact that liability coverage is separate, admittedly. My insurer might not want to pay out if I was driving drunk, for example.)
I expect that by the mid 2030s this will be widely used across most car markets in both L2 driver assist form and L4 style autonomous taxis active in most large cities on all continents. L4/L5 level driving for privately owned cars might trail that a bit but I think the technical arguments against that are rapidly evaporating as well. The arguments are increasingly about regulation, insurance, liability, etc. I think the Chinese are a bit more pragmatic on this front and more likely to just get this done domestically by the early 2030s and will be deploying this globally by the mid 2030s. The only question real question is whether the US and Europe will do that before or after them and whether that will be with their own tech or somebody else's.
Rivian seems to have all the right tech and strategy.
That's brave. What about all the edge cases?
- Driving through an empty prairie
- Driving slowly through a crowded square, giving people enough time to jump out of the way
If you choose between hitting a pedestrian and swerving potentially injuring the driver and totaling the car, what would you like the car to pick? Or any given trolley problem variation, etc
As of August Tesla had 80+ unsupervised taxis in operation with more than 380k miles driven total, and they are required to report incidents to the NHTSA. At the rate at which they are adding miles, if they are not now safer than human drivers, we should know soon.
What I want is simple: accountability. If a human hits a human, we throw the book at them in the case of a modality of negligence. If a robot screws up, how do we remedy and discourage?
Anthropic has paid out an awful lot of money settling piracy suits.
An example where no one attemped to hold anyone liable isn't a counterexample to what happens when we do.
The rights holders that got a settlement out of Anthropic spent two years and probably a fortune fighting in court. If that’s “easy”, then sure, this is a solved problem and there’s no point discussing it.
> An example where no one attemped to hold anyone liable isn't a counterexample to what happens when we do.
Yes it is (specifically, a counter example to it being easy, not that it’s possible). The whole world saw HuggingFaces get hacked and the collective legal response was crickets. The same world that’s had its legal system throw the DMCA and unauthorised access to computers legal book at researchers, games preservationists and random people trying to access media that they can’t licence in their region.
I can't find evidence of it working in the US. Gets hard to pay fall guys when all company assets have been seized in court judgements.
Realistically no, unless he was hilariously reckless.
> SMITH: There’s the case that happened with the little boy on the Upper West Side, Cooper Stock. He and his dad were crossing the street. And a driver was making a turn, and he just ran over the little boy, didn’t see him.
> SMITH: So right now all that is is a summons to the driver for failing to yield. But it does not rise to the level of any kind of manslaughter or homicide charge. There was a study that showed that between 2008 and 2012 there were something like almost 1,300 fatal crashes in New York, and there were like 66 drivers arrested.
>Smith says that New York has some of the narrowest standards for conviction in the country. It’s called the “rule of two” — you need two significant violations of traffic laws in order to bring a charge, including some incredibly reckless or criminally negligent act. Otherwise, it’s just … an accident.
https://freakonomics.com/podcast/the-perfect-crime-2/
Maybe it's different in europe, but I doubt that'd be the case either unless there were several aggravating factors.
> Our neighbors have different vehicular laws than we do. Both Massachusetts and Connecticut have vehicular manslaughter statutes that punish traffic fatalities or serious injury that occurs because of simple negligence. New Jersey doesn’t have the same statute as Massachusetts or Connecticut, but even they have vehicular manslaughter statutes that encompass more behavior than what New York has, which is absolutely nothing other than drunk driving. So around the country — in Iowa, Louisiana, Georgia, Nevada, Kansas, California — all over the country, there are states with vehicular statutes that punish a failure to yield as a traffic fatality, that punish the driver.
So this seems to be a peculiarity of New York, I guess. Also, this is from 12 years ago on a podcast that has been widely criticized for inaccuracy and cherry-picking. I'd be curious if this was even true at the time, and if so, if it's still true or if the laws have changed.
Obviously local laws differ and I'm sure there are many exceptions, but it is very common to have your license suspended if you run over someone with your car.
And you had that one example to prove it. That settles it.
I know you want to make your point but stacking up fallacies (same in your "Timmy" comment) isn't helping [1]. Most humans grow up having personal responsibility, then at adult age the legal system puts even more on them. A lot more, on them personally, under threat of deprivation of freedom or monetary loss from their own pockets, or in the case of driving even something as small as losing the license. There are exceptions ("daddy pays" sort of thing).
Now contrast that with corporations, especially large ones, where almost every bit of responsibility is diffused behind a corporate veil that allows every single human to sidestep any personal responsibility whatsoever. Some shareholders' money take the plunge. And starting recently you can offload even more responsibility to the computer/AI.
I might lose my license if I cross a red light. Will a company lose the license to operate all autonomous cars when one crosses a red light?
[1]
> My anti-terrorism AI drone escaped the sandbox and shot up gruez's house. The drone was punished by removing its livery. What else can be done? What next, should we punish soldiers fighting terrorists too?"
> I tested my anti-cancer medication in the drinking water of the neighborhood. gruez got hit the hardest by the side effects, God rest his soul. But what can we do, ban cancer research?
To be fair, actual self-driving car business units have suffered similar fates. Uber's autonomy team was shut down after they killed Elaine Herzberg, and GM's Cruise was functionally destroyed after they contributed to a major bodily injury crash and tried to cover up their errors.
It's the level 2 consumer car companies that have mostly skated under a liability shield. Tesla FSD has been shit for a long time and paid very little price for it.
I do want safer vehicles and I do see a path to get there with this technology.
What I don't want is companies training their technologies with legitimate risk to the public, with a clear pathway to profiting off of this risk and public data, with no benefit to the public _unless_ they happen to succeed, and to some degree, unless you can afford their products. Something feels off about that.
Of course, everyone benefits if vehicles are safer, but it does feel like there is an extractive factor regardless. These companies benefit from public resources, are wholly enabled by public resources, but are trying to create a situation in which the technologies they create are more or less required to be used in order to utilize the public resource. Does that make sense? I suppose it's inferred, but, I don't think it's unreasonable to believe any car manufacturer with a working FSD technology would LOVE to have that technology mandated to be on every new vehicle. Imagine leasing that to every manufacturer who didn't develop it themselves so they could continue to sell in the USA.
On one hand, safer vehicles are a MUST because we allow an absurd number of people to die due to unsafe driving every day. On the other hand, I don't feel right about private companies benefiting so much from roads the public built, while training their technology with some degree of risk imposed on the public, and with no clear desire in these companies to give back. They tend to pay paltry sums in taxes, if any. Rivian was at a net loss of over 4 billion for 2024. I understand they're trying to build something that requires massive capital, but, I also understand that these companies are experts at operating at a loss for prolonged periods of time. Amazon is a stellar example.
This definitely cuts both ways, but I'm not sure exactly where I stand. I need more information to decide, but I'm confident that there is some degree of exploitation of public data and resources here. I'm not trying to imply Tesla, Waymo, or Rivian are the only ones guilty of this either. The tragedy of the commons is a well-known phrase because it's such a typical phenomenon. I'm also not suggesting that there's a simple solution. There can't be because the fragmentation of the polities in which these companies operate and train their technology within.
First, our current city driving environments were designed for being used by humans, so take into account human strengths and weaknesses.
Second, that kid has spent 16 years in such an environment learning how humans function there. He's got a very good model of how things work.
By the time I was 16 year old Timmy I'd spent a whole lot of time walking next to roads with cars, biking next to or in those roads with cars, and riding in cars on those roads. What was new when I was the driver was mostly learning the physics of cars and driving.
(I suppose nowadays, with the way parents can get in trouble letting kids out of their sight, we might have to worry about a generation of 16 year old Timmys who have not spent any significant time walking next to or biking next to or in roads with cars).
Third, that kid will have a pretty good model of predicting future behavior of other drivers, pedestrians, and cyclists based on figuring out their intent, which can even let them make some pretty good predictions about them even when they are hidden or masked from the kid's sensors.
Fourth, Timmy has a bunch of knowledge learned from listening to other people, reading, movies, etc., that can be applied while driving. For example suppose they live somewhere where there are many cows, but no deer. If they see a cow near the road at a place with no fence, they will have an appropriate model from experience that tells them cows are generally slow moving. Be cautious when passing, maybe giving it some extra space.
If the see a deer even though they have no experience with them they likely know that deer often bolt rapidly into the road right when a car gets close due to hearing stories from people who live in deer heavy regions or from news, books, or TV/movies, so go slow and be very cautious passing that.
Autonomous cars have a big advantage in reaction time, so if a bad situation starts to unfold they are likely better at saving the day than Timmy (or an experienced human driver) would be, but at least the systems so far deployed so far don't seem to be up to Timmy's understanding of other actors and their intent. They also seem unlikely currently to have that widespread non-driving knowledge that can be called upon to understand novel situation involving things that was not in their driver training.
We never blame kids playing on publicly-subsidized subway infrastructure.
edit: https://www.aap.org/en/advocacy/community-health-and-advocac...
How dare you call my kids pedestrian. They're better than that.
A situation which would be in obvious conflict (negative expected value) with the goal of saving the man hours and lives currently being cost by manual driving.
Current trend nowadays is watching tiktok, though thankfully I've only seen it at red lights, but that might just be because I'm not staring at other drivers when driving around at 40mph.
"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."
I don't own a lot of things, but they all look nice.
Until then, I enjoy the tech story.
It's a fairly generic body shape, but the headlamp and grille shape is distinctively "happy" instead of "aggressive" (this is a real thing, and it's intentionally designed for in lamp/grille shapes on cars)
Personally, I don't want "aggressive" near my cars. They kill enough people already, and I'm tired by how common it is for manufacturers to push aggressive body shapes.
I had a coworker who sold his small lexus, because it made dating weird.
Why would someone with no ego care about looking foolish?