Tesla releases FSD Beta v12 with AI vehicle controls | Hacker News Reader

Tesla releases FSD Beta v12 with AI vehicle controls

electrek.co

28 pointsrntn40 comments

> The biggest difference with previous FSD updates is that the vehicle’s controls would now be handled by neural nets rather than being coded by programmers.

Does anyone have details on what this means? To me this sounds like neural nets are now part of the low level control loops for the throttle/steering as well - but that doesn't seem particularly beneficial. Especially if you're interested in control stability/predictability.

Yeah - Tesla software has two parts, a vision system and a driving system. The vision system turns the camera image into a description of what is in the image (road, cars, people, etc), and the driving system actually controls the car based on the vision input and route it has been given.

Up until now the driving part had been hand coded in C++, but now it is a neural net that has learnt what would a human do in similar road situation (not sure what they trained it on). Is this an improvement? Who knows - should certainly be different. This doesn't mean the car suddenly got smart and now understands what it is looking at - it just means it's using a different set of rules for when to switch lanes, brake, etc.

I would argue reasonable to believe it will be notably better until it is catastrophically wrong. The issue with neural nets is that they can be really well trained within the bounds of training set; however, when faced with novel, there is no underlying abstraction inference on which to revert. Look instances where autonomous cars are disable or have weird responses when people modify signs[1]. The driving will be better, but the when the information sits squarely outside of the norm (art on the road, abstract images that defy normal depth cues, etc), the car will do very unexpected things.

As humans, we have this issue already. We call them "optical illusions." Optical illusions are really brain neural net failures where there are competing narratives between evaluations of the information provided by our senses we conclude invalid results. We don't have the ability to process absolute light values, so we infer colorspace by relative color within our 3D mental model, which can result in inferring a difference in color where there is none.

1. https://spectrum.ieee.org/slight-street-sign-modifications-c... 2. https://en.wikipedia.org/wiki/Checker_shadow_illusion

You’re right. comma.ai is making the same mistake and it shows in their current experimental mode.

I find the argument that you can allow both longitudinal and lateral control to be expressed by a black box difficult to accept. The proof of the driving is in the handling.

There’s a video of an engineer at comma.ai mocking self-driving teams for semantically labeling objects on the road in SAE Level 2 and beyond systems.

Which screams stupid to me. Why would you be proud that your system has no idea what a pedestrian is. Or what a traffic cone or a speed limit sign are? Idiotic.

At least Tesla still semantically labels. But I argue that you can’t get smooth driving out of neural network controls because they inherently have no idea what smooth driving explicitly is. They estimate it.

The best they can do is attempt to mock it, which without programmer-defined rails WILL result in excessively high jerk and acceleration numbers, as proven by openpilot’s implementation.

The problem is manual labelling doesn't scale. Switching to an unsupervised approach is what made many huge AI advances over the past couple years possible since we're mostly limited by compute now. For Tesla (who have practically unlimited driving footage) this is particularly advantageous.

It’s not the labeling itself, it’s whether or not you’re allowing unfiltered longitudinal control to originate from neural nets.

Good but erratic lateral control at worst results in hugging a lane line.

Anything less than perfect longitudinal control means the driver will toggle it on and off during portions of traffic where it won’t follow the flow of traffic closely enough, or if it follows to closely, or if you’re experiencing a high cabin jerk factor.

You cannot screw up longitudinal control. It’s nearly the whole experience.

I think this is what end-to-end models will be best at, since they'll be able to account for far more environmental factors than a manually written planner flipping between the stop light deceleration curve and car-ahead curve constantly, for example.

Yes it means there is no difference. Neural nets are coded by programmers.

> They were also encouraged by Musk saying that Tesla would deliver v12 not as “beta” software.

> The update is still listed as “beta” in the release notes.

Musk being Musk.

Kinda hate that "Beta" is now mostly a marketing term for "be nice to us". When you're shipping software with potential safety implications to thousands of vehicles in the wild that doesn't seem like a beta in the classic sense. It's just a release that you don't want to be judged too harshly.

What is your suggestion as an alternative program?

“public danger edition” is pretty accurate.

Just don't call it a beta. It's released software at this point. If you want to limit the release to a smaller group of select users you can do that regardless of what you call it.

Edit: If you need a marketing term, Microsoft calls similar releases "Insider Preview Builds"[1]. You don't have to call it that, but this example is a lot more descriptive than "beta". I wonder what Tesla calls actual pre-release software that's not ready to go out to customers. "Alpha"?

[1] https://www.microsoft.com/en-us/windowsinsider/

Musk’s statement was that FSD will exit being a beta program in the v12 major version chain, not that that v12.n is not beta.

... So, v1 to v12.x where x is unknown: beta, v12.y where y>x: not beta.

That is a _spectacularly_ weird version scheme, but of course has the benefit that there is no rush; provided they always stick with 12.something, it is always true.

Given that this is an entirely new FSD engine, Elon should reset the counter for "X million miles driven with only Y accidents".

This Electrek headline is quite poor. I think the title should be updated to:

> Tesla releases FSD Beta v12 with AI vehicle controls

Which uses the various phrases present in the Electrek article and more concisely gets to the point, without the 'last hope' spin which is tangential and not itself newsworthy.

(I would personally have selected "ML" rather than "AI", but I wanted to try and write a better Electrek headline using their own article's phrases, so AI it is.)

Ok, let's use that above. Thanks!

Some background here: in the autonomous vehicles world, it's common to split things up into 'perception' (detecting things around the car and creating a world model), 'planning' (deciding what to do), and 'controls' (giving gas, brake, steering etc so the car smoothly follows the plan). Saying telsa is using AI 'controls' here is probably a misnomer, Musk's statement indicates they are moving the entire planning stack to a NN.

'End to end' is also a bit misleading, as this usually indicates that perception and planning are fused into one network. They aren't doing this (which is probably a good idea), it's only 'end to end' from the planning and possibly controls side.

“FSD Beta v12 upgrades the city-streets driving stack to a single end-to-end neural network trained on millions of video clips, replacing over 300k lines of explicit C++ code.”

That sounds terrifying.

No it doesn‘t. What‘s terrifying is the previous non-FSD code that just centers the car in the lane and will hit anything protruding into the lane. There is little reason to believe that the vision will fail worse than before or that hand rolled control code can do a better job based on wrong and unreliable input data from a net.

Additionally this does not preclude a safety check around the outputs, preventing, for example, an extreme acceleration. This would be similar to any checks put on ChatGPT output.

I wonder how they will handle the previous visualization though. Maybe it would make sense to also predict the visualization from the same network based on labeling the data (perhaps using the old methods) but unless that is somehow checked against the control decisions, it will not instill the same confidence or feeling of understanding in the user. But that doesn‘t mean that the driving is worse.

> What‘s terrifying is the previous non-FSD code that just centers the car in the lane and will hit anything protruding into the lane.

Are you sure this is what happened before? I've definitely seen Teslas avoiding obstacles in the road in various YouTube videos. Even "classic" (non neural net) robotics navigation systems can avoid static obstacles in their path pretty reliably.

Well, the non FSD autopilot always does this for me. It would certainly hit things regularly. It does recognize moving cars but it would have hit countless parked cars on the right side had I not intervened.

Autopilot is entirely different from FSD.

It is not entirely different because the classic Autopilot was migrated from radar to vision-only and that surely was done by sharing some code. However the classic autopilot is outdated by 4 years. But my point was not about Autopilot vs. FSD but about the fact that the C++ control code is often rather stupid and unlikely to be as robust as end-to-end learning.

Some object detection code maybe, but behavior is entirely different. Zero code overlap

Currently it appears the new planner net is not trained directly on video but on the semantic labels from the current gen perception system (e.g., BEV Lanes/Vehicles/VRUs). Accordingly, they still show them on the screen.

... Until FSD v13, of course.

The thirteenth "within 18 months FSD!" announcement yet!

I loved how the AI Lex used to interview Musk (most recently) poked fun at this very topic. They're all three (AI included) master bullshitters.

How can this title be true? Did Musk force them to use these exact words?

I think the biggest lie about the Tesla sensor suite as it exists today is that it’s suitable for Tesla Vision and that it can satisfy Musk’s fair concept of “if humans can drive with only vision, a computer should be able to, too.”

As far as I understand, those cameras are roughly equivalent to functionally 1080p devices which can’t even coherently recreate license plate characters a few meters away from your vehicle.

The current resolution of the cameras on the HW3 systems is 1.2 megapixels. HW4 is 5 megapixels. So, less than 1080p per camera for most of the Tesla cars is HW3. 1280x960, I believe.

Not sure why reading license plates is a meaningful indicator of capability though. Most people can't read license plates clearly from a distance that doesn't impede driving, and there's many lighting-relating issues that make reading license plates difficult for cameras.

At less than 1080p, you can’t make out digits one car length in front of you.

How are you going to read the speed limit?

Speed limit signs are much larger, and much clearer than license plates. Anecdotally, speed limit signs are detected quite a distance away in my Model 3. Certainly much greater than 1 car length.

> The Tesla v12 software update is expected to introduce what CEO Elon Musk has been calling “end-to-end neural nets”. The biggest difference with previous FSD updates is that the vehicle’s controls would now be handled by neural nets rather than being coded by programmers.

Up until now Tesla's software used a neural net for vision (i.e to turn camera view of road into an internal representation of road, cars, people, traffic signs, etc), but used conventional programming (300K lines of C++!) to actually drive the car - change speed, steer, switch lanes etc to follow the desired route given the cars position on the road/etc provided by the vision input.

This new release is now replacing those 300K lines of C++ with another neural net that has learned when to change lanes, how to steer etc. There is still just a vision system that "sees" the road, and a driving system that steers the car based on this vision input.

Too bad the real world isn't a video game or Tesla simulator world ...

This sounds very scary to me. Neural Networks have great results on many classification tasks, however they "hallucinate" on completely random and/or easy inputs in an unpredictable way. I already delivered hundreds of classifiers to production in my career and I always had to safeguard inputs for cases I knew the answers or it was wrong. Production ML is always hybrid with hand-made rules. I hope this is just a marketing-term and not something forced to please Musk.

Why would this be different with a net that only handles the input? It would be the same. All v12 does is learn control outputs directly from camera data instead of, for example, learning to detect a traffic light and then have some hand rolled logic. If the input is unpredictably new it would have failed in either case.

Clickbait by the author's own admission

>Sure, maybe I should have written "my" last hope, but come on.

https://twitter.com/FredericLambert/status/17494587384428711...

Has anyone else been able to download the Tesla Network app from the Apple App Store because I can’t find it.

Reply on news.ycombinator.com