What baffles me is that the car doesn’t have any trouble plowing right into things which are visible in the GUI. It looks like it sees the cyclist and goes right ahead into them. Anybody have an explanation for this behavior?
Part of the difficulty of trying to have reasonable discussions about FSD is that so much of it is black-boxy nobody really knows what's going on. I don't care if Tesla wants to keep its ML models proprietary, but I feel like we deserve to know how it uses those models to make driving decisions on public roads. And not just Tesla either; every self driving company should be required to open source their "dumb" code so that we understand them better. The competitive advantage comes from the ML models anyways, so it wouldn't be anti competitive to regulate that disclosure.
"data to understand fault and liability following a collision must be accessible;
[There are] sanctions for carmakers who fail to reveal how their systems work"
The proposals are very detailed but do seem to be addressing a number of areas around how autonomy is marketed, and how to ensure vehicles are safe and that vehicle makers are making available material information on the inner workings of their software.
It’s like GPT-3. In 9 cases out of ten, it responds in a way that lets you think the model has “grokked” facts of the world, and a human like sense of logic. In truth, it hasn’t. It’s just pretending. It’s taking the easy path by parroting back what it has seen before.
They say that ML requires a re-thinking on the part of the developer. Whereas with traditional programming it would be his job to “structure the problem and the path to its solution”, in ML, he should hold back his human notions and let the network discover its own structure.
To use an analogy: Whereas we teach our children concepts step by step - you and me, identifying objects, gasping objects, referring to objects, etc., all the way up in complexity to writing thoughtful comments on HN - the neural network is to be bombarded by unfiltered real world data. And the hope is that, if you just do this long enough, the AI will come up with the important underlying concepts by itself. That is to say, for GPT-3, the idea that words are really describing a real world out there, with objects that have certain properties, and that claims may be true or false. Or, for Tesla, that the car is a physical object in space, and that it can collide with other objects.
This is very unintuitive. It might be the right approach. But if I was building a self driving car, I would want to build that system block by block. Learn the basics. And only proceed to the next level of complexity once I have verified that the previous step has been properly learned.
1. Let the car learn to predict it’s own acceleration, steering and braking. 2. Repeat on different road surfaces. 3. Add other objects to avoid. 4. Train ability to tell objects from non-obstacles (fog, raindrop on camera) 5. Train lane markings, traffic lights on empty roads. ... X. Throw real world data at it. Now you can hope (and validate) that the model will have the foundation to make sense of it. Like a kid after Highschool. It will still make dumb mistakes. But at least you can talk about them and understand why they happened and take corrective action. Because, while you do not know the other’s mind, you do speak the same language.
With GPT-3 and with FSD, we just have no idea.
So if an incident like this happens and we don't hear anything from Tesla it tells me there's a far, far bigger problem here with their failure to collect diagnostics, or failure to analyze these problems. Ideally we should see a hyper detailed root cause analysis even going into the vision model inputs in this exact case and exactly how they failed to classify and detect this cyclist, and most importantly how the beta software is being improved to fix issues like this. Silence is deafening here.
Its sensitivity was probably lowered because it would brake "too often" for a comfortable consumer product if it was sufficiently cautious enough to avoid hitting pedestrians.
This is what happened with the Uber self-driving vehicle that struck and killed a pedestrian[1]. The system detected the pedestrian as an obstruction, but Uber had programmed the system to ignore it because of "phantom braking". Instead of braking, it proceeded to strike and kill the pedestrian.