I think the overlooked part is "seamlessly integrate". Signal to noise ratio is important, and it is not trivial to "just add sensors". I think the analogy of "too many cooks in the kitchen" applies here.
Case in point: the whole radar removal and reintroduction flip flop. It seems like they don’t believe themselves that cameras are sufficient.
Object detection is inextricably linked with prediction which are both inextricably linked with planning.
As you point out, the companies that have "solved" it don't have scale.
They don’t have scale yet for different reasons (adverse weather driving, long testing cycles, costly operations setup). “If they don’t have scale, that must mean sensor fusion doesn’t work” is quite a ridiculous argument.
If you think sensor fusion is not possible, then you must also believe Tesla’s attempts at re-integrating radar will not work and that they won’t achieve their stated goal of FSD.
But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles. This is what I mean when I say it has not been proven at scale. The edge cases matter (arguably they are all that really matter), and "solutions with more sensors work better at object detection in artificial benchmarks" is not nearly as convincing to me as it seems to be to you. Again, object detection does not exist in a vacuum, and a solution that generally behaves better in a very constrained environment might actually behave worse when you throw all the edge cases at it and add on prediction and planning.
How actionable you make it depends on how good your engineers are. Diverse input data is definitely more actionable than just RGB pixels from images.
> But maybe it's not. Maybe the additional flops required for sensor fusion would be more useful if given over to the planning part of the problem. After all, you don't have infinite cycles.
On the contrary, it reduces your compute requirements. For example, adding lidar means you are getting direct distance measurements instead of wasting compute cycles predicting distance using voxels. It's a much more effective use of compute.
LIDAR doesn't work great in the rain/snow, so you still need a system that is predicting distance using voxels.
Doesn't really sound like less compute to me. I feel like I'm repeating myself but again this is what I mean by scale matters. A self-driving system that performs better in perfect conditions is not even a good system if it doesn't work in the rain.
Which is the point the detractors of "fuse all the things" have been making. Don't get me wrong, I think probably those same detractors (e.g. Elon Musk) have also not been entirely intellectually honest with their arguments and have ulterior motives like cost savings and saving face, but that doesn't make the arguments wrong.
Specifically this is the relevant part: https://youtu.be/g6bOwQdCJrc?t=1370
They removed radar because it was making things worse.