Is anyone arguing for replacing vision with lidar and radar instead of combining it?
So far their biggest problem is that they allow flicker. That's why sometimes the software picks the wrong lines. (Of course this is a very hard problem. Our brain conveniently smooths over sensory changes for us, because that's how our everyday reality is. Things don't flicker in and out of existence, nor does a car suddenly appear as a different thing, then switches back.)
And if it turns out the sensor(s) failed, it has to be able to handle that too.