Your brain fills a lot of gaps, and your senses have a lot of noise, but it's still closer to reality than it isn't, particularly for the fovea.

Still, your brain doesn't make up what's behind a building violating our reliance on straight lines of light (unless you use sensorial "enhancement" drugs), or make up detailed images of what's in your peripheral vision (again, a known effect of many psychedelics), then presents it to you as truth, that's rather imagination and it uses a very different part of the brain, though connected to the visual cortex through the V1 area.

I'm sure it can have some uses (AI racing, anyone?), but most self-driving car decisions are ruled by what has been seen (signs/lights) and what can be seen, because that's how we arranged our streets and roads to be safe for us.