I think this is an oversimplification. People don't ask that question because humans have many many more senses they drive with than just our eyes. Humans even have accelerometers. If someone said "I'm deaf, blind in one eye, and can't sense acceleration" people would probably ask if they are fit to drive. Just being deaf is enough to start that conversation. The AARP even mentions that if someone is deaf, they should seriously consider not driving. The conversation of sensor fusion is important because humans require sensor fusion to be effective drivers too.
https://www.aarp.org/auto/driver-safety/driving-with-hearing...