If we're talking about our own vision rather than another driver's, yes, fog lamps might detract a bit from our night vision. The truth is, if you're using them in an actual fog, your distance vision is fucked anyway, and it's your regular headlights (which do not power off when the fog lamps are on) which are doing the damage to your night vision, since they're focused at eye level and reflecting back. The fog lamps aren't a big worry in that regard.
> I remember reading about this in an article written by a rally driver in ADAC magazine maybe 20 years ago now.
I've seen those arrays of fog lamps installed on rally cars, and I'm not sure what they're about but they're a pretty different paradigm than anything that comes installed by the factory on a passenger car.