Another possible explanation could be that animals feeded on an artificial monotonous diet and set free from predation pressure, start showing genetic/colour defects quickly. Similar colour issues happen with captive birds and fishes unless we provide extra charotenes to complete its diet. In the wild the diet is much more diverse and finding charotenes is not a problem.

Or that when the mice population increases the animals just fight more. Domestic mouse males fight and bite other males. Mother with litters will attack males also. More animals in the same place = more scars by bites, and this could easily lead to an increase on white (regrown) hair patches. A simple explanation that does not depend on human contact. Is normal in birds to create white feathers after a predation event to quickly replace missing ones and cover bald patches.

And another explanation could be that when the mice population increases, their individual size decreases because there are less food for each one (and different hormone interactions non necessarily human related). If we extract some of those mice and put it in a new cage subject to the same human manipulation but with a more diverse sources of food, would the average skull size quickly increase again? Would the animals with white patches decrease? I bet that yes.