The majority of people will still cluster around the same best guesses, and that’s all that matters to the algorithm.

Yes, it’s annoying, but that doesn’t matter to the algorithm.