It's not a small incentive though. It's a toy model where the only thing that the agent cares about is how many of his neighbours are the same color as him. Every single person in this model society will immediately move house if 1/3 of their immediate neighbours are the wrong color and not move to any such neighbourhood for any reason, since there are no other motivations in the model.
Which apparently was considered a 'mild' preference by the person who made the model, in the middle of the civil rights era.
It basically renders any immigration inpossible for example, is that really a mild preference?