I think that the most visible advantage/disadvantage of having different skin colors, are the frequency and severity of sunburns. Black people are more resistant to sunburns than white people. And given that they would need to move hundreds or thousands of kilometers, to be exposed to lower latitudes and heavier solar radiation, and that it was a loooong trip in those prehistoric times, maybe they didn't get to see the advantages/disadvantages.

But then again... who knows?!