I am not OP (they seem to have been downvoted to oblivion) but a fascinating book I read recently on this topic was "The Invention of the White Race" by Theodore W. Allen.

The book traces the historical development of the concept of whiteness from early Britain, where the Irish were considered non-white as a means to "other" and impoverish them, to pre-revolution Haiti, where people of mixed race could attain certificates of Whiteness that would grant them the same rights and privileges as non-mixed French people.

Those are just two of the anecdotes I remember from the book. It's really worth a read if you're into the whole "race as a social construct" thing, since it digs into the mechanics of how that social construct was created.