Because it breaks with how we always imagined Tolkien's world to look.
I find it mildly offensive when they reduce my identity to the color my skin. My identity is far richer and includes everything from food and clothing to customs and traditions.
An Indian actor playing an Egnlishman with English customs and traditions does nothing for me. It would take an Indian actor playing an Indian man for me to take this “diversity” seriously. Of course, that would mean creating entirely new narratives and stories, which, of course, Hollywood can’t and won’t do.
So we get brown and black actors shoehorned into pretending to be white - the lowest common denominator of “diversity”.