So here's the thing - in reality black slaves actually existed. So if you cast a white guy to play a black slave then yeah people would be a little bit confused. Dwarves are a fictional species which have never existed, and even if all dwarves have been described explicitly as being white in the books (and I don't think they have) there has been enough artistic license taken in the various recent films that straying from this is really not too outrageous.

I think you're kind of revealing a bit more about own feelings on race than you realise.