Do you not think slavery, Jim Crow, civil rights, etc. is a worthwhile topic? Is it not clearly an extremely important area that deserves an entire museum, rather than a couple exhibits here and there?
And finally, imagine a hypothetical country where we enslaved black people for 100 years, and then ceased all slavery and racism completely. One day there were slaves, the next slavery is abolished and nobody is racist. Is the problem solved? Are black people now equal to everyone else? No, because history didn't disappear, they still used to be slaves. They have no money, no land, no friends in power, no education, etc. Their lives still suck, no matter how impartial everyone else is. Again, you can't solve a problem by ignoring it. Pretending black people have no disadvantages and can be treated exactly the same as white people is just naïve, and impractical. It is not racist to acknowledge existing racism, and try to counteract it. There is a line, and it's completely possible for racism to negatively affect white people, but building a black history museum does not come anywhere near that line.