Because there is in fact no such thing as a "white person".
Believe it or not, there is a coherent shared cultural identity of American black people, most of whom are descended from slaves brought here from Africa who had their original cultures stripped from them, and who belonged to a community segregated against legally into the 1970s. That those people came from several different countries on the continent of Africa is of little meaning, because their culture was deliberately cut off from its roots, and was built largely anew in the United States.
Your inclination is to assume that if "black culture" is a thing, then "white culture" must also be a thing. No, that is not the case. What there is in addition to "black culture" is:
* Irish-American culture, which has a museum, parades, and a holiday,
* Italian-American culture, which has a museum, parades, and a holiday,
* German-American culture, which has a museum and parades,
* Polish-American culture, which has a museum, parades, and a holiday
I can go on.
The notion that there is some notional "white culture" is artificial, a lie begun in earnest in the 20th century. No, really. Before "white culture", the Anglo- and Germanic-American communities in the US targeted Italians, the Irish, and the Polish (as well as Catholics of all nationalities). When the writing appeared on the wall for end of Jim Crow, "white culture" emerged as a solidarity movement among those people prejudiced against blacks and black culture.
This is a big part of why talking about "white history museums" generates so much flak. It's ahistorical, doesn't make much cultural sense (the Irish and Germans do not in fact have a huge overlap in music, art, language, or food), but is very much connected to 20th century racism. It's an Occam's Razor problem: the simplest thing to conclude someone would be talking about with a "white history museum" is racism.
People can hardly be blamed for wondering aloud about this stuff. After all: practically every government form you fill out asks your race, and has "white" as an option. These forms aren't all designed to perpetuate racism --- in fact, most of the forms that offer you the option of declaring yourself "white" have the opposite intention. But the concept of "whiteness" actually does derive from racism (again: the concept of "Irish-Americanness" does not!), and while the concept of "blackness" does as well, black people really do have an important, coherent, complex, historically rich culture built inside that racist framework we forced them into.
Worth checking out, also because it's just kind of amazing that you can: the Baldwin/Buckley debate:
https://www.youtube.com/watch?v=oFeoS41xe7w