I'd like to offer two thoughts.
The first is that poor whites have much the same relationship with authority as people of color, but are almost completely excluded when it comes to any kind of social advocacy.
This is the essence of racism: it separates people. Those poor white people would be great allies, but no one dealing with issue of race cares about them or enough about their allegiance to approach them in a way that does not alienate them.
The second thought is on the nature of white privilege. All white people have privilege, but the privilege is this: We do not wear our status on our skin. So a black person can have high status but remain effectively low status while a low status white person can earn high status and be treated the way we should be treating everyone.
But, people being people, we show our status in many other ways, and so for a lot of white people, white privilege is irrelevant because they have no capacity to capitalize on it.
Hence, we get a bifurcated low status populace half of which believes prejudice==racism and that is the root cause of the issues; and a second half which believes those prejudices do not exist because they experience the same issues and incorrectly conclude that racism==prejudice does not exist (or is overblown) and/or feel oppressed themselves (which, of course, they are, but not by the people they think are oppressing them).
Sadly, the reality is that racism is this system of bifurcation. As long as they are disunited, the system will not change. At best, the system will flip; and it is this flip that people like the GP seem to fear.