One of the few times I was caught was at a psychoanalysis conference where I stood out for being roughly 20 years younger than most attendees. I wish I had a badge in that case because I was very obviously called out due to my age alone.
As best I can tell the distinguishing characteristic between people who believe in the existence of white/male privilege and those who do not is political affiliation.
The people who are not white, not cis-male, or not straight are the people who have encountered consistent barriers as a result of these negations.
These are all immutable parameters, not political affiliations. No one chooses to be white (or not). No one chooses to be male (or not). No one chooses to be straight (or not).
Women? Barriers.
Trans? Barriers.
Black? Barriers.
Middle Eastern? Even though we're classified as white by tax code? Barriers.
Gay? Barriers.
Jewish? Muslim? Even Catholic? (Considering the mental trauma that often arises with rapid disillusionment from religion, this is largely considered immutable) Barriers.
Hispanic? Asian? Aboriginal? Native American? Barriers.
I get that it's uncomfortable to hackernews because we like to think that every startup operates in a meritocratic manner, but that's been consistently proven false just with the sheer volume of white straight cis-men managing and leading companies in the US, Australia, the EU, wherever you look.
tl;dr: it's not political because the people who understand that it's an issue overwhelmingly tend to be the victims, and the people who don't understand that it's an issue mostly tend not to be victims, and the victims tend to be victimized because of immutable traits rather than ideologies.