The points you made in your prior comment are all valid, but don't really alter the reality of the difference or of the idea of privilege (which everyone has in some circumstances; as you note, women working in child care have a massive amount of privilege over men). Interestingly, I know some radical feminists who view nearly all of it as symptoms of patriarchy.
For example, a side effect of men being grownups, and women being child-like, is that men suffer consequences in court at a much higher rate than women. Women are viewed as both weak and incapable of making rational choices, and so when they commit crimes, including violence against men, they receive child-like treatment. Where an abusive man might spend time in jail, an abusive woman gets a slap on the wrist and the man gets snickered at for being too weak to stand up to the woman. Woman==Weak and Victim, Man==Strong and In Control.
Similarly, women are seen as safe to be around children because they effectively are children according to this worldview. Men should be ambitious and driven, and shouldn't be wasting time playing with kids, and should be viewed with suspicion if that's how they want to spend their time...there must be something wrong with them, some ulterior motive. This is insulting to both men and women; and it just falls out naturally from the way men and women are currently viewed by our institutions and our society. We could go on for days talking about ways women and men get treated differently, and it's not really about who gets treated worse (though, again, most of the time, us white guys get a hell of a lot of advantages over just about everyone else as long as our goals align with what we're expected to want out of life).
So, I think it's unfortunate that so much time gets spent trying to pick apart the idea of privilege without realizing that it's dangerous for everyone for these kinds of inequalities to exist, but it's generally more oppressive for non-white, non-male, non-straight folks.
I absolutely concede that it's complicated. I doubt anyone, even the most militant old school feminist, would argue that it's simple. It's just a matter of checking your privilege now and then; it's all most folks are asking for when they talk about privilege, and I think that's entirely fair.
I'll give an example: I'm involved in activism in my local community, and have found myself talking to media a lot lately. One of my friends noted that all the people in our activist community who were talking to media with regularity are white, middle class, able-bodied, straight-looking men, even though plenty of women, black and brown folks, disabled folks, obviously LGBTQ, etc. people exist within this community. Media were simply walking into the room, looking around, and picking the person who looked most like they were in charge...wouldn't you know, they'd pick the middle class straight white guy to talk to? It's the way the world is. Changing it means checking that privilege on a conscious level. The next time I was asked for an interview I redirected him to a woman in a wheelchair who is probably a much better speaker for our cause than I am; in a world without this pre-existing bias, she's the person he would have talked to, because she was heading up the meeting. But, due to that bias, he came to me. When you start looking for this bias (and you can call it whatever you like, but this is what people mean when they say "privilege"), you'll start to see it everywhere.
Anyway, what I'm trying to say is that we can recognize that it's complicated and that everyone is dealing with shit, while also recognizing trends and societal bias and favoritism.