Fry was saying that in response to people who were offended by his being gay. I hope you find at least a little humor in the fact that you've taken a quote from a man trying to fight bigotry and used it to defend bigotry? No?
It's not thought-crime to say, "Hey, women are people too, and women are way more interesting when they're treated like people, not like sex toys." It's not gender-invisibility to say that there's more to a woman than her vagina, or even that there are funnier things about a vagina than your bragging about being in it. It's not political correctness to say, "When the things you say make women – not just a single anecdotal woman, but women plural, many women – uncomfortable, then probably you can find a better way to get laughs than through their uncomfortableness."
Humor is at its finest when it's used to prod those in a position of power. Satire, farce, bawdiness have all been used as tools to make the oppressed in society feel better about themselves, to make people who don't notice oppression realize that it's there, but to laugh at their own uncomfortableness. Humor's at its lowest when it's used to kick people who're already down. When it's used not to help humanity rise above itself, but to remind the lowly of their lowliness.
The great comedian Louis CK, of whom you've doubtless heard, has a routine where he goes after white people who get pissy about not being allowed to say the word "nigger". Like, it's racist that black people can use it, but not white people. And his response is: Dude, does it really matter that fucking much to you that you don't get to use a word? Is your being asked not to say "nigger" really as unjust as the fact that, fifty years after the Civil Rights Movement, we still have to have a conversation about why a word associated with lynchings and segregation is maybe not a word to be used lightly?
After decades of women fighting for equality, we still live in a society whose pop culture revolves around the sexualization and objectification of women. Where women make 81 cents to every man's dollar (source: narrowthegapp.com, made by Hacker News's favorite Gina Trapini). Where our government is primarily male, and its two highest positions have never been occupied by a woman. Where these male-dominated governments try to pass laws that deny women control over their own bodies. Where nearly every Fortune 500 company is headed by a man. Women have it better now than they had it, say, a hundred years ago, but that's not saying fucking much.
Do you really care so much about "that's what she said" that this is where you feel like making a stand? Buddy, it's a stupid joke, it exists only to make men feel manly for bragging about their penis, and it's not even very funny. Want funny? Watch Bridesmaids. There's a hilarious movie written by and starring some very funny women. They even make jokes about sex and vaginas! OMG so un-PC!
George Carlin once said, "Have you noticed that most of the women who are against abortion are women you wouldn’t want to fuck in the first place?" I'm going to paraphrase him: Have you noticed that most of the people who defend idiotic sexist jokes are people who aren't actually funny?