I'm not sure where this "frat house" thing comes from (scare quotes, not direct quotation). Have you ever been to a frat house? Believe me, they have nothing in common with a summer at YCombinator. I've described yc dinners as being "like a high school lunchroom where everyone is happy to see you and every table is the cool kid's table". Women are utterly and completely welcome. Minorities are welcome. Bring them your nerds, your socially inept, your ambitious hackers yearning to be free. Frat houses are all about pecking orders and childish humor. YC is genuinely about mutual support and an open exchange of ideas.
If "frat house" means that there aren't many women present, I can only guess as to why. There are a variety of social and cultural factors that push the majority of women away from hacking at a young age. I can't point the finger of blame at anyone in particular, but I can report on what I have observed. Women are generally underrepresented in computer science departments, engineering programs, computer clubs and yes, startup incubators. It has nothing to do with Paul Graham or the YC partners. We're all responsible as members of society at large.
I understand your reasons for not appliyng to YCombinator. TechStars is a great program, and I'm glad that you've thrived there. But there's something to be said for seeing things with your own eyes. I would be very unhappy if someone dismissed me out of hand because of something that they'd heard. I can only believe that YCombinator’s positive reputation will outweigh whatever negative reputation that they have fairly or unfairly received.