This is the concept of affirmative action at it's core. Do we base admission partly on demographics instead of merit? On one hand it may help promote the minority, though there are downsides as well.
I'd pose the same question to professional women as has been posed to black males for decades now: "Are you wanting to define yourself as a great lawyer/doctor/engineer, or a great _black_ (or in this case _woman_) lawyer/doctor/engineer?"
Personally, I have far, far greater respect when the gender and race is not part of the definition. Otherwise it seems there is a message of "I get extra points due to the mistake of my birth". Is that not just another form of 'privilege'?
I feel much the same way.
Personally I think affirmative action is more justifiable in something like college admissions where it's a critical gateway that a majority of people are expected to pass through on the way to success, versus conference speakers who are ostensibly there to talk about something they've already done rather than some future potential.
In other words, if you let more minorities into college, I think it's a credible argument that many will fulfill their potential and that will help correct structural problems over time, whereas I don't see how getting more women speakers at tech conferences does anything to solve the larger gender imbalance other than paper over the problem at the edges.