I am in the same position as you. I constantly and graphically think about sex with my female colleagues. However it has actually helped my career because I don't interrupt my female colleagues at the whiteboard or in meetings, rather I nod and affirm them while I fantasize about sex with them. As an ironic consequence I have gotten a reputation for being respectful of women's ideas and encouraging of their career development. Shows how low the bar is in Silicon Valley -- simply not interrupting/berating women makes you a well-regarded manager.

While this has led to some technically poor decisions because they don't benefit from engaged debate, the women who have worked for me have been basically very competent so the errors have not hurt me as much as my personal reputation has helped me.