If you are going to use that expression, you should use it in it's entirety. A few bad apples spoil the bunch.

A few sexists are creating a a negative situation that permeates the community/industry. Not everyone is to blame, not everyone is guilty of this sexism, but the problem is everybody's.