The GP is simply suggesting that you're selective and biased in the issues you're trying to solve. Which isn't necessarily a bad thing: you have to choose your battles.
But someone else could propose only eating meat to solve the massive use of pesticide. Apart from the possible health issues, he would be as right in his position as you are.