The English idiom is precisely "throwing out the baby with the bath water." So, as they say, you "hit the nail on the head."