It's a "few" who actually commit the crime. It's the vast majority who refuse to prosecute them for committing the crime.

Coping by only hearing the first half of the expression. “Spoils the bunch” is just as important as the number of bad apples.