Do they? It seems to me that a Code of Conduct just gives these sorts of people an entirely new set of rules to litigate/bikeshed. If the Code of Conduct prohibits personal attacks, for example, the sorts of people who will harangue mods for admonishing them to be nice will, instead, harangue the mods about how their statement wasn't actually a personal attack.
>It is sad that we need to make rules like "do not be a jerk"
I don't think the problem is in the rules or lack thereof. I think the problem is there in the (lack of) enforcement. Many internet communities are allergic to anything that smacks of "censorship", and thus find themselves at a loss whenever someone comes along and willfully violates community norms. We should remember that "well kept gardens die by pacifism" [1], and no matter how detailed and elaborate the rules, someone needs to be willing to actually enforce them. A Code of Conduct (or some other formalized set of rules) can help here insofar as it allows the moderator to point to something outside their own thoughts as a justification for why they're taking action against a transgressive community member.
However, it's possible to go too far. Wikipedia is a great example of what happens when rules become too elaborate. There are many trolls on Wikipedia who are around simply because they've mastered the arcana of the various rules and procedures Wikipedia has built up in its efforts to create an "objective" standard of community behavior. Whenever they're brought up for discipline, they can point to some previous case where superficially similar behavior was excused.
[1]: https://www.greaterwrong.com/posts/tscc3e5eujrsEeFN4/well-ke...