I run an organization (hypatia.software [Hypatia Software Organization]) where the Code of Conduct is enforced and it is effective.
In the end, it's probably equivalent to your answer. Because when the people in your organization feel someone is harassing people and you decide to kick him out, you're probably simply describing his behaviour and referencing your CoC, but not doing some detailed analysis how and why he did so.
Personally, I tend to steer on the side of a broadly defined CoC backed up with a very clear dispute resolution policy (there's plenty of examples of both of these - Debian has a pretty decent set for large projects). In a functioning community, reaching for this should only be required very rarely, so they shouldn't affect anyone most of the time - and they quite obviously don't in the case of open-source projects, or we'd hear about actual cases where people have been kicked out of projects for very little a lot more than we do.