You don't get to be the arbiter of who is "actually offended." I agree that there are people who take any amount of power and want to expert it for the wrong reasons, but that isn't an indictment against CoCs, it's an indictment of bad behavior.
That is exactly my point. Some committee member doesn't get to say "that may be offensive to someone" either. They need to wait until someone actually reports being offended.