By that argument, it's we who are doing all the upvotes, downvotes, and flags on the site. I don't think most HN readers would look at it that way.

The distinction between moderation done by users (votes, flags, etc.) and moderation done by admins (killing posts, banning accounts, etc.) is long established and well understood by the community. It's not about blaming.