I have never hit my kid (but my ex-wife has) as for whatever reason, he has an amount of respect for me that my body language alone is enough.

Here is the thing, you probably not doing something wrong. As long it isn't a everyday thing, it isn't so bad as people make it to be. The 'tap' to stop, then talking with him/her works, worked on me when I was a kid, probably the only thing that did to be honest. The 'physical violence' was the awakening call, then the conversation after that grounded it.

ps: if you just slap the living crap of him, say he is bad, and send him to his room without talking with him, then sure, that is an absolutely shitty behaviour, but I don't think this is what you are doing from your text.