Yes, but on the other hand, fallacious reasoning is rarely the source of disagreement or controversy in a debate. At least in my experience. All you're doing by saying "ad hominem" is shifting the focus to technical delivery of the argument rather than focusing on the real matter of the discussion.

And most people forget that calling something an ad hominem attack does not mean that it's wrong. Most things which get called ad hominem are usually not, in fact -- saying "you are dumb and your arguments is wrong because of X, Y, Z" is different than saying "your argument is wrong because you are dumb" (the latter would be ad hominem, the former is merely an irrelevant statement)