Ad hominem is a weird one where the more abstract the argument, the less it applies, but the more concrete and visceral it is the better it is as a first-pass filter.

Like, think of the worst guy you know. Now think of what that guy thinks about the state of the world, in as concrete detail as you can. That's probably a pretty good reason in itself not to think that, because if you think like that for too long, you risk becoming that guy yourself.

This feels like it should work far less often than it actually does.

This is less true, slippery slope as a fallacy is having large unclear jumps chained together to make a really improbable outcome because each improbable chain scales off all the previous pieces. But ad hominem as an argument is only really valid when the attack against the person is highlighting conflicts of interest, which is more reason for scrutiny of the original argument than it is cause for ignoring it entirely. You should still attack the argument itself.