Wiping your hands on your pants when there's towels or something is kind of silly, but how on earth does the article of clothing define if you can wipe your hands on it?
Are you wearing vinyl pants or something?
If I'm wearing $50 jeans, I might wipe my hands on them.
If I'm wearing a $2,000 suit, I won't wipe my hands on it.
a) it's just water, and very little of it at that since you do the directed shake into the sink first
b) the thought of what I'm wearing never enters my mind when I wipe my hands on my pants
I don't think anyone really does it purposely, you just blank for a second and the action is done and now you have a wet spot on your pants
The too nice viewpoint is less relatable, but could be from fastidiousness or OCD applied to the clothes themselves, rather than being concerned about your hands.