Get used to it. I think paying someone else to deal with billing stuff is a fine idea, but don't distance yourself too much. I could easily delegate out the "refund reply" job, but I choose to do it personally.

It's really easy to turn a refund request into a repeat customer. If you're getting people "gaming" the system, change your policy.