Of course you can. For instance, if you would reply to the request by insulting the physical appearance of the one making the request, that would be rude.