This is what I do, and it's never a problem. The reasons why you wouldn't want somebody touching your scren are pretty obvious. I'm not apologizing to someone else for /them/ smudging their filthy hand marks all over /my/ screen!
There's a "please", it's not like it's rude...