It annoys me either way. Just put your phone on silent (unless you have certain contacts that need to get through in an emergency).

It's especially annoying when you don't look at your phone because you're just making noise for no reason. It's on the same level as playing music through your phone on the subway with no regard for those around you.

I know someone who will sit there for an hour texting people and each text that comes in rings the phone. They're literally staring at their phone and think that they need the ringer on.