If it isn't rude, why are so many people offended by it?
People invented the "everyone's phone goes in a stack in the middle of the table, and if you touch it you pick up the check" custom because they find it so rude that people check their phones during dinner conversation.
When you're with someone, pay attention to them. Sure, if there's a long break in the conversation and neither of you has anything to say, maybe you can check your phone. But don't start checking it during an ongoing conversation; that's rude. That's a way of telling the person you're talking to that they're less important than the random minutia of facebook, texts asking if you're up for COD, the latest score in the football game, etc.
There are obvious exceptions. If mom's in the hospital due to a heart attack, or your best friend is stranded in the flood zone and you don't know if he's made it onto a helicopter yet, it's OK to signal to the people you're having a casual conversation with "hey, this is important enough that I'm willing to cut you off mid-sentence to keep up on it."