To me checking a message is much more rude than checking the time. It's actively stating "I have more important people to talk to than you".
I let everyone know that phones, with me, are low priority - I have an answering machine. Call logs. I can see who you are and when you called whenever I have time - call me twice if it's actually important.
But I'll still check my phone to see who's calling me when it rings, because doing so is such a minor interruption and might be important and not everyone will try again - I'll probably pick up if a hospital or police department calls me. It's not saying "I have more important people to talk to than you", it's saying "Talking with you is not the most important thing that can possibly exist at this time", which is true and expectable - wouldn't you like to know ASAP if your family is in the hospital?