Then we invented everything else and along with that UTC. Which works until we have colonies on another planet. We should go ahead and move to a galactic time standard.
In practice, if you keep in touch on a regular basis, you don't actually need to remember their time zone at all. You can remember the correct time to call in relation to your own schedule: "Got to call Grandma as soon as I get home from work. If I wait until after I eat dinner, she will be asleep."
In actual practice, that's how most people seem to handle long distance relationships where they keep in touch regularly. (Or maybe that's just me.)
Plus, it's the relatively rare Luddite that you can only contact by phone these days. A lot of people keep in touch with long distance relatives via email, Facebook or text messages, all of which allow you to send the message whenever you feel like it without worrying if the other party is currently available for live interaction. If you can contact them via such a method, you can arrange an appointment for a live discussion by asking them when they would be available for such.