One can argue that's just social conditioning. If you grew up with 3 am being lunch time, then it won't feel awkward. And you'll just accept that everyone in the world has a different lunchtime.
Having the whole of North America on one big time zone would be fairly easy. This is basically what China does.