I get that waiting around doing nothing is annoying, but it doesn't have to be that way. For example, if I'm meeting up with some friends to go out, I'd much rather meet them in a pub rather than meet them at a bus or train station. That way their evening can start without me.
I'd also happily live somewhere with other people that don't put a high value on punctuality. As an example, I've heard in Costa Rica they have cultural trait of running on "tico time". I've only ever met a couple of people from Costa Rica before, so I can't say how common this is, but what I can say is that I'd not be offended by it, and I'd hope it was something we would all be a bit more laid back about than where I currently live.