I think that deliberately building in buffer so that people's expectations are explicitly set is much better than constantly having to absent yourself "early".