Actually this is something that I've ran into with my girlfriend. We both work in multinationals that have offices close to each other so we leave and come together.

Her boss is in another country and doesn't enforce a starting hour for her schedule. But I'm having trouble convincing her to start earlier (I want 8 AM) in order to leave at 5 PM. Her reason: she doesn't want that the other girls in her department (Recruitment) mistakenly consider her a slacker behind her back. She thinks that sometime down the road, when asked, somebody will answer that she doesn't work hard enough because she's leaving earlier than 6 PM without mentioning that she's coming in earlier.