It may be that you’re working so hard at work that you’re exhausting yourself for the time you have outside of work. Especially if you’re using caffeine at work to push through when a caffeine free human would’ve been unable to continue.
Yeah, I've limited my caffeine consumption to only in the morning. I really envy friends who have the same type of demanding jobs and still have energy left for other things. They're probably genetically blissed and have more stamina.