Working for extrinsic incentives often fail to keep up - it is very hard to be motivated to do something because it will go on a promo packet after the first year when a packet doesn't happen.
Doing things for fun is great & that's what I do, also usually a step removed from my core expertise where I can do less damage in general.
But fun tends to have its own "flow" which might insulate you from other signs of organizational stress.
I've had these sort of "fun side quest" blow up in my face because I didn't realize there was a spotlight on me.
If you can't observe the spotlight coming down high on up, the same quirky day jaunt into some old code can be seen as "hard to direct".
This was also a side-effect of being a remote worker in 2005, because if I was in office I might have noticed there is a shift in tone.