But that's my external perspective.
I'd heard otherwise, but am glad to be corrected - it's a model many other businesses (NB: Not all) could utilise.
(Perhaps 20% of your time is alive and easy to take; on the other hand, engineers who've been working on the same fast paced project for years and have built large bodies of domain knowledge often don't see it the same way. Not a single engineer I work with does 20% time, to my knowledge.)
And, as everywhere in any company, you can do whatever you can get away with. From a pure career point of view, you can make a choice here: put in the 20% time for another project of your choice, to keep your skills sharp, have some fun and get some additional visibility.
Or put the extra 20% time into your main project for some marginal improvements. I can see that social pressure (real or perceived) in lots of teams might be towards the latter. For a 20% project you have to find your own group of peers to get recognition for it.
But, you can start a 20% project, impress people with your prototype, and get a real team started on that.