To me, this thinking seems like a sure way to get an unmaintanable mess. Now, just polishing a codebase forever without adding any features obviously isn't a valuable way to spend time. But the difference between changing one value in one configuration struct and it having no unforseen side effects and having to go trough multiple classes and reasoning about possible thread races etc. Can make a huge difference in feature adding time. However, how do we measure this?
If all features were equal, like widget production, we'd definitely be able to prove or disprove the productivity boost, but I haven't ever worked in a context where this is the case. There's no guarantee that feature implementation time will go down, maybe it did just not go up as much as it would otherwise, maybe people manage to make more useful features in the future, or at least not as much less useful as they would have been (we would expect utility/feature to go down as a system matures)
When I worked in a two-person team it was easier, of course, the time I spent on improving the code base I soon got back when I had to implement new functionality on a short deadline, this reflected well on me in performance reviews. The problem in a larger team is that it might be someone else who is able to implement a feature quickly due to my improvements.
My guess is that we should be lucky that there are guys like the OP who naively spend time on "low impact" code improvements even though they might not reap the rewards. Without them we would only have people doing the bare minimum to implement new features with a big ball of spaghetti as a result, where projects are abandoned just because no one is willing to maintain them any more. (does this sound like some company we know?)
I guess what I'm describing is that we might end up with a prisoners dilemma situation, where optimising for short term measurability represents defection.