Problem is even if your manager's manager understands the value of it, his manager's manager doesn't, so time spent on it is seen as wasteful or "doing nothing" versus duct-taping things together for some poor sap to have to refactor ("for free") under pressure while implementing more features non-technical PMs or managers want ASAP.

You have to be skilled at office politics or make sacrifices to keep code quality high. Usually not worth it. You get yelled at less if you externalize all the costs to future suckers, instead of trying to reduce those costs upfront.