It's a mixed bag. One thing refactoring is good for is getting past friction introduced by people who don't like a good idea. Early in my career I got a lot of "that will take too long" for features in the High Cost, High Value quadrant of the feature graph. A couple months of picking away at the accidental complexity and you can come back with a <6 week estimate instead of 4 months, and your proposal gets more consideration. I'm not saying 4 months is a reasonable estimate, I get why that causes concern, but refactoring can be so much more effective when you have long-term goals in mind. When you have a list of 'target of opportunity' features. When you can romance an epic into the code, instead of forcing it in.
I still get that pushback of course, but I'm more used to it and also sometimes don't ask questions I don't want the answer to. It's all subterfuge and puppet mastery. If I wanted to be an accountant or a lobbyist I wouldn't have enrolled in a CS degree at a semi-respectable institution.