The exception being obviously a team approaching someone else's codebase -- including that of their predecessor, if they can factor in for Conway's Law -- to re-factor it.
But the same person or persons announcing re-factoring? I always try to walk away from those discussions, knowing very well they're just going to build a better mouse trap. For themselves.
Don't get me wrong, iteration of your own then-brain's product is all well and good, but it takes _more_ to escape the carousel. It takes sitting down and noting down primary factors driving poor architecture and taking a long hard look in the mirror. Not everything is subjective or equivalent, as much as many a developer would like to believe. It's very attractive to stick to "as long as we're careful and diligent, even sub-optimal design can be implemented well". No, it won't be -- this one is a poster-child exception to the rule if there ever was one -- your _design_ is the root and from it and it alone springs the tree that you'll need to accept or cut down, and trimming it only does so much.