I believe this is related to our brains being hard-wired to prioritize negative phenomenon and stimuli. Most of those projects shipped and were successful. Users did like using them. However the developers often only saw the bug reports and the endless requests for new features. They never got a chance to talk to users and see how the software was being used and improving the lives of real people. They only see the barb-wire and duct tape.
I had a young developer ask me once about refactoring: how do you get time to refactor your code?
Never. You will never find a person willing to hand you a paycheck to take a month to change absolutely nothing. It gives the business no advantage. But the bugs! the young developer says. To which I say, dear summer child, risk tolerance. The business is willing to take the chance that 1-2% of the time a user may get frustrated, encounter a bug, and file a report or complaint. If the company has to pay you to fix it later that's fine. They can tolerate the cost.
You refactor your code as you go. After you write the code and your tests pass and you think you're done -- refactor first. Abstract away superfluous variables with function application, remove duplicated code, break up long branches with decision tables, etc. Fix that code up before you check it in.
As you get better your refactor steps will be much shorter as you internalize the data structures and patterns that lead you to clean, concise code to begin with.
When you write software commercially you need management that understands how to balance capital, people, and technology. And you need engineers who are pragmatic about their work and the real outcomes: the systems and the people who use them.