In the first case, you have a system that mostly does what you want, but it has architectural issues that make it hard to change. You plan a rewrite. You get buy in from management. But the problem is that the business still needs those changes. If your rewrite takes more than a few months, then you're still going to have to do the changes.
Over time, you end up with 2 groups: legacy and greenfield. The legacy group is usually made up of crufty old salts that don't care what they work on. They just grind it out. The greenfield group contains usually younger people who what to do something new. They want to "get it right" this time. They care deeply about what they work on and so there is usually a fair amount of conflict on the team about how to approach just about anything.
The legacy team grinds along, slowly solving business problems and the greenfield team provides no business value until they are finished (while constantly promising that it will all be amazing as soon as they are done). But the business types see only that the legacy product is doing what they want and that the team is quietly plugging away. The greenfield product does not do what they want and the team seems to be doing a lot of excited talking, but it still isn't finished yet.
So they say, "Wouldn't it be better to move the greenfield team back on the legacy product so that we can get things done faster? After all, they said they couldn't move forward with the old architecture, but it's moving forward just fine. And with more resources it will go along a lot faster. Besides we have these business emergencies that we have to deal with, so let's just put the greenfield project on hold for a while until we can sort out exactly what's best". And the greenfield project is effectively buried.
Now, it doesn't have to turn out this way, but powerful forces point you in that direction and you have to be very careful not to have it happen to you.
The other main problem is that when people think about doing a rewrite the requirements document usually looks like "Do exactly the same thing as the last system". But the problem is nobody actually knows what the last system does precisely while simultaneously they all think they know precisely how the old system works.
You get a lot of push back from the business when you start gathering requirements because the only thing they really want to say is "Just do it like the old system". Only, after months and months of development you end up realising that there were a lot of corner cases that you missed in the rewrite. Oh... and it's not possible in the new architecture to do that without jumping through a lot of hoops.
So, incremental rewrites are best, but only if you can get the business to actually use the greenfield project. Normally they will completely ignore it because their paycheque and their promotion and their happiness depends on being able to do their job at least as well as they could with the old system. And the new system doesn't do everything that it needs to do (because we've released incrementally). What's more the developers keep asking stupid questions like "What do you want it to do" and the business people are responding with "Why can't you listen to me? I've told you one hundred times to do it the same as the old system!"
And so the new system is shunned by the business. It becomes radio active. Unless some upper management type sends a mandate down to force the business to use the new system, nobody will touch it. The upper management, in the meantime are wondering, "Why are we doing a rewrite again? The old system seems to do what we want, while everyone is complaining about the old system".
Yep, it can be done, but it requires considerably amount of help from upper management. You should avoid it at all costs unless you are sure upper management is supporting you all the way.
But even when you are successful, it may just be the beginning of the end. You've managed to stave off cancellation. You've managed to keep user engagement and work through the requirements so that the new system is equivalent. But it turns out that there is some small detail or decision that makes the new product non-viable.
For examples of products I've been personally involved with: Word Perfect 5 was written in assembly code and was not based on Windows. Word Perfect 6 was rewritten in C++ and was based on Windows. The team that did the rewrite were very proud of their work. But they through out all the keybindings in WP 5 (because GUI is way better!). The new rewrite was also very, very slow compared to WP 5. This was the beginning of the end for WP, even though one could definitely say that a rewrite was necessary. They just made the wrong choices.
Similarly, I once worked for Nortel and they had a telephone switch that they sold for $10 million a pop. It had 31 million lines of code (and stinky, stinky, stinky code at that). They realised that they could rewrite it in a fraction of that amount of code. They had 3000 developers working on the rewrite and after a few years they succeeded. Only... it didn't work with all of the business equipment that the old switch worked on. And it turned out that nobody wanted it unless it worked with that equipment. And.. the new architecture was not conducive to making it work with the business equipment. In the end, they gave a few switches to China before they abandoned it.
Rewrites are hard even when you've made the right choice to do it.