Intel wanted in on this space and didn't want AMD to be able to produce compatible parts so... enter EPIC [1]. The projections (as you point to) were wildly optimistic. Of course the whole thing was heavily delayed, produced in small volume, offered limited to no performance gain, required a ground up write of pretty much everything and was super expensive. What could go wrong?
What did AMD do? It just invented 64 bit extensions to x86 (ie x86-64). Was it ideal? No. You probably wouldn't design an instruction set and architecture this way if you were starting from nothing but you aren't starting from nothing. AMD released the wildly successful Opteron (and Athlon 64) and for 5-10 years completley ate Intel's lunch. Cheap parts, good performance, easy migration part, etc. Intel got to use these extensions by the same licensing agreement.
This coincided with the Gigahertz wars on the desktop front. From the Pentium for almost a decade Intel focused on Gigahertz and once thought they could run this up all the way to 10GHz+. We know how that went. Clock speed was a key marketing bullet point. AMD processors had higher IPC but that (at that time) was a harder sell.
About the time Opteron came out the Gigahertz wars were hitting the 3GHz barrier and Intel was stuck. At the same time their mobile parts used a completely different architecture. Thsi was Centrino with the Core Duo processors. These had much higher IPC (than Pentium lines) and more power efficiency. Some early enthusiasts constructed desktops from these mobile chips and the results were great.
Losing out to Opteron/Athlon and hitting the 3GHz barrier ultimately forced Intel to abandon their desktop architecture in favor of Centrino. Intel hung on as long as they could to milk some extra profit but eventually Core Duo was the future and this ultimately because the foundation for the Core processors we have today (but by now there have been many revisions). It might be more accurate to describe the Pentium years as the Netburst architecture years (IIRC).
But the lesson here is that a complete rewrite was an almost fatal mistake for Intel. It's almost always a mistake. You can't ignore what's there now and your desire to lock out competition doesn't necessarily translate to something customers actually want or need.
[1]: https://en.wikipedia.org/wiki/Explicitly_parallel_instructio...