That problem was noticed way back in 2004:
https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...
"There are two opposing forces inside Microsoft, which I will refer to, somewhat tongue-in-cheek, as The Raymond Chen Camp and The MSDN Magazine Camp .... The Raymond Chen Camp believes in making things easy for developers by making it easy to write once and run anywhere (well, on any Windows box). The MSDN Magazine Camp believes in making things easy for developers by giving them really powerful chunks of code which they can leverage, if they are willing to pay the price of incredibly complicated deployment and installation headaches, not to mention the huge learning curve. The Raymond Chen camp is all about consolidation. Please, don’t make things any worse, let’s just keep making what we already have still work. The MSDN Magazine Camp needs to keep churning out new gigantic pieces of technology that nobody can keep up with ... Inside Microsoft, the MSDN Magazine Camp has won the battle."
This telling of the situation is rather unfair on both sides, of course.
The actual reason behind what happened is straightforward - Microsoft had to abandon Win32 because it was a terrible API even by the standards of its time that had no hope of keeping up with competitors, but their intended replacement (.NET) was too big of a leap and didn't manage to fully take over. They were trying to compete with Java and mostly managed that, but it was too far above the metal to replace Win32 in every context. In particular the Windows team rejected it as a way to build and ship Windows itself, leading to severely dysfunctional internal politics and dramas.
Plus their whole strategy for the web was "try to kill it by making IE dominant and then defunding it, whilst pushing people back to the Windows API". They didn't try very hard to understand why devs were flocking to HTML despite its massive disadvantages compared to Win32. In fairness nor did Apple or Sun or anyone else, really.
So: Win32 was dying, and was designed in such a way that incremental progress wasn't going to fix anything. The API itself was just too badly flawed. But they ceded the battle of ideas to Java and then couldn't find a way to incrementally port existing Win32 apps, because C# was too different to C++, and the various .NET C++ dialects just weren't a compelling story for games, the MS Office team, the Adobe suite etc.
Roll forward a decade or so and we have a needle scratch moment - the Windows team finally decide that they need a proper replacement for Win32 that can actually be incrementally adopted. WinRT is their answer. It's sort of COM but evolved and with databases of metadata that allow the auto generation of high quality language bindings (including a standard C++ binding), along with a pile of new APIs, which can also be incrementally adopted inside Win32 apps. It's actually, dare I say it, not that bad. At least in terms of overall design. Unfortunately by the time they finally get this strategy right the world has long since forgotten about Windows programming, with experts in it confined mostly to the video games industry, the Chrome team, and random old farts like me who occasionally still need to write a Windows app for whatever reason. Currently for me that's about making distributing apps easy - Conveyor [1] uses a small custom app that mixes Win32 and WinRT in the same program, a small EXE that drives the download/install. And it uses the WinRT packaging scheme (MSIX) which is Linux-like and has a whole bunch of advantages over other installation mechanisms.
Unfortunately for Microsoft whilst WinRT does get the foundations right, the implementations of the new APIs at the UI level (WinUI) aren't really being used by anyone outside the Windows team themselves, and it's a whole new UI toolkit written from scratch, not based on an evolved Win32 codebase. So a lot of little details are wrong. Try triple-clicking to select text in the Settings app for an example.
I guess the core question is how things could have been different. The WinRT strategy could have worked or at least resulted in slower decline, if it had been started around the Win 2000 era and if .NET had been a properly integrated feature of Windows. In other words if they'd retained strong technical leadership over the platform as a whole. They'd still have lost out to the web and Linux though, because even now their platform doesn't really grok why web apps or Linux are popular.
[1] https://conveyor.hydraulic.dev/