Look at the scenario from the customer's standpoint. You bought programs X, Y and Z. You then upgraded to Windows XP. Your computer now crashes randomly, and program Z doesn't work at all. You're going to tell your friends, "Don't upgrade to Windows XP. It crashes randomly, and it's not compatible with program Z." Are you going to debug your system to determine that program X is causing the crashes, and that program Z doesn't work because it is using undocumented window messages? Of course not. You're going to return the Windows XP box for a refund. (You bought programs X, Y, and Z some months ago. The 30-day return policy no longer applies to them. The only thing you can return is Windows XP.)
...
This is just the tip of the iceberg with respect to application compatibility. I could probably write for months solely about bad things apps do and what we had to do to get them to work again (often in spite of themselves). Which is why I get particularly furious when people accuse Microsoft of maliciously breaking applications during OS upgrades. If any application failed to run on Windows 95, I took it as a personal failure. I spent many sleepless nights fixing bugs in third-party programs just so they could keep running on Windows 95. (Games were the worst. Often the game vendor didn't even care that their program didn't run on Windows 95!)
Can't find it now but someone posted a Linux mailing list where Torvalds ripped a Dev for breaking something that impacted a user. The dev responded something like the parent: It's a 3rd party app/not Linux responsibility.
The OS has to be stable and the users experience is Paramount. Even a technical crowd (based on the thread) is 50/50 split on this being a Microsoft issue. A non-technical users response: "this software is shit, these idiots broke X in [this version] Microsoft sucks"
This would be perfect to re-add Win16 support to W10, for example. Try to double-click a 16-bit app? It opens with its handler, "Windows 3.1 Hyper-V Association Background Launcher". As long as no W3.1 apps are running, it unloads, and the rest of the OS can be completely free of 3.1-era cruft.
You could deprecate features from your latest OS at a breakneck pace if you really took this strategy to heart. WinRT really could have been just Metro/UWP + a "Desktop" that's actually a Windows 7 VM with a high-level paravirtualized shared filesystem.
(And it's not like Microsoft didn't do this themselves before; this is essentially how DOS applications ran under Windows 3.1, in a [hypervised!] DOS VM.)
---
EDIT: let me clarify the specific point I was getting at.
If you ship $OldOS as a VM within $NewOS, very tightly sandboxed, then you never have to update $OldOS again, nor do you have to keep the code from $OldOS around in $NewOS. The $OldOS VM is now a "static artifact"—something that will never change, but will continue to run old software perfectly.
So, if $OldOS gains more security flaws, you don't update $OldOS; you update the virtualization software to make the sandbox stricter.
And when you find that your $NewOS is now, itself, old—well, you just do the whole thing again, writing an emulator for $NewOS within $EvenNewerOS, and then your $OldOS programs run by launching a VM-in-a-VM. If you like, you can port the $OldOS emulator to run on $EvenNewerOS—but you could also just consider it legacy software of exactly the type your $NewOS emulator is built to run.
The point of this is to decrease maintenance burden, by freezing the previous OS, so that you never need to think of it again. This wasn't what was done with "XP Mode"; Windows XP continued to receive updates as a component of Windows 7. The point here would be to EOL the older OS, and then virtualize it.
You could even hold down the Commodore key while booting to get a mode compatible with the previous system.
Once a install base has been built up, compatibility (or lack there of) can make or break you.
[0] : https://en.wikipedia.org/wiki/Mac_68k_emulator
[1] : https://en.wikipedia.org/wiki/Rosetta_(software)
[2] : https://en.wikipedia.org/wiki/List_of_macOS_components#Class...
My point was less about compatibility, and more about security. If you took this approach, old software would still expose the new OS it was virtualized atop of, to its old security flaws. So you'd still have to move away from old software, eventually, for security reasons.
But with proper VM isolation (and checkpointing, and a few other things), you could still have programs like IE6 around today, serving in its last purpose as the '90s-era Intranet web-services equivalent to the '80s 3270 mainframe terminal clients.
Most any modern VM software can offer this level of isolation, but most have "host integrations" that turn them back into security holes. To be able to ship a static, EOLed copy of XP that could run IE6 indefinitely, without that causing problems, your compatibility layer would need to be a lot less like "Windows XP Mode" or Rosetta, and a lot more like DOSBox or qemu.
There is one case where this already happens: many pieces of IBM z/OS software have been wrapped in layers of hard-isolated emulators for decades now, such that programs from the 70s are able to continue running without recompilation, atop a stack of VMs, where many of the intermediary pieces of VM software have themselves long been EOLed.
https://technet.microsoft.com/en-us/library/ee681616(v=ws.10...
While the virtual environment integrated somewhat with the base (eg audio, printers, some networking shares and hard drives), it still is a separate window running a separate virtualized OS. I'm not sure I would call that too tightly integrated.
Nonetheless I personally found it handy nonetheless to run the occasional very old 16-bit application.
I think Raymond Chen discussed this on this blog at one point. The answer is that users want a single integrated environment for all their apps and not a bunch of isolated sandboxes. They want to drag and drop, copy and paste, share files, inter-application communication, etc. They also want a single look and feel.
The other reason is that creating a new platform from scratch is a good way to lose features and alienate developers. The core of Windows NT doesn't look anything like Windows 3.1 but for developers the API they present is similar enough to get them on board.
MS tried to rewrite and get compatibility by using these sorts of features. It was looked at by the PMs and dismissed because it was completely broken.
Turns out most hacks are in the software for a reason. They actually solve a problem that needs solving. Who knew ?
https://www.microsoft.com/en-us/research/project/drawbridge/
There is no reason to ditch the backwards compatibility in my opinion. All components can be kept functional alongside each other (like how UWA can function next to Win32) with some extra effort. I love the fact that there's a host of applications that target Win32 that I've been using since windows 2000 and still work on windows 10.