>other operating systems have caught up, while NT was pretty close to its present design even in the early 90s
The one thing you will find almost as a theme with Linux design is that there is minimum of "future" it. That applies to most of the Linux user space as well. This is a good thing and a bad thing.
For example multi-gpu systems with multiple outputs were not taken into consideration when both the Kernel and Xorg parts were designed. They took the most common usecase of single GPU, probably made some assumptions around it and coded something that works for that usecase.
Microsoft, on the other hand (almost out of necessity) had to actually design the NT Kernel while making no assumptions - multi GPUs may not have existed when XP was released but none the less MS had to design for the eventual reality when some OEM will stuff 2 GPUs and will have to write drivers for it without Microsoft being able to change the XP kernel design or even implementation in any way. I am sure the same can be said about scalability - when massively multicore CPUs weren't around NT kernel was mostly up for task of supporting them, whereas Linux adopted with lot of scalability related work that followed after the fact. (Remember the early NT vs Linux benchmarks?)
Designing for future is bad because it comes with certain baggage and to an extent added upfront costs. It is good for future compatibility, code maturity and obviously saves rewrite efforts - you don't have to rip off all the code and start over and take care not to break existing software etc. There is less code churn in Windows world I am sure than there is in the Linux world where people are constantly adopting to new realities. This creates an environment ripe for new bugs, instability and only with the amount of volunteer effort the cost factor can be lesser than prohibitively expensive.