End result should be a forward looking Windows or MacOS or Linux equivalent.
I want to know your estimates of 1. Costs 2. People needed 3. If you think there will ever be another major OS gaining popularity.
The reason are a few words in your problem statement: "top quality" and "Windows or MacOS or Linux" equivalent. Successful products don't get built this way, where you shoot from the beginning to build something excellent and basically copy what counts as "excellent" in the market right now. The problem is that you're chasing after the current state of the art, but in the time it takes you to catch up to it, the state of the art will have moved forwards. And so on, in a Zeno's paradox situation. Eventually all your engineers will get disheartened and leave, and you write off the project as a failure.
"Nobody should start to undertake a large project. You start with a small trivial project, and you should never expect it to get large. If you do, you'll just overdesign and generally think it is more important than it likely is at that stage. Or worse, you might be scared away by the sheer size of the work you envision. So start small, and think about the details. Don't think about some big picture and fancy design. If it doesn't solve some fairly immediate need, it's almost certainly over-designed. And don't expect people to jump in and help you. That's not how these things work. You need to get something half-way useful first, and then others will say "hey, that almost works for me", and they'll get involved in the project." -- Linus Torvalds
If you want to write a new OS that has a chance of market success, you should pick one dimension that current OSes do a shitty job at. Security might be a good choice, or making it work on whatever embedded processors are in vogue with hardware startups now, or maybe give it 10x better performance when running Node.js application servers. And then let yourself suck in every other dimension. If you're building a secure OS, make sure that everything from the drivers to filesystem (if it even has one) to applications are ironclad, but let the UI suck or even omit one. If you're targeting hardware startups, make sure it works for their needs, but forget about UI, or networking other than Bluetooth, or running on x86 hardware. If you're making the fastest Node.js server around, optimize for performance, make sure it can run stock apps, but maybe the only thing it'd run on is AWS and it won't have a GUI or OS scheduler.
I guess I find it a bit disheartening there are no game changer desktop OSes having had even moderate success for the past few years. I like variety.
Seriously though, this might answer one of your questions https://www.reddit.com/r/compsci/comments/qfsob/rob_pike_on_...