(I'm pretty much unbothered by all the complexity, mostly because I'm not particularly into deep customization and tinkering, and I'm fine with unused features sitting around everywhere, so anything I say might be totally irrelevant!)
It seems to me like the best place to start is(as usual) with concrete use cases on what you'd actually want to build.
If it's simple enough, perhaps, this entire OS, text editor, compiler, and browser could be 1000 lines of FORTH and a 100 line interpreter. Simplicity fans and full stack DIYers seem to adore FORTH!
If you haven't read the Moore 500 line CAD story(https://retrocomputing.stackexchange.com/questions/25506/did...) do it.
Definitely not an app I'd want to use, and I am convinced that the part about it being better than commercial CAD is nonsense but it's impressive.
(For reference, I suspect that what actually happened was the simulation used a much simpler model than the commercial CAD, not taking into account worst-case effects which happened to not show up in the real chip.
His intuition might be good enough to design without the more accurate sim, but I would still trust the commercial CAD and not really want to produce a product that the pro simulator didn't like).
I'd suggest reading up on TempleOS, and also on Dean Roddey's much lesser known CIDLib(https://www.reddit.com/r/cpp/comments/hti9zs/open_sourcing_m...) for examples of mega scale DIY stuff.
I'm assuming your big issues would be:
Initial setup time. I don't see how you could DIY any of this without lots of intense heads down coding. To do it in reasonable time might require full time focus, which could be lonely and expensive.
Complexity sells better. As an end user, I'm willing to make an effort for performance and features, but not for simplicity itself. As a FOSS dev, I mostly work on the kinds of things I'd like to use myself, unless it's something that's going to really benefit the world, or something by a specific person I want to work with.
It's very possible to get others onboard, but I suspect anyone who is that into minimal low level OSes probably has their own vision of how things should be. You'll might need to do things in a way that is compatible with very different approaches.
You would not have autocomplete, inline hints, linting, or any of that other Language Server Protocol stuff unless you wrote it yourself.
Text editors, as I understand it, are notoriously difficult. I have not written either and don't want to, but it almost seems like it could be as hard as the compiler!
You will not have any optimizations unless you write them. Your GUI library might not be able to do certain things all that fast without GPU help.
Security could be an issue. It's an issue in the best languages with the best coders, so it's probably gonna be an issue for everyone.
You won't be able to use it professionally without angering people unless it takes off. I sure don't want to rely on languages maintained by one person.
You won't have any cross platform support if you're doing stuff at the machine code level, unless you also do that yourself.
If you want to do zero dependencies, that's a LOT of very diverse areas of algorithm to learn. Quicksort, graphics drawing, compression, cryptography...
Now, if I personally wanted to write everything from scratch, I just would simply not be able to, because... I'd just want to make VS code and a language like Dart or Rust and the Flutter toolkit that runs on all platforms... and that would be very hard!
Personally, I do in fact agree with the masses that this level of wheel reinvention is usually a waste of time, but I fully acknowledge I might be biased or Just Not Get It.