Desktop Linux works very nicely on smartphones actually provided all the drivers are there. I lived with a PinePhone running FVWM on Xorg for a couple years and if the hardware didn't crumble away I'd still be using it today.

No need to "build a complete from-scratch alternative OS" when that was already done 30 years ago.