Reaching parity with Linux's immense suite of device drivers is perhaps the single biggest hurdle.
That's one of the biggest reasons why alternative kernels either remain fringe or fail.
Initiatives such as NetBSD's Rump kernel but for Linux may provide a bridge to Linux's drivers but it's brittle. There's already LKL - the Linux Kernel Library project with similar aims but not much traction.