Things like this typically require a monolithic kernel.

NetBSD's rump kernel drivers might be portable to userspace