It can, but Linux does not have a stable driver ABI. Whoever wrote the out-of-tree drivers would have to constantly update them whenever there was a breaking change to the kernel, which I understand is relatively common.
I think nvidia uses out-of-tree drivers that are dynamically loaded - does that mean that nvidia drivers are tied to specific kernel versions?
RE: nvidia - yes
Closed source modules like nvidia frequently have a kernel-independent proprietary piece and a kernel-specific (open source) ABI piece. Whenever you upgrade your kernel, DKMS will re-build the kernel-specific shim and re-link the proprietary blob. The result is a .ko tailor made for your running kernel, even though most of the code is in a kernel-independent blob.
[1] https://www.phoronix.com/news/NVIDIA-Exploring-Upstream-KMD