Closed source firmware is one thing that actually runs outside the Linux system... but there's also the user space libraries that are needed to interact with the drivers (eg libgl etc... or the vendor partition in most Android phones)
The stability layer also doesn't actually let you seemlessly update the kernel. Those userspace binaries are coupled to specific kernel releases, and it requires work on the vendors part to facilitate new kernel version upgrades. Maybe being upstream will force them to actually take backwards comparability with older userspace binaries more seriously though.