I do not think that I would mind so much having to re-compile between kernel versions if it's just a matter of running "make" and copying the binaries to the target.