I am not a lawyer, and I have no real idea how likely said argument is to hold up in an actual court. It also seems unlikely that anyone would actually file a lawsuit about this sort of stuff. But if you want to be as careful as you can be, you won't go wrong if you use the GPL v2 on principle for a Linux device driver.
On this topic it is worth noting that Linus Torvalds has stated different seeming opinions at different times. In general as more and more of the Linux kernel has been made available to loadable modules, he's become less and less inclined towards the argument that they are not derivative works. See http://linuxmafia.com/faq/Kernel/proprietary-kernel-modules.... for a comparison of his views at two points in time. (I don't know his current views.)