GPU drivers are not just simple pieces of code that are used to access hardware. They contain very fast implementations of extremely complex libraries, and optimizing compilers for complex programming languages. Just to give you an idea of the ballpark, the nVidia GPU drivers are a more complex system with more code than the entire Linux kernel.

As the performance of this system is absolutely critical, a large part of the value proposition of a GPU is its driver package. Since nVidia clearly leads the pack in the quality of their drivers, they have no intention of opening them up.