Disclosure: I very briefly worked for a competitor.
A) Drivers reveal a surprising amount of "secret sauce" for a GPU. The "public-facing" interface to a GPU is something like HLSL or GLSL, but the real interface is some proprietary ISA. The manufacturers are slowly moving in this area, but the culture of the ISA being proprietary remains. That leads to things like implementation details getting inserted into ISA docs, or code comments, or driver architecture designs, because they're for internal use only anyway. This philosophy is slowly changing, but it's still the case quite a bit.
B) There's much less difference than you would expect between a $500 gaming GPU and a $2,000 workstation GPU. It's totally plausible that you are one FOSS driver away from a much more expensive GPU. That is a problem for everybody, including the gamers who want to keep buying $500 GPUs.
C) There is a VERY complicated balance of power between OEMs (who control the purse), Intel (who sometimes competes and sometimes complements the video card manufacturers) and Microsoft (for better or for worse, a de facto standards body) and increasingly, new kinds of competitors like PowerVR. So this is just a hypothetical, but one way it could go down is like this: NVidia decides to fully embrace FOSS. Microsoft gets pissy and tries to write them out of the standard. NVidia forced to lower prices to get PC makers to buy. Apple swoops in and places a huge order of the now-cheap parts. Intel gets pissy that Apple is shipping too many Nvidia GPUs per Intel GPU, raises Apple's CPU pricing to cover the loss. The price of Macs goes up driving some customers to iPads. PowerVR's royalty fees double, and since they don't have to manage any inventory, they are increasingly more attractive to investors than traditional manufacturers like NVidia. NVidia's stock price falls, and some C-level forgoes a vacation in Europe.