I think you're retrofitting the common clichés about proprietary APIs into your memories. First (circa 1995) 3D accelerators were all different and incompatible with each other: NV1 was using Saturn hardware (and only got used in some Saturn ports), 3D Rage had its own API (creatively named CIF, from “C interface”), S3 Virge had its own API (S3D), Vérité had its own API (RRedline), Voodoo had its own API (Glide). Interoperability wasn't even discussed; when (some) games added support for (some of) those, it usually was a completely rewritten executable, i. e. port to that graphical architecture. The main difference between those accelerators was that the former targeted fillrates of top consoles of previous years, still hanging on early nineties 3D vision (a couple of low poly models; fake perspective, like in racing games; mix of 3D and 2D sprites and backgrounds; z-buffer? you can simply draw polygons in proper order!), while the latter competed with professional accelerators (in fillrate, not in shading capabilities), and made a breakthrough.
(By today's standards, those famous early 3D games were laggy crap, both hardware- and software-rendered, on many/most period-correct computers. It's rarely said that simple things like texture interpolation (computationally way too much for a CPU) were new, and impressed people more than fps counters. Yes, the vaseline polygons were once the look of the Future!)
Then two companies leveraged their positions to change that course. First was Microsoft, which really needed everyone to start using system-wide multimedia APIs instead of dozens of unmanageable DOS era bypasses offered by this or that hardware manufacturer. So Direct3D was introduced, and then quickly remade in a numbers of later versions to match the breakneck progress in hardware features (or directly wrap around them, if you wish). On one hand, it was a universal API, on the other, people would still write “ATi code” and “Nvidia code” years later.
The other was id Software, though not in a straightforward way. Quake I was ported to accelerated hardware (vQuake, glQuake). Partially because of vQuake experience, Quake II compartmentalized the renderer into a library, and only that library was supposed to be written with the help of hardware makers (or by them). Ironically, OpenGL support was more of a tech demo thing because id Software had the expensive professional hardware, and regular users obviously didn't, despite the calls to support that API in consumer 3D accelerators. Even more ironically, 3dfx decided to make a bare minimum OpenGL wrapper library for Glide to let already existing glQuake run on Voodoo instead of trying to make some deal on the porting. They called it MiniGL. Soon other manufacturers, game makers, and people trying to make this or that existing software work on consumer PCs followed the example, and the world saw a lot of custom hacked "opengl32.dll" semi-implementations tightly coupled to respective applications. Of course, if any of them happened to be copied into system directory, everything but the specific software running with specific hardware that library was supposed to support crashed or broke. Those who tried to write new programs by the book using these wrappers probably had some problems, too.
Then Microsoft was forced to act, even though it never really intended to support alternatives to DirectX family. It made OpenGL (standardized) an official system-wide API, but made the system library to be just a wrapper over complete OpenGL implementation provided by hardware manufacturer (if present). So it continues to this day, and making everything work properly has been AMD/Nvidia's problem, not Microsoft's. This is why people on Windows can still choose between OpenGL and Direct3D in game settings, and see something on screen using both. But when it all got started both were just thinner or thicker wrappers over what chip designers initially had in mind.