Things like having a texture ID, which you then bind to a particular target on a particular texture unit in order to use it make so little sense to someone learning it for the first time. On the CPU, I just pass the pointer to my image to a function to manipulate it. I don't have to put it into a special slot of a special structure in a special place in memory! It took me years to understand many of these things, and I see others struggling in the exact same way on Stack Overflow, for example. So sad.
IIRC in modern versions of D3D (10+? 11+?) they've expanded on this and just have the general concepts of views and buffers, so that you can treat textures and vertex buffers as if they share underlying properties, and shader code can manipulate them in similar ways. This is great for compute and GPU-accelerated processing and GPU feedback.
The challenges arise in validation and actual development. Validating that a driver works correctly is VERY difficult due to the complexity of the spec, and you can't really afford to have a huge test team with as much experience and knowledge as your driver development team. Even once you've validated and shipped your driver, you can't know how end-users are going to exercise it.
Then as a developer, not only do you have poor knowledge of the spec, but you have no knowledge of how each vendor interpreted the spec and whether or not their implementation matches their expectations.
As the surface area of OpenGL and the complexity of each entry point grow, this is only getting worse.