A similar situation applies to 3D graphics for game engine programming. You'd think if a piece of hardware can run a AAA game with impressive graphical effects at 60 FPS, it should be able to run your simple indie game engine with simple graphics for proportionally less development effort. But, no, it turns out unlike CPU's where we got so many "speed ups for free", most of the things that make the AAA game able run so fast with so many effects are bespoke, opt-in features.
(The following is based on my experience learning from the Irrlicht engine. YMMV) You find a random N-year-old basic OpenGL tutorial and it doesn't run N/1.5 iterations of Moore's law faster now, it still runs inexplicably slowly (compared to what you know that hardware theoretically should be able to do with modern games). You have to figure out, oh it's not using GPU memory for the meshes, how do I tell the driver to do that? Fixed that, but wait I'm using immediate mode, retained mode would be faster. Wait is retained mode going away, what's replacing it? So I gotta learn shaders now too?
The problem is the APIs are terrible and the drivers are buggy. The levels of abstraction keep being chosen all wrong - too low-level for the library or drivers to give user code "speed ups for free" and too high-level for a power user of the API to be able to be able to tell directly what's happening and profile what's making things slow.
A better model would be to break the high-level and low-level into two layers, like the Clang / LLVM compiler. The high level API defined by an organization like an OpenGL working group and they could provide a reference implementation front-end (corresponding to Clang or another compiler front-end in this analogy). The hardware vendors would just provide the low-level implementation (corresponding to LLVM).