I generally agree with this opinion and would love to see a proper well documented low-level API for working with GPU. But it would probably result in different "GPU ISAs" for different vendors and maybe even for different GPU generations from one vendor. The bloated firmwares and drivers operating on a higher abstraction level allow to hide a lot of internal implementation details from end users.

In such world most of software would still probably use something like Vulkan/DX/WebGPU to abstract over such ISAs, like we use today Java/JavaScript/Python to "abstract" over CPU ISA. And we also likely to have an NVIDIA monopoly similar to x86.