Vertex arrays have been part of OpenGL since 1.1, released in 1995. VBOs since 2.1 (with extensions available earlier). glBegin/glEnd aka "immediate mode" has been "legacy" for a LONG time and were even removed in the core profiles with OpenGL 3.2.
The biggest problem with OpenGL for is/was the context model that made multi-threading almost impossible. And yes, it being stateful is a big part of it. Plenty advances and low level functionality was available in OpenGL before vulkan was released.
Vulkan also reduces overhead since its abstractions better match today's hardware and it allows excplicitly moving more calculations up front that had to be done just in time with OpenGL.
Also, it's not like Khronos (a standards body made up of industry participants) "refused to advance" but rather that different interests had different ideas - e.g. some care more about backwards compat - so that they can tell their clients that their legacy software runs on the latest OpenGL - than getting the last edge on performance so having a separate 100% performance-oriented API was probably an eventuality anyway.
Also, remember that Microsoft and Apple are/were part of Khronos too and giving these two comanies' incentives and past behaviors it would not be entirely surprising if they did their part to hold back the standard.