OpenGL can introduce a lot of overhead, especially when moving data back and forth between CPU and GPU. As just one example, OpenGL gives you no control over texture swizzling, which can be a serious performance issue during texture upload.
That's sort of the fault of the hardware vendors though for not proposing any viable extensions that expose texture swizzling. Why would it get any better with a new API?