It doesn't need to be inefficient, it depends on the implementation. Immediate mode is describing the API and usage, you can implement it such that it's retained mode behind the scenes, or only redrawing when necessary, etc.

I've written an immediate mode GUI for my projects and it's fine for performance. I've written a music player for instance which uses a negligible amount of CPU, it uses slightly more resources than Clementine, my previous music player.