I just had a quick look at Dvorak, and then picked Colemak instead because of more similar shortcuts. — It took weeks or months to get up to speed in Colemak. But it was a bit fun, fortunately. (How long did it take, with Dvorak?)
When the typing speed difference between Dvorak and Colemak is so small (it is?), I would say to myself that any of them is better than good enough and be happy :- )
There's also tools like tmux and fzf / skim, don't know if you're using them already; if not, I'd guess they'd save more time than learning Colemak.
> especially because vi keybindings are quite important
Oh I have nice Vi keybindings in Colemak — I use Vi always: VSCode, IntelliJ etc.
On the keyboard, I've mapped the N physical key to down, H to up, and left is Backspace (and Y which I never use), and right is Space and U. So I don't use the HJKL keys for navigation (well, H but it means Up for me).
This leaves the J K L keys available to do their usual things, in Vi — they map to N E I in Colemak (apparently you've noticed this :- )) which I use all the time.
> Is the big improvement that it leaves {} in their original place
I don't think so. I'd say, it's probably the shortcuts, and that many shortcuts continue working with the left hand only (e.g. using the mouse to select text, then the left hand to CTRL+C +V copy-paste).