So I've just started getting used to using the same keybindings just in a different spot. So far it's not too bad. Like anything, you get used to it.
Other than that, I did not configure any other vim keys.
Overall, I love colemak and couldn't imagine going back to qwerty. I've internalised it and find it natural to type with and I'm still often amazed at how much I can type with the home row and I love how common key combos just roll off the fingers (ie they use keys that are right next to each other or otherwise really really easy to type in combination).
I dunno if I'm really much faster. Probably not, but my accuracy has improved and I find it much more comfortable for long periods of typing.
hjkl is awkward: only h stays the same, j moves down but is on the upper row, k moves up but is on the lower row, l is on the upper row (though you still have space)
You can't use 'jj' for entering normal mode from insert mode anymore. 'nn' (what is at the same position as qwerty 'jj') is quite common when typing stuff, similar for all other keys you have your fingers on (maybe not 'ii'). So I settled for 'hh'
The other stuff isn't really an issue in vim.
I don't have problems with anything else really. Most standard gui shortcuts remain unchanged