Have you tried the neovim plugin instead of the regular vim one? I find they have different trade offs, but the neovim one is actually using neovim inside a vscode window and requires a neovim install.

Particularly, I’ve found the undo functionality to be far better with the neovim one, it never screws up. Holding alt and moving a line up and down can sometimes accidentally move your cursor one line higher or lower though.

Biggest personal issue with VS codes vim mode is that it only works in the text editor. The beauty of vim/neovim is that the keybindings work everywhere in the UI, not just the text edit window.

I basically had to disable entire VS Code. Didn't remove it yet and keep it just in case I need to remember how crappy it's. It has some advantages over neovim though, but they don't outhweigh neovim.