I had a really hard time getting into Vim until I found out everyone I knew who was using it had mapped something else to escape. Once I did the same, it was much easier to get started.
It's also useful to use Bash in vim-mode and also remap the esc-key, e.g.
set editing-mode vi
set keymap vi
$if mode=vi
set keymap vi-insert
"jj": vi-movement-mode
$endif