I use these settings in my .inputrc. The get me a few more commands and even the ability to escape using jj.
set completion-ignore-case On
set bell-style none
set editing-mode vi
$if mode=vi
set keymap vi-command
"gg": beginning-of-history
"G": end-of-history
set keymap vi-insert
"jj": vi-movement-mode
"\C-p": history-search-backward