Ctrl + [ instead of Esc gets one back into normal mode, with no delay
Takes a while getting used to it (I still haven't)
Try to run `xxd` and then press Ctrl-V and either Ctrl+[ or Esc, then Enter and Ctrl+D. Typical terminal setups will show the same codes both times (1b0a for esc and \n).
Alt+Space (or Alt+J Alt+K Alt+B to go up, down, left one word from current position after returning to normal mode) is what I often do which should work universally.