Found the vim user! The vast, _vast_ majority of Unix command-line or text-based programs do not use the Esc key because they were not written with a particular terminal or keyboard layout in mind. (Unlike vim.)
But for whatever it's worth, I use vim every day and I notice no 1 second delay when switching modes with the Esc key, so perhaps this is something you can configure in your terminal.
> I notice no 1 second delay when switching modes with the Esc key, so perhaps
> this is something you can configure in your terminal.
Try to run `xxd` and then press Ctrl-V and either Ctrl+[ or Esc, then Enter and Ctrl+D. Typical terminal setups will show the same codes both times (1b0a for esc and \n).
Alt+Space (or Alt+J Alt+K Alt+B to go up, down, left one word from current position after returning to normal mode) is what I often do which should work universally.