Yes; it's a readline command called quoted-insert and is by default bound to ctrl-V in both modes.

    $ set -o vi; bind -q quoted-insert
    quoted-insert can be invoked via "\C-v".