<ctrl-v> essentially means "insert the next key I press verbatim". So, like in Vim, <esc> would switch modes, unless you press <ctrl-v> first. Helpful if you're trying to insert a control character into a file, or echo a literal <ctrl-c> etc.
Whichever facility receives and processes the inbound keystrokes. In simple software that uses the terminal in canonical mode, the line discipline (usually in the kernel) handles the batching up of bytes into lines before passing them onto the software, and thus provides the line editing. In more complex software that puts the terminal into raw mode, it's then handled in that software; e.g., by libedit or readline or some bespoke terminal handling.
