Exactly. There's such a great difference between Emacs with half a million of lines of Elisp loaded and a vanilla version that it's easier for me to just use Vim (it's everywhere anyway, but I can work with Vi too).
Using vanilla Emacs is like writing a GUI application with nothing but the primitive X (or GDI) operations. The potential is there and is visible, and it's sometimes the right thing to do, but normally you'd use GTK or QT, right? Same is true with Emacs: in its vanilla form, it's just an API for text editing with a bunch of outdated defaults. It's superb as a platform for writing text-editing related (and sometimes others) applications in Elisp. I think that Emacs is not supposed to be used in that state at all, even. Vim, on the other hand, is pretty much designed around one UI & text-editing philosophy, which makes it less extensible, but much more usable out of the box.
Edit: mg is an emacs-like editor that is even leaner than vi, has off course the same key bindings as emacs, and also shares the same openbsd roots as tmux. It is also the default editor in openbsd.
