Honestly, I don't much notice emacs being slow, esp. given how much it's doing for me under the hood.

One key thing is to always always always use emacs in dæmon mode, rather than starting up a new emacs for each file (as would be done with vi). With an always-running emacs instance, start-up time is no longer a factor, and the editor itself is plenty fast enough once it's running.