Perhaps I'm just a 'sensitive' soul, but I really do notice these minor differences in performance. If there is ever a slight slow down in the editor then it takes my focus off my work and onto the editor.
I 'feel it', rather than being able to put it down into specific testable specifics.
I use ST3 mostly and I instantly notice if switching tabs is taking fractionally longer or if undos/redos or scrolling is taking longer.
The only reason I don't switch to Vim is that ST3 is much more user friendly. So it means that I have to think too much about the commands in Vim which slows down my thinking - which is the equivalent slow down of having unresponsive editors.