This might be an unrelated problem, but I find that when I copy a large amount of text and paste it into iTerm, it takes a long time to paste. This often happens when pasting into Vim, so it could be a vim issue. For example, copying the code of a Javascript library. When I do this in a text editor outside of iTerm, it's nearly instantaneous.