I have a hard time changing functionality of the alt key between alt and meta. I Terminal.app I just go to preferences and it is there. Here it takes a little more..

I'm not aware of any downsides, and I've been using iTerm2 for quite a while (a year?... haven't kept track of time).

iTerm 2's visor mode was still a bit wonky so I keep Terminal.app around just for visor. But that was when I was using the alpha and beta versions.

I've been using iTerm as my main vim screen for over a year now. iTerm 2 alpha chugged right along for this.