the right people is definitely the key! not all people are made to work remotely... just a fact of life.

being more of a front-end guy in web development I would LOVE to be able to brainstorm on the whiteboard (or paper!) with other team members. For me, this is the biggest drawback since I feel like the digital tools (balsamiq mockups and others) just don't quite accomplish the same thing.