The problem I have found is not early optimization, but knowing what to optimize early and what can wait. Everything in a system can be optimized, but there obviously isn't the time to do this.

I work in Android, so optimizing I look at first is bitmap loading, backgrounding tasks, and sqlite queries. Usually, this is where 90% of the performance benefit comes from.