1) the iphones touch screen. I don't care that much about the smoothness of the scrolling, but I really care about the ability of the touch screen to accurately track my finger position. While the N1 often is /good enough/, sometimes it fails badly and recognizes taps miles away from my finger. This is really bad when trying to type.
2) this might be an issue of my specific device, but sometimes I have a really bad hissing noise in my headphones. I'm constantly listening to Podcasts and Audiobooks and I can't live with that. HTC has some knowledgebase entry about this one and recommends to periodically shut the device down, remove the battery while holding the power-off button, and then reversing the steps. This, of course, isn't something I'm willing to live with.
So in the end it's build-quality that kills an otherwise superior phone for me. Too bad.
I hope Google rev's the hardware and we get a N2 that fixes these issues. Seing all the trouble with manufacturers not updating the OSes, I'd rather stay with a "Google Experience" phone as that more or less guarantees updates, so no Desire or Droid or whatever for me.