I've been an iOS user since shortly after the first iPhone came out. At a previous company, I had to do extensive cross-browser mobile testing on iPhone/iPad, several Android phones, and the Xoom (Android tablet). Every time I picked up one of the Android devices to do something, I had to "think". Its annoying as hell.
And by think, I mean that when I went to turn on the Xoom the power button wasn't in an obvious place (its on the back). When I was in either a phone or tablet, there was a different way to access the wifi settings on each device. Getting back to the home screen was annoying. Why do I have to page back through 3 previously opened apps when I just want to go to the homescreen. (To be fair, there is a separate button to go to the homescreen directly. But why are there 4 buttons at all?)
Clearly, my experiences have been shaped by my years of experience using iOS. To that extent I've even been using iOS 5 beta for the last few weeks on my phone. While it clearly has some rough patches, the new features are well thought out and polished. (Sorry, not going to talk about specific features because I don't know if they are uncovered by the NDA yet.)
So when I say that Android makes me have to stop to think, I mean that any time I have to use one it has these annoying gotchas that require mental effort to examine and bypass. I would be happy to say that it was just me, but a lot of my friends have switched from iOS to Android phones over the last year and I hear a lot of the same complaints from them too. They aren't technical illiterates by any mean, but they don't spend their days immersed in this stuff like I do. One runs a print shop, another is an audio engineer. One even edits video at a porn company.
I think a good way to summarize the difference, and the problems, is in a recent ad proclaiming "your wife will love the Tegra2 chipset". iOS devices inspire people. Android makes them think.