Most of the devices I've seen are great by themselves. If I played with one in a store I'd want it. But as a group, when they're what my company has to target for distribution, things could get really ugly.
Practically; I was an early adopter for the HTC Magic. Despite that, all the apps designed for the G1 worked fine. I could even use Emacs in ConnectBot (ssh client) with the soft keyboard. Now that's abstraction!
When it comes to other features of the phone, e.g. camera or accelerometer, like one of the posters said- you'll have to do progressive enhancement. Intelligent web developers already know how to do this. You feature test, then you enable a particular feature. Its not all that difficult.
It works, but it means giving up a lot of control, and the results tend to show it.