I'm in the same boat. And the number one annoyance is that it's not keystroke compatible. I have the standard editting keys in my fingers for loading files (C-x C-f C-a C-k pathprefix TAB path TAB etc..). That stuff breaks with ido and I get stalled and have to compose myself and figure out what I'm doing.
Wouldn't it have been possible to preserve both the classic editability of the path string and have fuzzy matching?