I'd been using fish for a year or two before switching to zsh too.
The main reason for me to give it up is because I could not get it to work smoothly with rvm.
I've gotten quite used to zsh by now, and I have to say, I don't feel much for having another go at fish just to run into the same limitations again.