I am a happy Fish user, too. The first downside that comes to my head is that if you want to share a script you wrote that you have to translate it to bash first. Much less of an issue if you're using zsh.
zsh does have the emulate builtin available to help with some basic things, but it is still easy to produce scripts littered with zsh'isms.
However, I do use zsh for personal scripts. In fact, I go so heavy on zsh'isms I know I'd need to rewrite things to share them if non-zsh users are an issue. It is easier to keep track of things that way, instead of tripping over the little easily forgotten quirks.
Aliases and macros are fish script though.