The downsides are similar to Mac. Less standard/compatible with other stuff. Zsh seems more like a traditional shell. Meaning it is far more complex. Fish really aims to keep simplicity.
I have used Fish for many years and swear by it but I am also not the kind of guy who writes long shell scripts. I use regular programming languages for that. For me fish is mainly about having a shell that works well in interactive use.