The downside of this is that you do need to "context switch" a little, especially if you've been using bash/zsh for most of your life. I find it easy to differentiate between when I am using the shell as a user interface and when I'm using bash/sh as an interpreter, but for others that might not be as intuitive.
I've also found that since I've stopped using POSIX compliant shells as my daily driver, I tend to "promote" my scripts to a more general purpose language much sooner than I used to when nearly everything could be done in bash.
Just use your Bash/zsh to run Bash/zsh scripts: `bash script.sh`.
> turned off by it not being POSIX-compliant.
I was like this when I first started using Fish. I spent some time to learn some different syntax to get away from the POSIX-compliance. Overall, I think the saner syntax of Fish is worth the effort. Now I use the terminal much happier.
Besides, 90% of the time I don't need to follow POSIX-compliant. Many scripts or extensions nowadays support Fish.
For enterprise use, I still stick to Bash/zsh. They're the standard.