On macOS, one major drawback is that if you use a non-standard installation directory for Homebrew, you can no longer install binary packages and everything is compiled from source.

If you can live with that, you should be fine though.