"xcode-select --install" grabs only the command line utilities included with Xcode (git, swift, clang, python3.8, etc), and it's only 400MB on my machine. That command has been there for a while, since Leopard or so, and Homebrew prompts you to run it before you continue with installing brew.