That’s just how things are on Windows. Building just about anything requires Visual Studio (not the UI but the tooling that comes with it. E.g. msbuild and C++ headers)
You won’t have to do this on Linux or if you use a pre-built exe
I have a tip on how to install it faster and easier with Chocolatey:
You can use winget for that now as it's included in Windows itself
That's just the reality of developing for Windows. You don't need their compiler but you do need the headers. Same goes for macOS which requires installing XCode.