You only need clang and codesign to build an app.
if you want to build ios apps installing xcode is a requirement (to get the proprietary cli tools) but launching xcode is not required
to build native mac apps, neither installing nor running xcode is necessary, as you can build with the smaller cli-only tools which include only mac os headers/libs