I feel like this needs a big asterisk. Can you ship a a non-trivial iOS or Mac app that uses SwiftUI or other first-party APIs without Xcode? Is it practical? Those are real questions, some cursory searching did not turn up a concrete answer.
You obviously still need to use signing pipeline etc, but that can all be done via CLI.
As for "non-trivial", here are Chrome's build instructions for macOS: https://chromium.googlesource.com/chromium/src/+/HEAD/docs/m...
Which contains the words "While using Xcode is unsupported" :)
Is it possible for 100% of situations? I don't know, because I haven't tried 100% of the situations. And in one case I haven't figured out yet (AUv3).