It's not great, but for an IDE (not a text-editor with plugins), I've always found it miles better than the Intellij suite. I preferred Visual Studio when I was working on Windows, but that's a much bigger project, so perhaps not a fully fair comparison.
Xcode from ~2021 onwards, with Swift package support, has been a significant improvement too. Now you can avoid much of the IDE configuration and just work with source code more, something I prefer. There was also a period of a few years where it crashed all the time (Swift 1-4?) but those days seem to be long gone now.
The source control integration has always sucked and I steer clear, but I was always doing the same in Intellij/etc or Visual Studio anyway.
Perhaps today VSCode is sufficiently IDE-like with enough packages, and perhaps it's fast enough, but I still feel like there's a difference, and even though I can do Swift in it I prefer to use Xcode. Changing Xcode's shortcuts to something more reasonable made a huge usability difference too.
I'm interested to hear what you think is so terrible about Xcode. Perhaps I'm missing something!