1. Startup time is too long. Often I find myself double clicking a code file in Finder to preview it only to be greeted with jumping Xcode icon. Riiiight, here we goooo, boss. Just 2-3 seconds, boss. It should open the file straight away, and then lazily load the rest of its frameworks if necessary.

2. Code editor is very bad with large files. Be it large JSON data, or C++ files from Unity's code generator, navigating and editing those is painful. Every typed symbol will incur a delay, every switch to different place in that file is noticeable too. This is by far the worst of all other code editors I used, but I get it, few people have to work with huge amount of text.

3. pbxproj format for project. Ever worked in a team where several people adding and removing files in their commits? Happy project merging then. Or rather, happy searching for a tool to auto-regenerate the project. This format has to go.

This is from my experience working with a couple of large iOS team projects written in Swift, with CocoaPods dependencies. YMMV for other setups. I've hit a bunch of problems in the IDE itself, the Swift toolchain, the iOS Simulator, and in the integration between them.

First, it's slow: compilation is slow, running tests is slow, starting the app is slow, debugging is slow (generally tens of seconds to attach when it hits a breakpoint), inspecting the view hierarchy is slow (and often just doesn't work). Code completion is slow and unreliable (and currently doesn't work at all for third-party frameworks in Xcode 13).

Besides the slowness, things get out of sync a lot, I think because it's doing tasks in parallel in an attempt to do them faster. I frequently see out-of-date compile errors and warnings highlighted in the IDE, because it hasn't finished the appropriate bit of the build yet, or because you're currently working in another target. Production code and unit tests are in different targets because that's how Xcode wants it, but because they drift out of sync so much and the tooling is slow, test-driven development is really tedious and slow. It's usually easier just to build and run the whole app.

The UI is usable, but tabs are very clunky and limited compared to other popular IDEs (e.g., you can't just drag to create a new split in any direction). The "assistant editor" for things like associated header files isn't necessarily a bad idea, but again it's clunky and limited. There are too many "mystery meat" icons with no labels attached -- I constantly get mixed up between Issues, Test and Debug.

The monolithic project file format is really awful, especially if you're working in a team. It's not usefully human-readable or editable, and every so often Xcode will make random changes to random GUIDs; you just have to shrug and check it in and hope that nobody else gets a merge conflict. The same applies to Interface Builder and all the other embedded rich editors. Just opening an icon or color resource will likely make some random meaningless edit -- why?

Adding a file to the project takes an age. "New File...", wait several seconds, "Choose a template for your new file", I don't care, type a filename, delete it again because that's actually the search field for templates, press Next, wait a few more seconds, and finally you're at the file dialog.

Why do files even need to be added? It would be so much better if it just reflected the file system. As it stands, files can be reordered (does anyone take the trouble to arrange them in a sensible order? if so, how on earth do you maintain that in a team project?), accidentally omitted from the project, in a different group than its actual position on disk... why?

Finally, some of this stuff might get better with more familiarity (e.g., if you set up keybindings) except that the UI tends to change substantially on each release! If you look for advice on StackOverflow on fixing some obscure problem with certificates (and I bet at least 90% of iOS developers have done this) you'll find five completely different solutions for five different versions of Xcode. Oh, and each new release is ~10GB and takes hours to install, literally hours of Archive Tool opening and verifying something called a .xip file.

There's no perfect IDE, and I'm not even saying Xcode is uniquely bad, but for each of my complaints here there's another tool that does it better. VS Code is really fast (I'm increasingly using it as a companion editor for everything other than .swift files); Android Studio for all its flaws has nicer tab management, and its interface builder equivalent works really well.