Xcode had no such problems with Objective-C though... Maybe Swift's design deserves some blame here for years of subpar developer experience.
Like many of Apple’s developer tools, if you stick to the happy path, performance is great. Practically that means compiling your source into a single Swift module and keeping the number of modules low. If you are a “basic” iOS or macOS app with just a couple of 3p libraries things work great. Real world projects are drastically more messy...
Autocomplete in Swift is still broken but I don't think it's an issue with the language itself (autocomplete is probably easier to implement in Swift given the nature of the language).
CMD+,
Locations
Derived Data
CMD+Q Xcode
Delete!!! Derived Data
Restart Xcode
Why not Clean Build Folder
CMD+B
success!! mv ~/Library/Developer/Xcode/DerivedData ~ && rm -rf ~/DerivedData &
And I use it constantly. This approach is fast because it does the actual rm in the background so I can relaunch Xcode right away.