You can build UIs without storyboards/Interface Builder in UIKit just fine. And writing your UI in code indeed easily solves the whole versioning conflicts issue that storyboards have.
So no, not a big argument for SwiftUI, but instead for writing UIs in code.
SwiftUI vs. UIKit and IB vs. code are two entirely separate discussions.
But yes, I totally agree, if you must use storyboards, keep them as small as possible.