In swift things have a module-wide visibility by default, that's why it often rebuilds the entire module after a single change that could implicitly affect other files. The only viable solution at the moment is to break the project into as many small modules as you can.