The Swift 4 compiler has a Swift 3.2 mode that is 99.9% source compatible with Swift 3.x. Further you can link 3.2 and 4.0 binaries together. This allows a phased upgrade approach that was not possible in earlier releases.

Swift 5 is about having a stable binary interface so you can link against pre-compiled code going forward. In other words a Swift 6/7/8 binary can link against one compiled with Swift 5.

The bar for source breaking changes is also much higher than the already high bar in Swift 4. We may not see any source breaking changes in Swift 5.