I'm not familiar with Swift, but I'd be weary of treating dead-code removal as an excuse for piling on libraries. Deciding whether code is actually dead looks like a halting-problem-level type of problem. For example, if I write the following Golang code:

  str := "World"
  fmt.Printf("Hello %s\n", str)

Then the compiler pulls in the entire fmt library, which can render all kinds of datatypes as part of Printf(); there are even reflection-based format strings like %#v which generates a Go literal for the value that you pass in. Without knowing how Printf() works, the compiler cannot understand that I'm never going to visit these branches of the Printf() function, so it cannot tree-shake these (or any of the functions called by these branches).