The LLVM project makes it's stance on patents quite clear [1].
His claim is that Apple ships a LLVM that is different from what you can get as source code (thanks to a lack of copyleft). See the saga of the two aarch64 backends...
> Apple's announcement remained completely silent on patents, and we should expect the chosen non-copyleft license will not contain a patent grant.
First off, some frontend features are only feasible by changing LLVM itself. This is uncommon but ultimately not rare at all, this is where IR features like musttail, inalloca, safestack and segmented stacks originated. It is unlikely that swift will never need such a change.
Secondly, it would directly contradict what Chris Lattner announced at WWDC: They are going to accept contributions (both ideas and code) from others.
Finally, a frontend which generates LLVM IR has a very tight dependency on LLVM itself. LLVM IR is allowed to freely evolve which, in turn, causes APIs to change or disappear. It is much easier to deal with this if your code lives on llvm.org because the community is obliged to make sure your code continues to work. People fix polly, lldb, clang, etc. if their change to LLVM broke them.
At one point Apple actually wanted to merge the entire of llvm into gcc, and were prepared to non-assert and patents that might be involved: https://news.ycombinator.com/item?id=8868994