[1]:https://github.com/tensorflow/swift/blob/master/docs/WhySwif...
Later on from your link: "[We] picked Swift over Julia because Swift has a much larger community, is syntactically closer to Python, and because we were more familiar with its internal implementation details - which allowed us to implement a prototype much faster."
Honestly, I think the best argument (and the only one I buy) for Swift is the latter part: “…we were more familiar with its internal implementation details - which allowed us to implement a prototype much faster.” – just like I said about a year ago [1]. If you are sitting on a team deeply familiar and passionate about a language – Swift – what kind of managerial fool would not let them take a stab at it? Especially with Lattner’s excellent track record.
[1]: https://news.ycombinator.com/item?id=16939525
The ideas behind Zygote dates to somewhere around spring 2017, but I think it took about a year to hammer out compiler internals and find time to hack, so you are still right that nothing was public when Google settled on Swift – I think there been at least one Mountain View visit though over XLA.jl, but do not quote me on that one.
The race is still on and I am looking forward to seeing what all the camps bring to this budding field. I have worked with an excellent student on SPMD auto batching for his thesis project and we now have some things to show [2]. This is still a great time to be a machine learning practitioner and endlessly exciting if you care about the intersection between Machine Learning and programming languages.
[2]: https://github.com/FluxML/Hydra.jl
My only request would be for Jeremy to explain “Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance.” to me. Is it the “serious” and/or “widely used” subset where the Julia camp is disjoint? =)
There must be points when a technology takes off and becomes mainstream, what predates those points? That to me, this is the interesting question. In 2015 Torch (Lua) dominated the mindshare, why did TensorFlow succeed then? I think Lua itself caused it, lack of a coherent object model, etc. – sure as heck it was not the speed as I joked around by writing `import TensorFlow as TensorSlow` in my scripts for at least a year past the initial release. There was resistance against writing Python bindings for Torch, but in the end it happened and PyTorch was born; at this point TensorFlow dominated the mindshare. Why have PyTorch now become the favoured framework among all my colleagues and students then? Despite them being solidly in the TensorFlow camp prior to this. I think the answer is eager execution and the move in TensorFlow 2.0 to mimic exactly this speaks in my favour. So what would the Julia moment be then? If I knew, I would tell you, but I and several others in the Julia community are at least hard at work cracking this nut.
Just like I said a year ago, I am biased in favour of the bazaar and big tent that Julia represents. Swift will not have physicists, mathematicians, ESA employees, HPC people, etc. present and I would miss them as they bring wonderful libraries and viewpoints to the Julia bazaar. For example, I think TensorFlow with its initial graph model was designed that way precisely because it fit the mindset of non-practitioners with a compiler background – or perhaps it was the desire to “write once, deploy anywhere”? PyTorch could then take a big piece of the pie because they saw eagerness to be essential due to their academic/practitioner background. Only time will tell who is right here and I think me favouring Julia is not a safe bet, but certainly a reasonable one given the options available.
E.g. both Swift and Julia (and Rust, and Clang ...) have a common underlying backend (LLVM) in common, whereas in Common Lisp, it's ... Common Lisp all the way down, and it could potentially be a much better world, but it would just require duplicating too much stuff to be a viable candidate?