Comment by ninjin | Hacker News Reader

Comment by ninjin | Hacker News Reader

Parent Full thread

As always, a disclaimer, I am deeply involved in the Julia community so beware of my biases.

Honestly, I think the best argument (and the only one I buy) for Swift is the latter part: “…we were more familiar with its internal implementation details - which allowed us to implement a prototype much faster.” – just like I said about a year ago [1]. If you are sitting on a team deeply familiar and passionate about a language – Swift – what kind of managerial fool would not let them take a stab at it? Especially with Lattner’s excellent track record.

[1]: https://news.ycombinator.com/item?id=16939525

The ideas behind Zygote dates to somewhere around spring 2017, but I think it took about a year to hammer out compiler internals and find time to hack, so you are still right that nothing was public when Google settled on Swift – I think there been at least one Mountain View visit though over XLA.jl, but do not quote me on that one.

The race is still on and I am looking forward to seeing what all the camps bring to this budding field. I have worked with an excellent student on SPMD auto batching for his thesis project and we now have some things to show [2]. This is still a great time to be a machine learning practitioner and endlessly exciting if you care about the intersection between Machine Learning and programming languages.

[2]: https://github.com/FluxML/Hydra.jl

My only request would be for Jeremy to explain “Swift for TensorFlow is the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance.” to me. Is it the “serious” and/or “widely used” subset where the Julia camp is disjoint? =)

Nice comment, thanks. While I have several years of professional TensorFlow under my belt, since last fall I have spent a fair amount of time at home trying Julia + Flux. I find Flux very nice to use but it is a chicken and egg problem: I think Flux needs a lot more users in its community to really take off. TensorFlow and Pytorch dominate mindshare.

It is indeed a chicken and egg problem, but that is always the case and is mostly an argument for the status quo. When I teach there is always a handful of students asking “Why examples in Julia? Python is what is used in industry.” and my response is always the same: “Yes, true, and I could have used the same argument against Python in favour of Java in 2004. I think Julia has a reasonable potential of being ‘the future’ based on my experience as a scientist and programmer, so my hope is that I am giving you glimpse of the future. Besides, are you not at a university to learn a diverse toolbox rather than being cast into an employee mould?”.

There must be points when a technology takes off and becomes mainstream, what predates those points? That to me, this is the interesting question. In 2015 Torch (Lua) dominated the mindshare, why did TensorFlow succeed then? I think Lua itself caused it, lack of a coherent object model, etc. – sure as heck it was not the speed as I joked around by writing `import TensorFlow as TensorSlow` in my scripts for at least a year past the initial release. There was resistance against writing Python bindings for Torch, but in the end it happened and PyTorch was born; at this point TensorFlow dominated the mindshare. Why have PyTorch now become the favoured framework among all my colleagues and students then? Despite them being solidly in the TensorFlow camp prior to this. I think the answer is eager execution and the move in TensorFlow 2.0 to mimic exactly this speaks in my favour. So what would the Julia moment be then? If I knew, I would tell you, but I and several others in the Julia community are at least hard at work cracking this nut.

Just like I said a year ago, I am biased in favour of the bazaar and big tent that Julia represents. Swift will not have physicists, mathematicians, ESA employees, HPC people, etc. present and I would miss them as they bring wonderful libraries and viewpoints to the Julia bazaar. For example, I think TensorFlow with its initial graph model was designed that way precisely because it fit the mindset of non-practitioners with a compiler background – or perhaps it was the desire to “write once, deploy anywhere”? PyTorch could then take a big piece of the pie because they saw eagerness to be essential due to their academic/practitioner background. Only time will tell who is right here and I think me favouring Julia is not a safe bet, but certainly a reasonable one given the options available.

Great response, thanks! I am sure Julia will keep growing in use and influence. BTW, really off topic, but I am sometimes inconvenienced by the opposite problem of being very used to using older languages like Common Lisp (and to a lessor degree Prolog) that are still very relevant but are losing mind share. On the other hand just this hour an old customer contacting me to do work in Common Lisp and Prolog so there is still some interest in older languages.

Do you think this might just be people wanting something familiar "at the bottom"?

E.g. both Swift and Julia (and Rust, and Clang ...) have a common underlying backend (LLVM) in common, whereas in Common Lisp, it's ... Common Lisp all the way down, and it could potentially be a much better world, but it would just require duplicating too much stuff to be a viable candidate?

Reply on news.ycombinator.com