Comment by gwd | Hacker News Reader

Comment by gwd | Hacker News Reader

Parent Full thread

> I like differentiable programming a lot, but can anyone explain the motivation for Swift as their language of choice for this?

Back in January, Jeremy Howard (of fast.ai fame) wrote up a blog post talking about why he was exploring using Swift for this purpose:

https://www.fast.ai/2019/01/10/swift-numerics/

The post is quite in-depth; but just to give a brief take on other potential languages:

8<---

Here’s my personal view of some languages that I’ve used and enjoyed, but all of which have limitations I’ve found frustrating at times:

Python: Slow at runtime, poor support for parallel processing (but very easy to use)

C, C++: hard to use (and C++ is slow at compile time), but fast and (for C++) expressive

Javascript: Unsafe (unless you use Typescript); somewhat slow (but easy to use and flexible)

Julia: Poor support for general purpose programming, but fast and expressive for numeric programming. ( Edit: this may be a bit unfair to Julia; it’s come a long way since I’ve last looked at it!)

Java: verbose (but getting better, particularly if you use Kotlin), less flexible (due to JVM issues), somewhat slow (but overall a language that has many useful application areas)

C# and F#: perhaps the fewest compromises of any major programming language, but still requires installation of a runtime, limited flexibility due to garbage collection, and difficulties making code really fast (except on Windows, where you can interface via C++/CLI)

> Edit: this May be unfair to Julia

Can’t understate this enough. Ultimately there is a trade-off being made between end user applications and numerical computing. Julia is surely superior when it comes to numerical computing. It has a super scientific stack and multi-dispatch means it is easy to mix and match these.

Of course there is no reason why Julia can’t evolve a similarly good ecosystem for webapps and other end user front ends. It is after all a _general purpose programming language_ which happens to have native numerical support.

Not sure why Julia is ruled out as not a general purpose language here - I got the impression that Julia was for numerical ie Technical programming

Because it would be a challenge to implement, for instance, an actual user-facing application in Julia.

Mostly because the end-user application community is not as strong in Julia (though you can create web services/sites and GUIs right now), in the same way the scientific/numerical computing/machine learning/HPC isn't as strong in Swift. Perhaps the point he made was that since his area is ML, then he could contribute better in trying to create a community around that in Swift than he could bring the other domains to Julia up to par with those languages by adding another competing library.

And in terms of the language itself, Julia is very much a general purpose language. Exceptionally general purpose as it's basically a Lisp below the surface, so it can not only support any domain, it can be comparatively easily extended to better support them. Differentiable programming is one such example, as the compiler was not designed for that, but you can just import this functionality as a library. Though the focus is still from desktop to HPC, as opposed to mobile/IoT to desktop (and apple focused) like Swift, which makes so having both languages supporting differentiable programming an overall positive over having only one of these languages.

Reading the link, it seems that Jeremy Howard only tried out Swift because Swift for Tensorflow existed. I got the impression from your comment that using Swift for this purpose was somehow "in-the-air" at the time, and many people independently considered it, but I don't think this is true. My impression is quite the contrary - when they announced Tensorflow support for Swift, I think many people were surprised.

In his lecture, he specifically said he interviewed both the Swift and Julia teams and got a view inside what happening at Google... and the the Swift teams work, development, and vision was well beyond what the Julia team had even envisioned.

Interesting since the manifesto hasn't mentioned something that's not already completed in Julia. Is their main project or motivation something that is not being shared? I find it quite odd that this manifesto doesn't mention anything that actually requires differentiable programming (old adjoint equations for their applications have existed since the 90's), which makes it a very odd justification for this work.

I suspect his comments are about the vision for the whole solution and environment, not just auto-diff, which is just a piece.

Auto-diff extends back 40 years. But the relative obscurity of its usage is that it is a "add-on", meta-programming step, or/and a limited sub-set of a language.

The Swift approach of 1st class compiler support is fairly unique. But also supports a future view of computing. If many problems can be solved by differentiable programming, not just the typical but somewhat niche NN/ML problems, then it needs to built into the whole language, not a sub-set.

But more importantly the future requirements, and death of Moore's Law means a fast, deeply statically analyzable language will be required to make the most of a heterogeneous computing environment.

Rust?

I’m still crawling like a baby with Rust but I’m quickly getting familiarized with low-level language notions while finding the struct/trait system notebook intuitive than Python.

C# and F# don't require the installation of a runtime per se . With the most recent versions of .NET Core, C# and F# applications can be published as completely self-contained applications.

(Yes .NET languages have a runtime, but I don't think that's what the author was referring to).

OT but I like your "snip" ASCII art, though it took me a while too long to parse it. :)

Reply on news.ycombinator.com