Swift for Tensorflow 0.3.1
groups.google.com
groups.google.com
Kotlin is successful because Google was able to drive the direction of the language. For example AndroidX Compose is being built with compiler extensions in Kotlin with Jetbrains. Dart is obviously driven by Google.
Swift is going to be controlled and driven by Apple. From a strategic perspective, I don't understand why Google would make an investment like this when Swift-for-Tensorflow has a fundamental need to drive changes in the language itself.
Now, I understand if Google didn't have an alternative. Are you saying Kotlin is soooooo worse than Swift that you can't fundamentally take a similar direction ? Kotlin is currently one of the world's most popular languages, born out of sheer LOVE from the community (without being pushed by the giants).
https://github.com/tensorflow/swift/blob/master/docs/WhySwif...
Presumably Kotlin wouldn't work for the same reasons they excluded Scala and Java and C#.
(I think that either of C++ or Julia would have probably made better choices, given what they spell out here, but they put a lot more thought into what language to use than most HN commenters seem to give them credit for.)
Now, it's possible that they were less objective about the language choice than they believe themselves to be, but they certainly have thought about it an awful lot, and they deserve some benefit of the doubt, unless you can demonstrate what of this justification is untrue.
Moving TF to Julia would have killed 2 birds with one stone: community size and outright number of packages would have both been fixed as a community grew up around TF in Julia.
But no ten horses would make me use it for anything else. Machine learning needs to be fully integrated into the larger language ecosystem. You can easily imagine developing a large scale software product in Swift, not so much in Julia (even less so than in Python). Julia is probably a Nightmare to develop in once you have more than 10 people working on the project.
The internal structure of Julia's compiler is also a mess compared to the Swift compiler. You could easily imagine building a compile service / large scale build system / tooling for swift, not so much for Julia. Plus they were able to easily extend the Swift intermediate language in the way they wanted to, while there is no comparable abstraction in Julia. The Swift compiler also fits neatly into the C++ based ecosystem that Google has.
There is a bigger chance that another contender arises within the C#/F# ecosystem as an evolution of the ML.Net work, then that a company invests significant resources in Julia to make it suitable for large scale machine learning in a non scientific "blue collar" setting.
What? I have had no trouble with the errors produced from multiple dispatch! More often than not they've helped me get straight to the fix, which is more than I can ever say for Python.
> There is a bigger chance that another contender arises within the C#/F# ecosystem as an evolution of the ML.Net work
I would rather claw my eyes out than do significant data work in .Net. C# is our language of choice at work, and it's pretty great for them, but my god is it the most verbose, confusing and convoluted language I've ever had to deal with. F# isn't too bad, but once again, lacks any serious data science community and package support. They're also not terribly fast languages, which is a pretty important factor in ML stuff.
First, julia has it's own IR (comparable to swift IR as mentioned by Chris), which is used to great effect in the autodiff packages to allow for compile time zero overhead gradients. The flexibility is actually the other way around, Julia's compiler allows these sort of compile time initiatives to be built in Julia rather than hacked into C++ like swift.
Regarding the duck typing vs type checking bit, that's a matter of tooling. It's much harder to replicate Julia's combination of speed and dynamism (for example to do flexible multistage programming) than to capitalize on Julia's the compile time information to build type checking (which are planned)
The compiler lag is going to be gone soon from two ends: more dynamic (interpreting code that doesn't need to be compiled, this is pretty much there already) and more static: Compiling and caching entire Julia programs. There are already fantastic improvements to this situation in 1.2
That leaves the error messages and stack traces, which ...well what's the last version you have tried? I find them to be ok, but they can and will be improved.
Regarding general purpose packages in Julia: check out http://genieframework.com/ as an example.
Julia's multimethods are awesome and a clear advantage over swift's methods. A protocol abstraction is similiar to julia's facility for traits, will at some point will be likely baked into the language (but even now allows for compile time resolution of methods just through multi dispatch abstraction , which again, is only one step away from building type check abstractions!).
If you want to know more about Julia's plans and philosophy in the regards, see: https://github.com/FluxML/Flux.jl/issues/614
Swift has clearly an advantage in that domain, as has C++, C#. In an enterprise setting such as Google no one actually wants speed and dynamism out of a language (C++ programmers are not allowed to use rtti for example), rather consistently boring, predictable, statically guaranteed results with as minimal fuzz as possible (Go, Java, C++) and enough competent programmers in that language.
I agree that it is neat that you can hack the Julia compiler in a library to do compile time automatic differentiation, but my point was that if you build it into the language like the Swift for Tensorflow authors are doing this will result in a more conservative, if inflexible, but potentially more performant solution. In other words precisely because you need a team of people to implement and integrate this feature in C++ you will get a stable "single source of truth" implementation in the language, not several (Flux, Zygote) constantly changing libraries.
Well I've tried DiffEqFlux.jl with Julia 1.1, it works well enough, but the stack traces are ridiculously long if something goes wrong, a ~200 LOC program takes seconds to compile and the library and package management story seems immature compared to rust/swift/python/c++.
That's a pretty weird statement, considering how hard it is to write a complex C++ program error free. I suppose Swift is much better in this regard - But compared to C++, Julia is a bliss and one can easily write huge applications error free, while I had my absolute worst debugging experiences in C++, where it's very easy to create obscure and hard to find bugs.
If you really need to get predictable results and actually know those results - you won't get away with a static type checker anyways, but will have to write & run tests, which then pretty much make it irrelevant whether your language is dynamic or not.
I just run my code in small batches while developing it, so at every point I already know that it works. This helps writing code that nicely separates into small chunks of functionality, and you basically automatically write tests for it while at it.
This is especially nice when refactoring a large code base. I can already run many small and usable tests in the middle of a refactor (while the program wouldn't even compile in a static language).
In my experience, this greatly helps to improve the quality of the refactor, and makes it much easier to see problems in your refactoring approach very early on.
In a static language, you'd only get these crucial insights after you're done with fixing all compilation errors - which is usually when you're already done with the whole refactor... and after that, you might still have lots of errors in it that you can only find with real tests ;)
Also, Kotlin/native is fully compiled.
Who knows, maybe in 5 years it will make the list.
Here's my claim - Kotlin simply BLOWS AWAY Swift when it comes to server side
https://engineering.khanacademy.org/posts/kotlin-adoption.ht...
https://medium.com/insiden26/5-reasons-why-n26-is-moving-to-...
https://devcenter.heroku.com/articles/getting-started-with-k...
https://twitter.com/yogurtearl/status/1012551386536394753
https://medium.com/@napperley/kotlin-server-side-adoption-d1...
Kotlin is pretty much a thing on Android, and only because Google doesn't bother to support anything beyond Java 8.
Outside Android we get to enjoy Java's improvements.
I like Swift and all but our ML/DL/RL/DS tools and libraries are in Python (and occasionally R). Most are missing for Swift without an awkward Python compatibility layer and I don't see a compelling reason to adopt it.
And people are still doing a lot of Data Science work in R and Scala so I wouldn't say it is at all Python centric.
As its very common to target this sort of code to GPU´s, and LLVM can target them as output, i think is mostly because you can design and shape the langage right on its high level representation and tune for high performant code in the low level.
The why, i think, its about opportunity.. Chris Lattner going to Google, and people from the language and the compiler side of the fence being open about bake this right into the compiler when necessary.
Edit: I apologise, just after posting this comment I saw this link had already been posted. https://www.fast.ai/2019/03/06/fastai-swift/
I do think what is being done with Julia, Cassette, Flux and Zygote more interesting since it's Julia all the way down (while Tensorflow's backend is still C++) and the compiler work is focusing on not being specific to one implementation or technique, but allowing any such language extensions (such as auto-parallelization and other forms of source to source transformations) to be done by any 100% Julia library. So if Tensorflow for Swift (regardless of the actual reasons behind the choice of the language) proves that the technique is a significant upgrade over what currently exists, it could spark interest in the competing approaches, and I think Julia can help pushing the concept even further.
the python interoperability allows you to use all python libraries but in swift
And there is the risk that the community simply ends up considering that good enough and just make wrappers (since it needs a lot of work to create something nearly as good from scratch). Thankfully that didn't happen with the Julia community, and the key is probably making the creation of the tools much easier so they can catch up to mature but constantly evolving environments.
As a stopgap while all the data science tooling is being built out for Swift you can use anything from the Python ecosystem (including np and pd) by using the Python interop:
let np = Python.import(“numpy”)
https://github.com/tensorflow/swift/blob/master/docs/PythonI...
> To accomplish this, the Swift script/program simply links the Python interpreter into its code.
I would imagine that having the interpreter in another process would be a gigantice performance hit.
You seem to imply that people who upvote both Swift/TF articles won’t upvote Julia ones.
For me personally, I have upvoted earlier Swift/TF articles because they were well-written, and integrating differentiation fairly deeply into the compiler seemed novel to me.
I think I also have upvoted some Julia articles in the past, not because they were about Julia, but because they were interesting and well-written.
”tools and libraries are in Python (and occasionally R). Most are missing for Swift without an awkward Python compatibility layer”
An upvote need not mean “that’s immediately useful for me”. It could also mean “I like Swift, and this looks quite an improvement for it, making it more competitive with the leading technologies for machine learning” (you like Swift, so you _could_ upvote articles like these for that reason, too)
Linux support for this can't be sub par since the whole stuff will run on linux like 99.99% of the time.
But i guess the pure Windows ecosystem is rough on the edges right now, but doable. And in WSL(1) things have worked pretty the same as in Linux.
Just download the packaged binary distributed on their site, and you are ready to go.
(Oh, a note; its not the tensorflow branch, if is thats what you want)
Absolutely not. Julia would have been a far better fit.
Disclaimer: I don't use Windows but read the Swift development forums from time to time.
The idea of LLVM/Swift ‘turtles all the way down’ might really pay off in the long run. In TensorFlow there is an un-understandable barrier (for me) between the Python layer and the lower level C++ code. Good to see this Swift effort along side Julia + Flux, which offers the same implementation language for the whole system advantage.
It’s not ready for prime time yet but the roadmap looks great. Building auto-differentiation into the language and having an easy way for anyone to experiment with core architecture changes (without killing performance) will be a huge win for everyone.
I expect lots of prototyping will be done on S4TF (because it will be easy for developers to keep things on the GPU with XLA and MLIR) and then the things that work will be back-ported to other ecosystems.
https://medium.com/tensorflow/mlir-a-new-intermediate-repres...
Good luck with those CUDA drivers, and visual GPU debugging.
Specially when Windows had the best GPU support.
Otherwise gamers would all be switching from win10 to Steam OS.