Julia is fast, with good syntax and lots of libraries. Infact it has a library called Flux.jl for ML/DL. Yet nobody uses it. Why? No major industry partner has picked it up to create bigger deep learning frameworks.
Why?
The ecosystem also feels weird. For example for CLI parsing in Python I do argparse/click, in Rust I do clap. Both work well (Rust especially so). Julia seems to have ArgParse.jl which I found unpleasant. My guess is that this sentiment spills to other areas as well in the ecosystem. I am sure that Julia has very well designed libraries, but most of the rest are not so honed for practical use, and most importantly, probably lacks some features that Python-equivalent ones have.
Also personally (I am far from a DL researcher but I do research), I find Julia's design headachy as well quite some times. I like Python yield statements for iterators but the equivalent in Julia is always more complicated. Making Julia code performant requires another layer of intuition of how the JIT works. Julia has type annotations but "type annotations as lightweight formal methods" often cannot be fulfilled due to design (for example one cannot annotate the equivalent of `map :: (a -> b) -> [a] -> [b]`). Of course I stand corrected as only a casual Julia user.
I also want to say that I only stated my criticism towards the language. There are many many good things about Julia that I will just omit due to time.
In academia code is the means to an end, which is usually a paper. This means that the experience of writing the code, making sure it's portable, readable, etc. don't really matter. These are all industry concerns; specifics of implementation, not algorithm design/mathematical questions.
This isn't reflected in Julia as a language as much as it is in Julia as an ecosystem. Some of the packages I work with are poorly maintained or designed with 0 thought to the issues I described earlier. And since Julia is so new, there are few alternatives.
Imagine if your software was written by your college professor who'd pushed to prod maybe like 3 times in their life. Instead of just fixing the damn bug they're liable to make you fix it instead as an 'exercise for the reader', or posit that the problem input is outside of the scope of their class and refer you to another one.
My example of this is the Colors.jl package. Images, rather than being 3D arrays of floats/ints, instead become 2D arrays of RGB/HSV. But you can't do any arithmetic on RGB/HSV values, you need ColorVectorSpace.jl for that. I'm sure it makes sense, but I can't help but feel as if sense is taking precedence over the programmer's experience in many of these packages.
Again, I don't think Julia as a language is unfit for widespread use. But before it can be, it needs to burst out of its ivory bubble.
If you just used Images.jl, it would load both Colors.jl and ColorVectorSpace.jl.
Let's say you obtained one of the matrices of RGB values, but you wanted a 3D array of numbers. `channelview` will provide you with a no-copy representation of the image as exactly that.
``` julia> using Images, TestImages
julia> img = testimage("barbara_color.png");
julia> typeof(img) Matrix{RGB{N0f8}} (alias for Array{RGB{Normed{UInt8, 8}}, 2})
julia> channelview(img) |> ndims 3 ```
While it took getting used to, have an image processing framework that explicitly has a concept of color instead of generically mapping everything into an array is actually quite useful. Having a 2D array representing a 2D image is conceptually quite nice.
Also you can do arithmetic on RGB or HSV types directly.
``` julia> (RGB(1,0,0) + RGB(0,1,0))/2 RGB{Float32}(0.5f0,0.5f0,0.0f0) ```
There are generic answers:
- Refactoring business logic costs. That's why it doesn't happen often.
- Interop between two different codebases (new Julia, old Python) costs (both at an org scale and at programming scale).
- It's a new lang. Hiring new programmers to a new meme language is very risky. You lose that programmer, your project is in trouble.
- You are underrating Python ecosystem strength.
- Python first mover advantage.
There are Julia specific reasons:
- No Julia programmers outside of MIT where as everyone has touched Python.
- Python is "simpler".
- Last I read Julia has some growing pains, like REPL boot times?
https://github.com/timholy/Revise.jl https://github.com/dmolina/DaemonMode.jl
For a language that builds much of its marketing around solving the "two language problem", it is somewhat ironic that they then turn around and create their own two language problem by ignoring most of the problems most programmers have.
I don't know how true that is but it meshes with my impression. The creators/maintainers seem to have a rather narrow view of who their target audience is and what use cases they have. And it reads to me as though they are unwilling to put any resources into making Julia a general-purpose language. Claims to the opposite don't convince me.
Of course they have the right to make that ecosystem as narrowly focused as they want, but then comparing it with Python is apples and oranges.
If we don't get hung up on the exact literalness of that statement, there's a lot of truth to it. Being essentially a "lingua franca" that's easy to pick up has accelerated it in various seemingly unrelated fields. This, I assume, it to the seeming detriment of use-case specific languages like Julia.
The fact that the ecosystem is largely created by academics is also a serious issue. Code quality is very low / things silently break / etc. This is partly an issue with Julia itself; the language offers few tools to verify the correctness of your code. (E.g. interfaces etc.)
Julia vs XYZ is an old topic. And yet this keeps coming up! There's quite a lot of blog posts on the topic. (E.g. here's mine: https://kidger.site/thoughts/jax-vs-julia/)
Do you have evidence for this claim?
It has many concrete examples of low quality and buggy code making it into pretty core statistics libraries. Most of these bugs are a combination of 'sloppy' coding and lack of vigorous testing of all the corner cases that the language has. Many developers seem to take a more 'academic' approach to things (it works correctly in all the cases I care about) rather than caring about creating robust libraries that are rock solid under all conditions.
From the blog: "My conclusion after using Julia for many years is that there are too many correctness and composability bugs throughout the ecosystem to justify using it in just about any context where correctness matters."
The good news is that this blog post seems to have brought some light to this problem and hopefully the Julia community can start addressing it.
Julia seems like an attempt at a better matlab, but the machine learning world moved to python first.
(Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)
You had me at 1-based indexing. This is sadly an anti-feature of Julia as far as I'm concerned.
I know that 1-based indexing is used in Fortran, MATLAB and Lua (and other places) -- but I just find 0-based indexing more natural to me. If I grew up using 1-based indexing I'd probably be saying the opposite but this is what I prefer now.
Julia is probably a more cleaner system overall. But then so is Deno compared to Node. The problem is that the alternative needs to be many times better not merely better. The "network" effort is quite strong in Python scientific computing this is still outweighing some of the real benefits of Julia.
It is always good to have alternatives. The quality of GCC improved when Clang came along. Julia's presence will keep Python on its toes. Both platforms will keep improving.
But, once again, Julia's 1-based indexing is something that adds unnecessary friction, for me at least.
julia also has weird behaviors scattered all over the place
> Guido van Rossum is a research assistant at the Centre for Mathematics and Computer Science in Amsterdam. Since 1987 he has been with the Amoeba project, working on an RPC interface specification language, a Unix emulation facility, user interface issues, and system integration. Earlier he worked on the ABC Programming Language project. Van Rossum studied mathematics and computer science at the University of Amsterdam and received a master’s degree in 1982.
I don’t think it’s a meaningful factor in case of Julia though: more like that Python is a safe and rather pleasant choice for most research-related problems, not only in ML. So no reason to change. And if you’re just starting, you’d better learn Python anyway, and if it’s enough, why look for something marginally better?
Julia needs a niche where it’s clearly better than Python (and R and C++ for that matter) and grow from there. Maybe there already is one, they need to market it more clearly.
When I want to experience a new language, I got to a place on the web, where I can try it.
For Python, there are https://www.online-python.com/ and many others.
For PHP there are https://3v4l.org and many others.
For Javascript there are https://jsfiddle.net/ and many others.
But every time I think "Hey, I might give Julia a try", I go to Google and type "try julia online" and find nothing. Every site either wants me to sign up or is dysfunctional. So I think "What? That cannot be. Lets try a different search". And search for "julia online repl". Again. Nothing works. And I give up.
I just did this again.
And I might try again in a year or so.
That's the situation here regarding Julia.
https://mybinder.org/v2/gh/binder-examples/demo-julia/HEAD?f...
. slow startup time
. 1-based indexing
. more complicated than python (many ML practitioners aren't really great at CS)
. native speed (a clear Julia win) doesn't really matter in the ML world: in ML, speed matters on learning and inference, where the language used to build and massage models is irrelevant, all of them compile models to run on GPU/TPU/etc ...
> [I]t looks like Plan 9 failed simply because it fell short of being a compelling enough improvement on Unix to displace its ancestor. Compared to Plan 9, Unix creaks and clanks and has obvious rust spots, but it gets the job done well enough to hold its position. There is a lesson here for ambitious system architects: the most dangerous enemy of a better solution is an existing codebase that is just good enough.
Python is good enough, and Julia isn't "Better enough".
That said I'm no lover of python, I just think this is a huge contributing factor to its rise in popularity. A little conspiracy theory, if you will ;)
R has multithread support: https://josephcrispell.github.io/2018/08/27/multi-threading-...
Julia's main draw was native really fast code with multi-thread/parallel support.
The other issue is that a parallelized c version of a data science algorithm can be scripted by Python or R pretty easily.
- Julia is faster at reading CSV's compared to using something like Pandas (presumably because it's multi-threaded and Python is single threaded) - Julia does seem faster with read operations, but it's important to write code that declares types (myIntArray = Int[] vs myIntArray = []), this way Julia would read from a contiguous block of memory instead of reading an array of pointers in memory - Julia's packages are limited to Python's equivalents
Our take is that Julia is amazing for ML/DL but Python is better for entry level data scientists. Once the data scientist gets up to speed with Python then Julia is a nice feather in the cap for more specialized use cases.
- pytorch or tensorflow - pandas - numpy - scipy - matplotlib - huggingface transformers - jupyter
- Pytorch/Tensorflow: There are several ML Frameworks written in Julia (as well as Julia bindings to ML Frameworks) the biggest Julia native one is likely Flux.jl
Regarding HF Transformers a quick Google points to https://github.com/chengchingwen/Transformers.jl but I have not had any personal experience with.
All of this is build by the community and your mileage may vary.
In my rather biased opinion the strengths of Julia are that the various ML libraries can share implementations, e.g. Pytorch and Tensorflow contain separate Numpy derivatives. One could say that you can write an ML framework in Julia, instead of writting a DSL in Python as part of your C++ ML library. As an example Julia has a GPU compiler so you can write your own layer directly in Julia and integrate it into your pipeline.
I’d say that Julia’s arrays are significantly better than NumPy. Julia’s broadcast operator is like ufuncs except way easier and way faster.
The machinery around GPUs is better, too, in my opinion. GPU support isn’t coupled to other concerns like automatic differentiation. You can write generic code in a broadcasting style (which is, again, a lot nicer than NumPy by default) or very easily write your own kernels.
Multithreading is also way better than in other scientific computing languages.
Julia is still far from ready for serious adoptation for heavy workflows.
For libraries, the optimization cannot be trusted. And it cannot be said many times that best practices were followed.
While you can trust NumPy or PyTorch, you cannot trust a Julia library at the same level.
Julia vs. Python
- PyTorch is standard and it is hard to convince other people to switch
- long compile time on startup during deployment (not so good for a robot) but also for plotting; other people really hated this
Julia vs. C++
- Julia has a JIT and is MUCH faster than Python if you cannot write it as a sequence of numpy operations, e.g. if you have loops and if-blocks in the main loop; C++ obviously also shines here
- however, similar to Python you can only detect problems of the code when running it - the linters etc. are not good enough; hence, I also fear changing only a few lines; programming in C++ is much easier and you have much more confidence that the code is correct if it compiles
After learning JAX in Python, which compiles numeric code JIT, I have almost no reason using Julia anymore. Of course, DifferentialEquations.jl and many optimization libraries are top notch.
Python:
>>> 2**2**2**2**2
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: Exceeds the limit (4300) for integer string conversion
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char* argv[])
{
printf("%.0lf\n", pow(pow(pow(pow(pow(2, 2), 2), 2), 2), 2));
return EXIT_SUCCESS;
}
% gcc -o main -std=c18 -O2 -pipe -fPIC -fno-plt -fstack-protector-strong main.c && ./main
4294967296
julia> 2^2^2^2^2
0Python computes the correct result here, since it uses bigints by default. (The error is when formatting it for display, easily worked around by an extra line of code.)
In the C and Julia cases, it's well-known to use bigints when working with large numbers. For example:
julia> BigInt(2)^2^2^2^2Mathematica (default settings):
In := 0.2 - 0.10000001
Out = 0.1
The Julia equivalent is:
julia> ((((2^2)^2)^2)^2)^2
4294967296
The limit of 4300 can be changed using https://docs.python.org/3.11/library/stdtypes.html#int-max-s... . Set it to 0 for unlimited. (I wonder if there will be a thread-local context for this setting in the future, or some other way to bypass the limit on an as-needed basis.)
In any case, your Python and C examples compute different values because you mistook a right-associative operator for a left-associative operator. On my Python 3.12.0a0:
>>> pow(pow(pow(pow(pow(2, 2), 2), 2), 2), 2)
4294967296
>>> ((((2**2)**2)**2)**2)**2
4294967296
>>> big = 2**2**2**2**2 # a**b is right-associative in Python
>>> len(str(big))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: Exceeds the limit (4300) for integer string conversion
>>> import sys
>>> sys.set_int_max_str_digits(0)
>>> len(str(big))
19729
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char* argv[])
{
printf("%.0lf\n", pow(2, pow(2, pow(2, pow(2, pow(2, 2))))));
return EXIT_SUCCESS;
}
% gcc -o main -std=c18 -O2 -pipe -fPIC -fno-plt -fstack-protector-strong main.c && ./main
inf
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char* argv[])
{
int value = 2;
value = value * value;
value = value * value;
value = value * value;
value = value * value;
value = value * value;
printf("%d\n", value);
return EXIT_SUCCESS;
}
% gcc -o main -std=c18 -O2 -pipe -fPIC -fno-plt -fstack-protector-strong main.c && ./main
0Its just that everything else is even worse for this use case, so nobody does.
An initial disclaimer. I am an academic that has been involved in the Julia community for many years and written plenty of Julia code myself. In addition, my entire team uses Python as their day-to-day language to drive research (I do not code nearly as much as I used to these days) and Python was what I used for artificial intelligence research for many years as a student and researcher.
Looking at the other comments, I find that most focus on language features, implementation aspects, ergonomics, etc. I think this is a distraction for machine learning code, at the very least when it comes to research. Rather, I think we need to ask: What determines success for a machine learning researcher/practitioner?
Firstly, if we turn back the clock ten or so years, most models were written from scratch. This was the time when there was an active “battle” between Java, Matlab, and Python for mindshare. With the shift to deep learning (and GPGPUs), big libraries/frameworks have taken over as these models can be easily composed from primitives of varying degrees and coupled with training algorithms, regularisers, losses, etc. Pre-trained models have also become increasingly commonplace and although they can be ported between languages, the safest bet is of course to go with the training/inference code that the original authors used: Most likely, Python. Julia is putting up a decent fight here against libraries/frameworks backed by some of the richest corporate entities on the planet, but it makes were little sense for most people to go against the grain here as their success is determined by how fast (think man hours) they can put together experiments or push things into production. Dare I say that no one ever got fired for picking Python?
Secondly, if you are a Julian and at this point decry how sluggish the snake is, how its implementations are suboptimal stacks across language boundaries, has ancient ergonomics, etc. You are most likely missing the point. I doubt that this battle is won based on any of these “nice-to-haves”. I had the pleasure to sit down with among others Flux’s creator back in 2016 and what I said then I believe still holds today: “We can not just better express what models others can express, we need to enable the creation of new models which other languages can not easily express”. We have seen Julia gain some traction when it comes to Neural Differential Equations for example, which I believe is akin to what I am talking about here. Perhaps there is hope in Julia having a better interface when it comes to bridging symbolic and non-symbolic models, but ultimately it is difficult to predict the future in terms of which models will succeed and dominate the field.
To me, if we focus solely on the language itself Julia is largely superior. It has better ergonomics to express machine learning (well, maths in general), a better packaging system (the fact that I have known Python for more than 15 years and it still struggles when it comes to deployment boggles my mind), etc. But, at the end of the day, I think that the delta here is simply not big enough – yet? – to motivate the great majority of researchers and practitioners to make the jump and I respect what I believe is a very rational choice on their part.
I would prefer as much as possible that we have the leanest tech stack possible. That means reducing the number of languages, libraries, etc to a sane amount.
Is this view, Python is a very good candidate for data science.
You can keep the same language for your data science logic and scientists, as well as for your more down to earth developments on infrastructure/techie stuff.
- 1 indexed arrays
- extremely slow start up times (1 min??)
- no way to create portable static binary apps (disadvantage vs rust/go)
- lack of tutorials, demos
- julia is not really faster for math when compared to jax/tf/pytorch/etc anyways
Is it similar in ML?
A Differentiable Neural-Network Force Field for Ionic Liquids. (J. Chem. Inf. Model. 2022)
H. Montes-Campos, J. Carrete, S. Bichelmaier, L. M. Varela, and G. K. H. MadsenCorrelation Tracking: Using simulations to interpolate highly correlated particle tracks. (Phys. Rev. E. 2022)
E. M. King, Z. Wang, D. A. Weitz, F. Spaepen, and M. P. BrennerOptimal Control of Nonequilibrium Systems Through Automatic Differentiation. M. C. Engel, J. A. Smith, and M. P. BrennerGraph Neural Networks Accelerated Molecular Dynamics. (J. Chem. Phys. 2022) Z. Li, K. Meidani, P. Yadav, and A. B. FarimaniGradients are Not All You Need.
L. Metz, C. D. Freeman, S. S. Schoenholz, and T. KachmanLagrangian Neural Network with Differential Symmetries and Relational Inductive Bias.
R. Bhattoo, S. Ranu, and N. M. A. KrishnanEfficient and Modular Implicit Differentiation.
M. Blondel, Q. Berthet, M. Cuturi, R. Frostig, S. Hoyer, F. Llinares-López, F. Pedregosa, and J.-P. VertLearning neural network potentials from experimental data via Differentiable Trajectory Reweighting. (Nature Communications 2021)
S. Thaler and J. ZavadlavLearn2Hop: Learned Optimization on Rough Landscapes. (ICML 2021)
A. Merchant, L. Metz, S. S. Schoenholz, and E. D. CubukDesigning self-assembling kinetics with differentiable statistical physics models. (PNAS 2021)
C. P. Goodrich, E. M. King, S. S. Schoenholz, E. D. Cubuk, and M. P. Brenner
Some threads discussing the merits of JAX, PyTorch and Julia:
https://news.ycombinator.com/item?id=30349687
https://news.ycombinator.com/item?id=29682507 https://www.reddit.com/r/MachineLearning/comments/shsfkm/d_c... https://www.reddit.com/r/MachineLearning/comments/st8b11/d_s...
https://www.assemblyai.com/blog/why-you-should-or-shouldnt-b...