I have to shoutout Chris Rackauckas for being a such badass, helpful person too. He'll probably be in this thread any minute because he's the best damn advocate for Julia there is. :-)
But "best damn advocate" is probably the opposite of how I'd describe him in terms of interactions here (and generally with people outside the core Julia circle). He very often comes across as dismissive, overly defensive, and passive aggressive in comments here. All of that is dwarfed by his package contributions tbh, in terms of impact on Julia. But still, probably half of the negative perception about Julia community that people have, come from reading these interactions.
This ambivalence might sound absurd on the face of the exponential recent growth of Python (which apparently enticed even some people with serious mojo to get into the act) but take two steps back with me and look at the big (if still hazy) picture:
We are going through a remarkable period where complex algorithmic applications left academia and research labs and diffuse into mainstream society and the economy like never before. This process carries enormous risks and opportunities, which are currently basically... ignored (well, the risk side).
Despite its undeniable strengths and loveability, Python is actually a poster child of the move-fast-and-break-things phase. It is not necessarily best placed for the next phase. The next phase will invariably see a re-examination of all aspects of the stack and qualities that will be prized will be those that eliminate the frictions and risks associated with the large scale deployment of algorithms. The stakes are high, which means there will be plenty of resources seeking to create reliable platforms. The future need not look like the past.
None of the usual suspects ticks all the boxes. In fact we don't even know all the boxes yet. Depends how fast and how seriously models and algorithms get deployed at scale. Python, Julia and R have been propelled forward by circumstances as the main algorithm-centric platforms, and they have each their various warts and blessings but the near and mid-term future will test how well they can deliver on aspects they may have not be designed for.
What risks are you talking about here?
It sounds absurd because trends don't reverse overnight. You can be fairly confident that Python will be the top language in this space for a while and that R will never be the top choice for most applications.
But now is a time where at various high places people will say: "Ok you got my attention. What is this snake language you are talking about and explain why I should bet the house on it".
And the answer is not simple.
https://www.tiobe.com/tiobe-index/perl/
https://www.tiobe.com/tiobe-index/objective-C/
In 2006 (for Perl) and 2014 (for Objective-C) it was clear they had the momentum for their particular space however their limitations were well known and as soon as a better language came along the momentum flipped in an equally dramatic manner. Python is much more widespread so it will remain strong in some areas but you could see the flip in ML/DS given challenges productionizing across broad capabilities (not just doing NN's).
As the joke goes -- python is the second best language for everything, if you know only two languages. With ML expanding beyond narrow big tech domains there will be need for specialized languages like Julia (and others perhaps like Mojo etc..)
But, there are still reasons I reach for Julia.
Interesting packages where I prefer Julia over Python/R: Turing.jl for Bayesian statistics; Agents.jl for agent-based modelling; DifferentialEquations.jl for ODE solving.
I would much rather data-munge tabular data in Julia (DataFrames.jl) than Python, though R is admittedly quite nice on this front.
Personally I reach for Julia when I want to use one of the previous packages, or something which I want to code up from scratch, where base Julia is much preferable to me than numpy.
I love Python, but I can also see eventually doing everything in Julia over the longer term. Mind you, it's entirely possible that AI continues to improve and in 5 years any package will be available in any language, you'll look at code mainly for verification purposes in whatever language you happen to prefer.
Let the language optimize more so you don't have to write a C++ library or figure out how to use it optimally. Don't waste as much time setting up your environment or worrying about platform compatibility. Don't worry about using multiple languages for different types of computation. And make the on-ramp fairly painless by being a convenient glue language.
It fills the gap of otherwise not having a managed and JITed language for general mathematical computation. If it's more burdensome for you to switch, then don't switch.
If you're writing code that is fundamentally based on mathematical principles and models, even if you aren't personally using mathematics every day, its going to feel a lot better in Julia. That is: Julia looks a lot more like mathematics than Python.
__
Longer version:
Obviously some people are mostly writing websites or GUIs or whatever in Python and won't see the beauty in this.
But if the problems you are working on have, at their base, a mathematical foundation (even if you don't actively practice the math), it's much more beautiful IMO. So, simulation, data analysis/science and machine learning, statistics, etc...
Once you get used to using it for that though you'll realize it's actually quite nice for a lot of other things as well and the "mathematical mindset" it somewhat pushes results in cleaner solutions for other problems too. Just in general the syntax and patterns are nice.
Here are some quick things using randomness in Julia that would be a bit slower and more verbose in Python:
Generate a random number:
> rand()
Pick a random message:
> rand(["First message", "Hello", "Foo"])
Generate a random 3x3 matrix of booleans
> rand(Bool, (3,3))
Define a function and run it elementwise on a random matrix of bools:
> myprint(x)= x > 0 ? "Happy" : "Sad"
> B=and(Bool, (3,3))
> myprint.(B)
Returns:
> 3×3 Matrix{String}:
> "Happy" "Happy" "Sad"
> "Sad" "Happy" "Happy"
> "Sad" "Happy" "Happy"
And many many more nice features...but the Julia design meaning functions like rand() just apply how you expect regardless of the input type are quite nice. rand(list of stirngs) *should* give me a random string and rand(range of numbers) *should* give me a random number in that range! No one would write an academic paper and define a new rand function for each input because well...it's clear what the user wants - rand of something.
Just in case it confuses anyone else: this one is supposed to be `rand` as well, not `and`.
likewise if you are solving differential equations, DifferentialEquations.jl is hugely better than any free alternative I know of and arguably better than paid packages. The broader SciML ecosystem that's built up around this has a lot of cool stuff in it too.
other than this it seems like you wouldn't care about the other potential advantages, and might be more put off than average by the disadvantages and occasional rough edges.
> unless it has interesting packages/functionality that my current toolset does not
Multiple dispatch, code-specialization, JIT compiling, automatic loop fusion, broadcasting, many ways of doing compile-time optimizations.... I'm sure there's more.
But I guess you can dismiss these features the same way that you dismiss "the code auto-differentiates" or "it's faster".
What I always tell people is the following:
If you are writing code using existing libraries then use whichever language has those languages. The NN stack(s) in Python are great, the statistical ML stack(s) in R are simple and include SOTA techniques.
If you are writing a package yourself, then I assume you know the core of the idea well enough to be able to write your code from the "top down" i.e. you're not experimenting with how to solve the problem at hand, you're implementing something concretely defined.
In this case, and tailored to your use, I would argue that Julia has more advantages than disadvantages, especially compared to R or Python. Here are a few comments:
1. Environments, dependencies, and distribution can all be handled by Pkg.jl, the built in package manager. There is no 3rd party tool involved, there is no disagreement in the community on which is better. This is my biggest pain point with Python.
2. Julia's type system both exists and is more powerful than that of Python (types or classes) and R (even Hadley's new S7(?) system). By powerful I mean generics/parametric types and overloading/dispatch built in. You can code without them, but certain problems are solved elegantly by them. Since working heavily with types in recent years, I find this to be my biggest pain point in R and I wouldn't want to write a package in R, although I like to use it as an end user.
3. New developments in scientific programming, programming ergonomics, hardware generic code (as in this post), and other cool features happen in Julia. New developments in statistics happen in R (and increasingly Julia), new developments funded by big companies happen in Python.
4. The Python and R interpreter start up faster than Julia. The biggest problem here is when you are redefining types, which is the only thing in Julia that can't currently be "hot reloaded" i.e. you need to restart Julia to redefine types.
5. Working with tabular data is (currently) far more ergonomic and effortless in R than Python and Julia.
6. Plotting is not a solved problem in Julia. Plots.jl is pretty easy and pretty powerful, Makie.jl is powerful but very manual. Time to first plot is longer than R or Python.
7. Julia has almost zero technical debt, R and Python have a lot. Backwards compatibility is guaranteed for Julia code written in >v1.0 and Pkg.jl handles package compatibility. If I send you code I wrote 4 years ago along with a Project.toml containing [compat] information then you could run the code with zero effort. (This is the theory, in practice Julia programmers are typically scientists first and coders second, ymmv.)
8. You can choose how low level you want your code to be. Prototyping can be done in Julia, rewriting to be faster can be done in Julia, production code can be done in Julia. Translating Python to C++ production might mean thinking about types for the first time in the dev process. In Julia, going to production just means making sure your code is type stable.
It reads and writes a lot like python (but nicer IMO), I don't think the learning curve is immense to try it for small optimizations. And it's also not unreadable so other people can verify your code
JuliaCall[1] and RCall[2].
Python<->Julia is similarly well exercised with PyCall[1], and recently PythonCall[2].
[1] https://non-contradiction.github.io/JuliaCall/index.html [2] https://github.com/JuliaInterop/RCall.jl [3] https://github.com/JuliaPy/PyCall.jl [4] https://cjdoris.github.io/PythonCall.jl/stable/
Moreover, I think sometimes people get their PhD and think they deserve to use the tools they put in their toolbox, on the problems they focused on, and don't see that all they really did was get a ticket to the game. Most scientists have a phd. Most scientists don't work on the thing their PhD is about ten years later. The sooner you open up to that the sooner you will get out of the postdoc chase and get a job that is a lot more rewarding (both intellectually and financially). All this means that there may be problems you are going to learn about and focus on that you never thought you would at some point, and being open to that and seeing it as an opportunity will carry you further then not.
Why are you asking to be convinced if you don't want to be convinced?