Rust vs. Julia in scientific computing
mo8it.com
mo8it.com
Briefly, for scientific computing, rapid iteration, prototyping and redesign is one of the main requirements. So is "open internals", i.e. generic code and/or the ability to open up someone else's code and reuse parts. Speed is somewhere further down the list - which is why Python (and previously, Perl) are the dominant languages, not C++ or Rust.
Julia excels at all of these and as such is nearly perfect for science, whereas Rust is unusually bad at the "prototyping and redesign" part. Like, you change the ownership model of some struct, or some single type, and now you have to unravel half your code base to change type signatures and fix borrow checker woes. It's also just slow to write.
Rust is nice for scientific code that is more "application-like", such as command-like tools whose task is very well-defined and mature, as well as for large-scale scientific appliations. It's also nice for very low-level code that needs to be close to the metal (e.g. scientific code running without an OS inside scientific instruments). But in most scientific use cases, dynamic languages are just way better, as they have been the last 30 years, and Rust doesn't change that, nice as it is.
1. https://discourse.julialang.org/t/blog-post-rust-vs-julia-in...
2. https://discourse.julialang.org/t/blog-post-rust-vs-julia-in...
3. https://discourse.julialang.org/t/blog-post-rust-vs-julia-in...
Nonetheless, I've found it to be better than Python for my use cases. I also hear that Julia's GPU capabilities are excellent
Julia is dedicated to scientific computing, where it is very normal to start out with solving a simple problem to verify that your approach can work. That code is usually very bad, because all it needs to do is function. The goal of Julia is to:
- Make writing that first prototype very easy
- Use that prototype to create a fast version, without reimplementing everything
That is what rust has to be compared to. And while I absolutely do like what rust is, it is not a language that lends itself to quick prototyping. For the prototype you don't care that your program might crash with the wrong parameter or that the generated matrix is not invertible or that you access an array out of bounds or ... By any measure rust is a complex language, which takes significant understanding to write and requires the programmer to engage with the code alot.
Maybe it depends on your perspective, or maybe it is simply personal preference, but I find rust pleasant and quick to prototype in.
Or, perhaps more importantly, on use case.
It’s hard for me to imagine (though certainly possible, I suppose) someone finding prototyping e.g. sensor processing algorithms in Rust more enjoyable than Julia, or faster for that matter.
To be fair I'm not familiar with Julia so I'm not making any comparisons.
The upshot:
1. If you need to eke out that last extra drop of performance, and can't deal with any mistakes, Rust is good because it makes it very difficult to write slow code. Julia makes it easy, because Julia needs to be easy to write.
2. Julia's greater permissiveness means the compiler can't catch as many bugs as Rust's (although it still catches far more than Python's static analysis tools or C, where every other line has some undefined behavior).
3. If you need interactive+dynamic code, Julia is your best bet. It's a lot like Python in this regard, but with a much better UX in the form of a better REPL, package management, etc. Well-written Julia code will also be just as fast as Rust or C++, meaning you can use it for high-performance computing.
4. If you want a scientific/machine learning ecosystem, go with Julia.
I think this post is strongest in hitting Julia where its problems are -- Julia can be a bit too permissive/promiscuous with letting you do dumb things.
Where it's weakest is letting Rust off the hook a bit too easy. Rust is good at making it hard to write bad code, but the cost is it makes it hard to write code in general, unless you handle every detail that might slow your code down. You're not going to get users happily writing Rust when they just want to make a plot. It takes about 5-10 lines of Rust to create one line of Julia code.
Personally, I have found the package management to be somewhat more difficult to navigate than Python's (which I also have little love for...). But perhaps this is a learning curve thing...
But that is probably the most important thing. I don't have time to re-write 40+ years of code in a new language just because it's trendy or safer or whatever. I have new science to do!
In some ways science suffers because of this, but it is also nice to have a relatively common ecosystem and institutional knowledge that doesn't change every 3 years.
In the short term it is of course convenient to just use the most popular thing always (in which case we'd all still be using PHP and Flash), but eventually things do switch on a large scale, and it benefits all of us to push this along when we can.
The more we double-down on an inherently sub-optimal ecosystem, the more we are trapped by it.
Imagine if all the effort making Python usable over the last decade had instead been spent on giving a compiled language better dev UX and GPU support...
I'm not even a fan of Julia, but I probably would be if it had received the same attention Python has for the last ten years.
Conversely, I'm _still_ not a fan of Python, even after all this time and effort has been expended, because the foundation being built upon is just simply a bad one for high performance and high security domains. Anything they manage to get working is akin to a hack, and is working in spite of the language in which it is built.
It's one thing to compare languages in the abstract, for e.g., to compare and contrast iteration or exceptions or whatever. It's a different thing to compare languages for doing something specific -- in this case, scientific computing. If someone want to get something done in a realistic ( or affordable) time frame, then things like like libraries, community, and documentation become especially important.
1.) The rewrite never seems to have all the features of the original (for many reasons), so you end up keeping the original around because science can be very niche. Now you have two or more packages to deal with.
2.) There is always a better language. Science (or at least parts of science) have switched before - from Fortran to C to C++ to Python. Some of the gains have materialized (some safety, performance, borrowing stuff from outside science). But it has come at a cost (language fragmentation, and also packaging is an absolute shit show right now, partly because #1).
But I'm sure the next batch of languages will finally solve all our problems once and for all, and we will never have to switch again.
(In general, I am talking about non-AI type science. I am a computational chemist, and our code really does date back > 40 years at times. That is not always a bad thing).
Sure and that's reasonable, but in the cases where performance/throughput/training time/RAM usage is the main limiting factor of a field, suddenly this stuff matters a lot. There's plenty of useful code written in COBOL and Basic from back in the day, but you don't see people using those things to train LLMs
Imagine if instead of letting Basic effectively die, we had improved it with a myriad of extensions to the point where you can run LLMs and GPU code and things in a performant-ish way on it. Now replace the word Basic with Python
This is honesty very 1990's brained, based on the idea that compute power is expensive and constrained.
All Turing complete languages can theoretically do the same things. That doesn't make them equally useful across all domains.
As long as developer hours are much cheaper than compute hours, we're going to be doing certain types of work in high level scripting languages.
This is honestly very 2015 brained. Compute power is back to being more expensive and constrained than developer hours. :)
There are many applications where the algorithms are written by a specialist which is then compiled into a static binary for use in another application or UI. This is why FORTRAN C and C++ are so popular in the scientific computing field for writing numerical algorithms. It's not just for speed, there is clear separation of concerns here.
Julia is still far behind here as you can't compile to a GC free static binary yet. There is PackageCompiler.jl but it appears it makes large binaries.
Scientific analysis and plotting is just one aspect, ultimately your algorithms would have to be distributed for use by others.
Fortran, C and C++ are popular due to the way they allow to explore HPC infrastructure, which Rust is still years away to support, and Julia is already ahead in that regard.
MPI, SIMD, OpenMP, NUMA algorithms across the computation cluster.
As for Julia on Cray,
"Julia — The Newest Petaflop Family Language We Have Started to Love"
https://www.avenga.com/magazine/julia-programming-language
> Julia is one of the few languages that are in the so-called PetaFlop family; the other languages are C, C++ and Fortrant. It achieved 1.54 petaflops with 1.3 million threads on the Cray XC40 supercomputer.
It seems to be pure Julia.
Julia uses the LLVM compiler; I’m pretty sure that’s the only platform, aside from experiments compiling to WASM.
It's my understanding that Julia aspires to join this group of languages if it is able to do so, which is why the Petaflops announcement was originally enticing to me, and then became somewhat less so once I learned that it was relying on MPI.
And Julia code is not Fortran/C/C++, not sure what you are asking.
I said "Can Julia take advantage of that?". So the "that" in my question was "vendor compilers [which] often outperform GCC and LLVM... on Cray systems especially".
It seems like the answer is, no, Julia cannot be compiled with those vendor compilers that often outperform LLVM.
But it seems that Julia nonetheless has been shown to perform well, just not specifically by using those vendor compilers that I was asking about.
https://www.openmp.org/updates/openmp-accelerator-support-gp...
Frustrating article. Not only does it levy unfair criticism against Julia, it misses on one obvious legitimate complaint.
Rust can be compiled today, no caveats.
In the lab where I work there are plenty of people real good at the physics/maths but not that good at writing code. Since they need performances to run their stuff, they often go to Fortran (because that's what has always been used here). How much Julia would make their life easier ?
(I'm the computer guy in the lab: I optimize their stuff when they need it but I'm not in the mindset of "model first, code quality after", I'm much more a rustacean :-))
I wouldn’t advise rewriting a substantial Fortran project in Julia; but for new projects, Julia is the better choice. Fortran is excellent in the scientific/numerical high performance niche, but it’s still awkward to move beyond that; it’s still stuck in the Formula Translator role. So the non-numerical parts of the program, that deal with input/output, for example, are awkward; Julia is more pleasant to program these tasks.
Julia lets you build your program interactively, by trying things out in the REPL. This alone may be sufficient reason to prefer it over Fortran.
Although most scientists won’t become sophisticated as programmers, for those that will, Julia can grow with them. It has extensive support for metaprogramming (for example), including real macros. Its type/dispatch system, and other features, allow you to write far more concise and better organized code than you can in Fortran. Sharing code and composing libraries² is easier in Julia.
Really, Julia is just more fun to program in.
1 HPCWire (2017) ‘Julia Joins Petaflop Club’. Available from: https://cacm.acm.org/news/221003-julia-joins-petaflop-club/f...
2 Phillips, Lee (2020) ‘The Unreasonable Effectiveness of the Julia Programming Language’. Ars Technica. Available from: https://arstechnica.com/science/2020/10/the-unreasonable-eff...
More generally, saying that programs written in a language that happens not to include a feature you think is important (in this case, interfaces) will “always suffer from correctness problems” doesn’t seem to be a serious point of view.
https://docs.julialang.org/en/v1/manual/calling-c-and-fortra...
So unless you think the Julia version will compile to faster code than the Fortran code (probably not if you use a fancy paid license Fortran compiler), then definitely don't rewrite.
Python didn't become the Lingua Franca of Data Science and Analytics because people on Reddit, Hacker News, the blogosphere etc, pushed Python and wrote presuasive pieces on why it was so awesome (there was some of this, but absolutely nothing compared to the cult of Rust). It suceeded becasue it turned out to be a very good tool for that particular job.
I don't know about others on HN, but the constant push for "Write everything in Rust" has really turned me off from the language. I know it's petty, but I now tend to associate Rust with being nagged and harangued.
It turns out that people write posts about things they like. Sometimes they like things for good reasons, sometimes they like things for bad reasons. But that has no direct bearing on how good a tool is.
It was indeed through a significant amount of writing and advocacy and persuasion that Python - through numpy and scipy and pandas - won out here.
For a time it looked like it was going to fade, as it was too slow and all the "big" tooling was built on the jvm instead. But it got faster and people made more interfaces to underlying faster components, and people did more writing and advocacy and persuasion, and now it's mostly python still, with a bunch of native or jvm code under the hood sometimes.
But none of this happened just through python being clearly better for this niche. People always write and discuss and advocate for the tools they like. It's normal. It's same as it ever was.
People are always talking shit about things to make people think they are smart, like people around me think they are Larry Wall, bashed Java for being slow, but they don't know that a lot of important, performant thing are coded in Java.
If you want raw performance, use C/C++. If you want simplicity, then use Python. If you want reliability and safety, then use Java/Go.
There must be a reason why C, Python (and Erlang/Elixir) is popular.
This doesn't mean, however, that Julia doesn't solve the 2-language problem. There are large classes of problems and applications — especially in scientific computing — where a complete static analysis isn't required to move into production. It is definitely helpful for maintenance and security (and a requirement in some industries), and Julia's static analysis tooling is continuing to improve... especially with ongoing investments from key companies pushing us towards this goal.
Rust is a great language when you know exactly what you want to write. When you know exactly what the inputs are and what the outputs are and what the algorithm is. And it's really painful when you're not sure. Pick the right tool for the right job!
False:
(0..10_000).into_par_iter().for_each(|_| {
counter.store(counter.load(Ordering::SeqCst) + 1, Ordering::SeqCst);
});
println!("counter {}", counter.load(Ordering::SeqCst));
Really the premise of Julia is to make their life easier, while leaving yours equivalently easy (or better).
But this introspection gave me a reality check. In my own work, as much as I love Julia, I fallback to quickly using Python libs. In industry, most of the people's focus is on using someone's already written code. The byproduct is inheriting Python and C/C++ interface mess around dependency management. People needing to write their own algorithms are majority academics and researchers where Julia has flourished.
The incentives of corporate work are setup to build on top of Python's ecosystem which means (sadly) Julia will stay in its niche.
Julia’s proposition is that the entire community should be able to read and contribute to these libraries. Unlike the python community where the vast majority of users could not read the source behind those libraries let alone contribute.
A fresher take on scientific computing like Julia, if it is mainstream, might enable more contributions and in general understandability of the black box algorithms.
Not knowing rust, a lot of this material looks like it would have high initial investment (to learn rust) and an ongoing variable cost to implement (ie, figuring out all of these complexities in a particular use case). There is surely a trade off on accuracy but everything in science is about trade offs.
If you can publish three papers working in julia while the rust programmer gets through only one… we know who is going to get tenure.