Why scientists are turning to Rust (2020)
nature.com
nature.com
It's a weird article. It seems more like a Rust sales pitch than anything.
Programming is a niche. Rust is a, albeit amazing and gaining traction at a fast pace, niche in this niche.
99% of mathematicians, physicists and biologists I have worked with have not even tried programming yet. And if they do, they will start with the stuff their friends are already using: python, fortran, etc.
Hell, 99% of the dev have not heard of rust yet. I go from companies to companies, every time I talk about rust, I have to explain what it is.
It's not even living in a bubble if you think that, it's living in an bubble from another dimension.
Having generalization from anecdotal evidence in nature.com is kinda ironic if you think about it.
Given that, I highly doubt any of them are even aware of languages beyond python and java and maybe they're aware that C and C++ exists but are scary.
Real Scientists use assembler, C, and Frink! ;)
Sad to say the engineering firm I worked at for 6 years was filled with 20-something and 30-something engineers with only one of 15 of them able to code. Excel use, sure. Coding, no. It wasn't that way 25 years ago.
Which is a very small part of the overall "scientist" world I think, who are probably not programming, or who may use matlab or python.
I just looked, we (a major UK university) as well as having a Computer Science department to teach programming to obviously Computer Science and Electronics students as well as related disciplines such as Mathematics, other departments have their own staff teach programming for Physicists, Chemists, Psychologists, Geographers, and a bunch of social scientists who need Statistics. I just asked the API for courses about "programming" and then checked they didn't mean some other type of programming. So if say Law has a course named like "Coding in Javascript" for some reason I wouldn't have seen it.
The most obvious numerate discipline I thought of which doesn't teach programming was Medicine, and I guess they just couldn't find time to add yet another entire discipline's knowledge into the timetable.
I tried Rust, but I am using SPARK2014/Ada for my needs. Easier to learn and has the formal methods/software integrity checks and other niceties. I started learning SPARK2014 while reading the book, "Building High Integrity Applications with SPARK". It was used by the CubeSat lab in Vermont [1], which the book ties in nicely. I can see programmers flocking to Rust, but scientists are not typically computer scientists, and mainstream shouldn't be a highly-weighted metric. I find Rust a lot more heavy than SPARK/Ada. I've tried Julia too, but you can write systems level code in SPARK/Ada too. The adacore site has a lot of great projects and articles on embedded software, drone low-level code rewrite in SPARK, etc.
The partnership between AdaCore and Ferrous Systems is very exciting [2]. I hope to be able to stay with SPARK and gradually learn Rust with this partnership when Rust has a bit more under its belt with real world embedded, high-integrity software like Ada's legacy.
[2] https://ferrous-systems.com/blog/ferrous-systems-adacore-joi...
> Köster, now at the University of Duisburg-Essen in Germany, was looking for a language that offered the “expressiveness” of Python but the speed of languages such as C and C++. In other words, “a high-performance language that is still, let’s say, ergonomic to use”, he explains. What he found was Rust.
He must not have looked very far. There are much better options if you're looking for the expressiveness of Python. Personally I'm very fond of Nim.
Edit: others mentioned Julia, and it indeed looks like a valid choice since they don’t seem to program embedded or safety-critical stuff.
I used to write games in C++ as a teenager, and using it daily was fine. Now I find, every time I try to write something in it, it's a massive pain, as I forget the myriad of tiny rules, the very specific ways to write types etc. And that's before I even think about dependency management.
Is Rust substantially better at this? When I read frustrations about Rust, it seems to be mostly about the Borrow Checker... does that affect "pedestrian" code, like "take this 4-dimensional array and munge it with an arcane for-loop"? Or, here's a basic immutable class with some fields and some methods, make it available in Python?
I'm liking the look of Nim because it is concise, safe and fast-ish, so a good companion to Python. I dislike C++ as that companion because, even though once you've written your code, it integrates very well with Python, it's a lot of pain to get there. Where would Rust fit in this picture?
Unfortunately, the reason why you hear complaints is because it does affect even normal code. Writing code in a setting where you have a strict type system is like writing a mathematical proof. You know that the thing you want to do is safe/correct/valid, but you have to find a way to write it formally, which is itself kind of a puzzle.
Don't get me wrong, I think stricter type systems are the future of programming, except maybe for very high level stuff like Python and Js, but there's definitely some friction involved when the language is a "puzzle language" that insists you do things in a specific way.
With Python or C++, most experienced programmers will be able to bang out just about anything.
With Haskell, Rust, Idris, OCaml and the like, there's this additional step of writing a very detailed math proof that your code is actually allowed. Which makes a lot of sense when you think about those languages' goals.
> Where would Rust fit in this picture?
There's PyO3, not as stable as pybind11 but it's getting there. Honestly Python is just great for stuff like this. You have a lot of ways to gradually speed it up (types -> cython, C++, Rust). It's about as elegant as an octopus made stapling extra legs to a dog, but it's hard to argue it's not effective.
I think the trade off you often get with Rust compared to C++ is that easy things get harder to do and harder things get easier to do correctly. This won't be true for every problem or person but I think it's a fine rule of thumb. Python is a lot more straightforward than either for most problems of any complexity but of course you don't get the same performance potential where you're not grafting it onto a different language.
The borrow checker is an issue when you are a beginner and have to unlearn all the bad practices you were doing in C/C++.
Also, as a person interested in Rust, but who hasn't yet found the time (and strength of mind) to really commit to learning it: the main difference between C++ and Rust is the cruft that C++ has accumulated over the decades, which Rust doesn't have because it's much younger. But seeing the pace of new features introduced into Rust, I'm a bit worried that some of the older features will soon be declared "non-best practice" and turn into C++-style cruft.
Let's take two serious C++ proposals I've looked at over the past years and imagine if it had Editions (it doesn't, Vittorio's similar Epochs proposal was shot down and won't be coming back). First a try operator. C++ doesn't have one of these, it would be usual to spell it ? but that's prohibited in C++ by the existence of a ? character as part of the ternary operator ?:. If you had Editions you could abolish ?: in a new Edition or change the way it is parsed so that some weird utterances now don't work because ? is the try operator, without them the proposers instead reluctantly suggest ?? (which is already an operator in some languages and it isn't try...)
Second, pattern matching with _ as a wildcard in the pattern. C++ doesn't do that today. Unfortunately _ is a valid identifier in C++ and people use it. Not a lot of people, but enough for it to be a real pain if this stopped working. So the proposers reluctantly suggest you could write __ in this case. If you had Editions you could forbid _ in a new Edition and in that Edition have the nicer _ while __ is provided in older Editions if they want new pattern matching.
Editions only cover syntax (they're actually mutating the Abstract Syntax Tree at compile time) but because this affords some great opportunities for backward compatible changes, it encourages people to get creative and do even more. For example Rust's arrays learned to be IntoIterator. But that's not a syntax change, so how did that remain compatible? Well, there's a hack, if you have an edition prior to 2021 Edition, then one function from IntoIterator is hidden from resolution by the hack, so that array.into_iter() does what it did in your edition before arrays were IntoIterator. But every other feature works, so you can write "for thing in array", you can pass arrays to functions which need an iterator, and so on, and in newer editions the hack isn't used so array.into_iter() does what you'd expect given arrays are now IntoIterator.
Is it just because you see additions of new features in general or there are some specific issues? For me new features do not make 'bad practice' from old ways. As example in older versions you had to be more explicit with lifetimes, but that is just convenience.
The pace of change in C++ considerably higher. Rust has regular releases, but the changes to the language tend to make it simpler to use as they remove limitations and improve the std.
Also it is not just cruft even if you ignore all the safety that Rust provides by default. The package system and ease of compilation alone makes the jump worthwhile.
I think language complexity in general is not a problem; language consistency is. I find Rust complex, but not inconsistent or surprising (with exceptions). I'd say that yes, it is substantially better in this regard.
Regarding the "pedestrian code", it depends. Rust is very unergonomic for algorithms, due to its handling of bidirectional references. However, some algorithms/data structures may work fine; the matrix operation you refer to can be very easy to work with, if the elements are Clone/Copy, which numbers are.
I'm a Rust fan, but the article seems to be a puff piece; it takes one case and generalizes it. I also think that Rust is not productive [than other easy, compiled, languages] for generic programming, at least, without significant experience. Relatively small programs (I don't know the size of the typical scientific programs in academia) may be easier, but once a program grows, it definitely requires an amount of design (due to ownership) that other languages don't require.
With C++ I do find human medium-term cache capacity to be an issue. By the time I use C++ again, I forget some of the many little things I need just to compile a glorified "fizz-buzz". Things like, is this a pointer to a const or a const pointer? What's the syntax for a move/copy constructor? Keeping headers and cpp files in sync. What's the flag to enable modern C++ extensions? How do I write this lambda? Or make a pointer to a function? And so on...
Agreed. I write mostly in Rust, but I don't recommend it unless you have a hard problem. If you need C/C++ speed, and there's enough complicated state coordination that you need safe threading and allocation control, Rust is useful. For typical webcrap, use Go. That's the job for which Google created Go.
- Things like Cython... let's just say experience is mixed. IME it is very hard to reason with its performance, and quickly it gets about-as-complex as writing your tight loop in a "natively fast" language.
- C++ is the standard Python companion, and frankly it can integrate amazingly well. You can define types in C++, native C++ functions, and define conversions between C++/Python. Then you can seamlessly make a C++ object in Python, pass it to the function and get the type back. It's not just about the speed, but also about the ability to really merge two languages together. BUT getting to that point is a massive pain of template errors.
- You could do it all in C, manually incrementing and decrementing refcounters. I'll leave it there.
- Other languages like Nim, Fortran and who knows what else seem to offer a limited ability to merge the codebases. You can use them, it seems, as a crutch to patch up some Python slowness here and there, but not as a comprehensive "here's two codebases that live side by side".
- Rust..? Might it be a replacement for C++ here? And is its complexity manageable.
Given that, I'm curious what bits of C++/python interop are easier.
I'm mostly asking because my experience of nim-python interop consisted of: Compile nim module with nimpy imported. Put compiled nim module somewhere where your python can find it. write `import <Name_of_module>` in the python file Done, you now have access to the exported functions and symbols defined in the nim module.
Is it that C++ wouldn't need any additional imports to make it work that makes it more seamless?
In C++, you'd write your C++ code as if you don't care about Python. Then, you add the "glue" code, which tells C++ how to convert objects to and from Python (there's more options than that but let's leave it here). That, plus a lot of boilerplate. Then, in effect, you can pass objects back and forth between Python and C++, create objects in C++, return them to Python, pass them back to more C++ functions, and so on.
I haven't used Nim+Python; but looking at Nimpy documentation, it looks like, for example, to export a class from Nim to Python, it needs to inherit from a special underlying class. You can't just take a class and make it available in Python. And even then, can you pass that back to Nim to call a function with?
Maybe yes, and if so, that bumps Nim higher up my priority list.
In terms of seamlessness, I can not confirm that it is exactly like having a python class, merely "fairly close". If you define a type, you can pass it to python, python can instantiate (with the typical way of Type() ) or even receive instances (if you define e.g. a constructor proc yourself). It can pass the instances back to nim to work with no problem (see e.g. quick example I whipped up: https://pastebin.com/v5wAqGxZ).
However, I have found that there are differences of a nim-type to a pythonclass:
1) In python you do not have direct access to the fields of an instance of a nim-type. You can access them via getters and setters, but you'll get runtime errors if you try to access them directly. To make that behave seamlessly you'd need to define a python wrapper class and hide stuff with @property decorators. Without such a wrapper class you could also not add a field to an instance of a nim-type like you could a normal python object.
2) An imported nim-type can not be inherited from in python (not sure if that works in C++, but at least it's a nim-python-interop limitation).
If you wanted to make a library in nim available in python that wasn't originally intended for the export, you'd likely also need to write a glue-layer in nim. The efffort there would consist mostly in copy pasting the proc signatures you want to make available and annotating them with the export-pragma, copy pasting the nim-type to make it inherit from the type needed for exporting and converting from the exported nim-type to the imported nim-type or vice-versa when jumping from python to nim or vice versa. Further, to hide the short-comings (that you can't inherit and that you don't have access to fields or can add fields) you'd need to wrap the nim-type in a python-class as described in 1).
My conclusion from what I heard from you so far and what I know of nimpy would be, that nimpy is 90% of the way there, workable but not as seamless, given that exported nim types and their instances aren't functioning exactly like python classes and their instances. Further the fact you'd need to wrap them in a Python class to hide that means, in comparison, extra work to C++ (though that may balance out with if the glue layer in C++ is more involved than the one in nim, I don't know).
One notable difference, though: in C++, if you forget a detail, you might get a compilation error or you might get a runtime crash and need to pull out something like valgrind. In Rust, if you forget a detail, you're extremely likely to get a compilation error, not a runtime crash.
Tbh most of the time the borrow checker was easy to use, and at the time (circa 2014) it was allegedly harder to use than it is now, according to a friend who still use it professionally.
>But for many Rustaceans, the human element is equally compelling. Hauck, a member of the LGBT+ community, says that Rust users have gone out of their way to make her feel welcome.
A welcoming environment for LGBTQ+ people has nothing to do with their genitals, and everything to do with them being able to be genuine and open about themselves without consequence.
My impression is that at least a number of scientists don't know and don't care.
R >>>>>>> Python >> Julia > Everything Else
This of course ignores that a bunch of the underlying code for both the R and Python things people use is written in other languages, because they'll never really look at that code.
In other words, a step-by-step developer growth process from simple to hard, maybe 2-4 levels, governed by a Cargo.toml setting probably (comparable to a compiler switch), and you never have to throw away what you already learned. Like maybe "learn once, do anything".
(Yes, compatibility with peer group is probably a larger factor. But as an option for some where it fits.)
I think even Rc might be a barrier, because it looks like more complexity than Pythonistas expect.
I still check _Are we learning yet_ [3] to see their recommendation on the state of ML in Rust. It still lists "the ecosystem isn't very complete yet.", which is how I felt at the time!
[1] https://github.com/google/evcxr/tree/main/evcxr_jupyter
[2] https://datacrayon.com/shop/product/data-analysis-with-rust-...
I like Rust but all the scientists I know work with C++/Python (PyTorch) nowadays due to that.
Rust is totally irrelevent in this context.