Unfortunately I'm from DataScience field, so I cannot see much motivation to learn Rust, but I am considering learning it, because language itself seems exciting!
Is there anyone on HN who is from DataScience field like me and has learned Rust? It would be much appreciated if you could share the experience.
I worked on data science, computational science, some system engineering and embedded projects. I tried Rust on all of them but it was valuable only for the third.
In data science the focus is more on speed of development and as much as Rust is more enjoyable to use than C++, it doesn't match scripting languages (namely Python) for the flexibility you get, the easiness with which you can adapt to changes and the lack of a need to focus too much on machine-related details. This is the least suited field for Rust between the one I mentioned, in my view.
Computational science needs to focus on algorithms and formulas, Rust can hide them a bit too much under "unwrap"s "iter"s and so on. Plus still no stellar library support and (my) experience with other tools (namely C + OpenMP, Julia, C++, numpy...) made me feel using Rust for that was unnecessary and slower.
When you are designing systems, however, that's where Rust is a complete game changer. Honestly I don't think C++ can stand a chance in its current level. Everything, from the package system, to the borrow checker, to the safe threading model, makes you pity your C++ ancestors for how hard they thought this kind of programming had to be. Rust makes it orders of magnitude more approachable, saner, and with better results too.
Embedded... it's nice but honestly all the dark arts of unsafe Rust are not standardized so it doesn't feel as future-proof as C already.
Rust threading is only safe between threads accessing data structures in process own memory. It does nothing for shared resources using OS IPC, or external resources shared among threads.
It will certainly improve, as hopefully C++ will, even it never gets 100% as safe as Rust, the ecosystem has too much weight for decades to come.
"Only". Intra-process shared memory is by far the most important kind of shared memory. 80% of the time you see someone advocating for message-passing multi-process over shared memory multi-thread concurrency or parallelism, it's because shared mutable memory is incredibly hard to get right in just about every mainstream language except for Rust.
> It does nothing for shared resources using OS IPC, or external resources shared among threads.
Only partially true. The very same language constructs which make shared memory safe can also be used to create safe wrappers for other shared resources.
Language constructs only help if there are no other applications accessing the same resources.
There's a huge world outside of microservices.
> Language constructs only help if there are no other applications accessing the same resources.
You're mistaken. Rust's language constructs can often help you write safe wrappers around inter-process communication and synchronization mechanisms. Of course, all bets are off if the other processes don't adhere to the agreed-upon IPC protocol, but that's not the point. The point is that Rust makes it easier to not accidentally screw up the protocol yourself.
In my opinion Bevy may be one of the better ECS implementations I've seen, as ex-gamedev it's just very well thought through. It's never going to supplant Unity or Unreal but that's less of a language issue than just the scope of those codebases.
You're seeing a lot more of it showing up in OSes across Android, Linux and I've seen a fair bit of movement in the windows space as well.
As for IPC, C/C++ doesn't help you their either, that's more of a framework level concern. If anything the threading model helps when you're doing concurrent IPC dispatch(ex: MTA COM or equivalent high throughput IPC) and preventing data races.
The rust data science ecosystem itself is starting to come together with projects like Polars [2] (Pandas alternative), nalgebra [3], Datafusion [4] and Ballista [5]. Of course nowhere near Python but that is hard to beat today.
[1] https://github.com/PyO3/pyo3
[2] https://github.com/pola-rs/polars/
[3] https://github.com/dimforge/nalgebra
Of course the ecosystem isn't as diverse as python yet, but if you're willing to call a bit back and forth between rust and python it's great.
Yeah me. I’m data science/data engineering.
Wrote and debugged a lot of R and Python, fair bit of Spark too.
Learning Rust was fine. Not trivial, but definitely worthwhile. I work on more “data systems” than once-off scripts and analysis in my work, so the performance and correctness of Rust has proven more valuable to me than the flexibility and development speed promises of Python. The former lends itself to difficult to maintain, hacky, patchwork code, and the latter falls apart after the nth time you have to fix some Python code in prod that’s failed 3.5 hours into a process, because some random package has thrown some as-yet-unseen error.
There’s some interesting stuff happening in the data space in Rust-polars data-frame library gives you pandas like functionality, but with better performance and type checking, the data-fusion stuff is making strides as well. There isn’t a 1:1 replacement for all your ML libs, but it’s getting there! Linfa and SmartCore are making strides on the sklearn-like space.
Would you use Rust to replace your ml stuff in Python at work tomorrow? No, probably not just at the moment. Is it worth learning because it’ll make you a better dev, and give you more options and tools the next time you have a non-standard problem to solve? Yes, definitely.
But: I'm not well versed in Python Pandas either, and my rust is better than my python (though Ruby is my daily language, so I'm very used to dynamic languages). The thing I miss most is the rapid REPL workflow in Python, where I work out the pipelines, filters, map-reduce and so forth, on the interactive REPL and then extract that into python scripts.
The polars scripts that I worked with, were small enough and the builds are incremental, therefore fast: milliseconds. But the build-step inbetween is somewhat clumsy when exploring data.
But the end-results are just so extremely cleaner, saner and especially faster that it certainly is worth it. I worked on a dataset where crunching it in python took upwards of 14 hours, and the polars under an hour. That feedback loop alone is worth days of fighting the borrow-checker. And that immediate feedback by the compiler/checker "this column isn't guaranteed a String, it could be missing, could be unparsable; deal with this now" is priceless. Too often did I have some ETL run for 8 hours, and then choke on some column that I swore should've been UTF8. Fix it, and re-run. Wasting 8 hours. A type-checker in ETL/data flows is priceless.
Edit: to be clear: I am developer. Not datascience. But I do need to understand my users, my business, my market. Developer/devops today do need to crunch large datasets: at least I do.
I ended up writing a book, "Data Analysis with Rust Notebooks", where I demonstrate using Rust + Jupyter Notebooks for tasks I'd normally use Python for. It's a good experience, although there are some additional limitations when using Rust as a notebook kernel.
[1] https://datacrayon.com/shop/product/data-analysis-with-rust-...
If your use case is to replace entire tools, and your team isn't familiar with static languages, then Go is probably the better option. There's no denying that it's much easier to learn.
[0] https://pyo3.rs/
But I wish there were a language that solved the two-languages problem (fast vs. easy to work with). I find Rust quite verbose.
That's the whole pitch of Julia.
That is the motivation and purpose of Julia.
For some people's workflows it already seems to have solved the problem, but in general the "easy to work with" part is still in progress IMO. Some of it is the (in)famously annoying compilation delays, but a large part of it is just a tooling and ecosystem thing (both of those sides of the issue are being actively worked on).
The difference from C/C++/other low level languages is that it's not the language itself that's difficult to work with, it's the current tools and workflows that are suboptimal. The language is pretty well-designed and fun to work with, so there's hope that it can actually solve the two-language problem for a wider variety of devs and their needs.
Rust itself can be made "easy to work with" at the cost of boilerplate. You end up with code that's littered with idiomatically superfluous uses of things like .clone(), interior mutability, Cow<>, perhaps even such exotic features as the Any trait for encoding "dynamic" behavior.
The flip side is that it's easier to refactor this code and make it more correct and performant; "simply" replace the boilerplate with proper Rustic code and deal with the resulting compile errors. It's a very viable strategy that has no direct equivalents in other languages, not even in C/C++.
Rust has a number of features (ergonomic and technical) and some really nice language design that permits writing quite “high-level” code. Personally, having a compiler and type system as powerful as Rusts makes it worthwhile alone.
With faster compiler toolchains and REPL like tooling.
In my space (data engineering and associated systems), Rust is a much more palatable and easier sell than an Ocaml or a Haskell.
> With faster compiler toolchains and REPL like tooling.
TBH I don't really miss a REPL, I know a lot of people love it, but between the type system, tooling like RA, and how Rust makes writing tests _so_ frictionless, I've not really had any need to touch a REPL in the last ~2 years. I've probably been abusing the frictionless test functionality to write little snippets to test out using a new library or tool, I certainly find it a lot manual work than the "type multi-line snippet into repl. Make a typo. Start it all over again. Discover that part x actually wants a y. Write it all over again. Repeat until correct. Painstaking trawl through repl history to find the set of code that actually does what we want and lift it into our actual code base" experience I have with reps-based development.
Rust has a great role to play, replacing C and C++ in all performance-critical and security-critical underpinnings of modern computation (from kernels to UI toolkits). But higher up the stack, where those concerns are outweighed by overall development costs, other languages will continue to rule; and if one's focus is there, learning Rust is probably a waste of time.
Eh, I've written and maintained data pipelines, and API layers for downstream teams in prod in Rust, and it was a much more pleasant and low-effort experience than either doing either in Python. I could have had a "complete program" written first in Python, sure, but that gain in velocity is rapidly overtaken by the hours spent debugging and fixing issues in prod, issues that in Rust, I've not typically had.
> if one's focus is there, learning Rust is probably a waste of time.
au contraire - I genuinely think that learning languages, especially ones that are _not_ similar to your day-to-day language is an incredibly valuable thing to do. Learn a Lisp, learn some Haskell, etc, it exposes you to ideas that you wouldn't have otherwise come across, and gives you extra skills and tools with which to solve problems better. Maybe not all those tools are always applicable, but when they are it's a force-multiplier.
When you have to prove your intelligence level (or rather "street cred") in a competitive market you have to show that you can learn the newest complicated things quickly. Rust is pure cryptonite in such a world.
Merriam-Webster's definition: https://www.merriam-webster.com/dictionary/kryptonite
It has been adopted by most large tech companies and several high profile open source projects within ~5 years of it's stable release. It certainly has been rapidly and widely accepted.
For important projects too: Google is using it in Android, Microsoft is using it in Windows, Amazon is using it for Lambda, Meta is using it for their source control tool, Dropbox is using for their storage backend AND in their client apps, Cloudflare is using it for their core proxy, etc.