Why is Rust so popular on HN? Honest question.
It is interesting because it advances the state of the art: it makes safe "systems programming" possible, which no production ready language previously provided.
It is interesting because it is accessible to many of the people who frequent HN. It's community is similar enough to the community of HN readers that many HN readers can use Rust and not feel out of place. The concepts it introduces are not such a departure from the concepts familiar to many programmers that a huge amount of learning is required. (Compare to, say, more research oriented languages like Idris).
More specifically, I would say that it lands in the Goldilocks range: It's not so alien as to make it inaccessible, while being alien enough that you'll learn something from it.
I think you have just made Ada and several other languages disappear from the history.
If you start a new project using Rust there is a reasonable chance you'll be able to find libraries for common tasks and developers willing to work on it. You can make a case for it and not lose your job. With Ada, outside of a few niches that isn't going to be the case IMO.
It's been a long time since I looked at Ada, but I believe it's memory mgmt is more inline with modern C++ than Rust. So perhaps it's not truly comparable.
Which is irrelevant to the claim that no previous language provided safe environment for systems programming.
For some people, a good library selection is essential to being "production ready".
And then, Ada had a selection of libraries for a long time. You just conveniently forget about the paid ones, so you can call Ada "not production ready".
That said, I agree that it'd be moving the goal posts to omit paid libraries.
--
I do think we have slightly different ideas of "Systems Programming", or at least what production ready means for that type of language. I think it boils down to the fact that people do a lot more in "Systems Programming" languages than just OS or implementing and providing interfaces. Specifically, people do it for performance and control, which I don't necessarily excludes the use of libraries.
I think the contention here is that people are thinking of production ready for these types of languages in terms of general use, not just the niche you describe. In retrospect, I do think it was a poor way to word it.
I guess I'd probably amend the original comment to say something like "Rust is the first production ready systems programming language that can comfortably (after a learning curve) be used at higher levels".
Where did you get your information from?
And then in both language, you can use unsafe when you need more power. So Rust might (and I say might, because I have no clear proof of that yet) be a tad more powerful in the safe subset, but it's not in a different class.
The memory safety concepts are novel and unique to Rust but enough has been talked about them already.
Additionally, it brings a lot of features from "advanced" programming languages (ML, Haskell) to a more mainstream language. Sum types, pattern matching, powerful macros. None of these are new or unique, but are executed particularly well in Rust.
In addition to the new features and concepts, the Rust compiler and ecosystem are well engineered and have a good amount of support.
It's the first "new" programming language in a decades worth getting excited about. Ok, maybe I was a bit excited about Julia too, but their execution is not as great as Rust's (and they have made some questionable design decisions).
Could you expound on this? The code I currently write and expect to write in the near-to-mid-term is numerical for engineering purposes. I have the luxury of being able to write greenfield projects, at least for now. My experience with OCaml has inculcated a deep appreciation for powerful type systems and functional styles, so Python, despite its awesome ecosystem, is not particularly of interest. Julia appears to be the natural choice in this space for new development, but I've been a little leery after reading about Dan Luu's experience a couple of years ago: http://danluu.com/julialang/
I'm interested in reasoned criticisms of Julia as it stands now, and what modern practical alternatives people have found.
Rust shows some promise of being that language, and so we hold out hope and enjoy reading about it.
I like rust for a few reasons.
* I really like Rust developers - I've met many, talked with many, and we get along and they've helped me through many technical problems, both big and small.
* I found rust extremely easy to get started with. I knew I'd have to learn the language, I did not also want to learn build tooling, package management, etc. These barriers exist in many other languages, it's frustrating because I just want to focus on programming. Cargo is excellent and I am a huge fan of how rust manages crates.
* I find it very easy to express what I want in rust, to reason about the code, etc. I know when a function might error, I know when I need to handle some unforseen case, I can express how a variable is used, when it is usable, whether it can be mutated. I can cut down on boilerplate with generics and macros.
The last point is what keeps me coming back. I find rust extremely easy to write, because it makes the things I care about easy to express.
I wouldn't use it as an alternative to garbage collected languages since garbage collection is just simpler overall, but I would consider using it as a safer alternative to C or C++.
I do not disagree, but I think one could argue to the contrary as well.
Since most garbage collected languages do not guarantee that objects are actually (timely) garbage collected, you cannot tie the lifetime of other resources (file descriptors, sockets, locks) to object lifetimes. So, the burden is on the programmer to ensure that resources are correctly finalized. Whereas in RAII languages you can properly tie all finalization to object lifetimes.
Note: I am not arguing that GC-ed languages cannot do RAII. AFAIR D has a GC and supports RAII.
This is not as good as the Rust model, but it's not awful, either.
Where things get complicated is when you don't have a clear scope to tie lifetime to and so you can't use these, but then, that applies to RAII too.
I'm not saying C++ doesn't have a bit of an advantage here, but I think it often gets oversold as "C++ has RAII and other languages have nothing even remotely resembling it", which isn't true.
In practice this isn't a problem that I encounter in GC'd languages anywhere near often enough to justify even a slight preference for a "true RAII" language.
I would say that in principle they work very differently ;).
They work superficially in the same way in that if you tie a particular object to the current scope in a RAII language, the cleanup happens at the same point as defer, with, try-with-resource, etc. would. However, there are cases where you really want cleanup to be tied to the object's lifetime.
For example, I have a Tensorflow binding for Go. However, I cannot pass Go-allocated memory (e.g. memory allocated to a slice), because Go does not allocate slice memory on 32-byte boundaries (using Go memory would cause an allocation + memcpy in Tensorflow to align the memory). So, you allocate memory in C-land and have Go structs that wrap your pointers. However, now cleanup becomes interesting. You do not want to rely on finalizers, since they are not guaranteed to run. However, using a Close method is also an annoyance. For tensors that live for the duration of a graph run, it is fine (you can use defer), but other tensors live longer and are reused between runs, shared by models, etc. It becomes unclear pretty quickly who is responsible for closing the model.
I also use a Tensorflow binding for Rust, which is drastically more convenient in this respect. Since ownership is clear, the lifetime of a tensor is bound to the scope or object that owns it. If the owner is dropped, the tensor is also dropped. If you need to share a tensor, you make an Rc/Arc the owner.
People tend to then get annoyed at me for expressing this opinion, but this sort of thing is the reason why. Go is really only merely adequate at interfacing with libraries in other languages [1] which scientific programming does a lot of, and Go has a type system that seems almost precisely tuned to get in your way if you try to program mathematical code in a typed manner but at the same time isn't so weak that you can pull something like a NumPy where at the Python level everything is just untyped so as long as you assemble it correctly up there, the C level can work it all out. Nor can you practically program in that manner, because while you can slather interface{} everywhere, you can't make it convenient to work with like a dynamically-typed language. I think Go is approaching maximally pessimal for scientific-type programming, personally.
I should clarify that when I say Go has "something like" RAII, I do mean pure-Go code only. And by no means is "defer" perfect. (I'm definitely in the camp that it should have been block scoped, not function scoped, and the performance hit can be quite annoying.) It's just that, as I said, it's not like the choice is "either RAII or you're in some manual-management only horrorland"... lots of languages have block-scoped constructs (not just Go) that can be used to 80/20 RAII. That last 20 may be important in some cases, but it's quite often a great deal less important than the 80.
(This post is brought to you by your friendly local "HN poster who has been accused of being unreasonably positive about Go".)
[1] Pretty much every modern language claims to have "great" interfacing with C, despite IMHO wild variances in difficulty. Go is "adequate" because it's not too difficult to simply call a C function, and with not much labor you can get binary-level-compatible structs between the two, which is a nice advantage over Python or Perl or something. But the semantic mismatch is pretty rough around memory management and threading model, and that manifests in slowness in the calls in addition to general semantic mismatch.
That's a good point. Go was my camping ground while Rust was still breaking every month and I didn't want to go back to C++. Although I don't use Go anymore, I have come to appreciate its simplicity and in a lot of scenarios I would definitely recommend it.
For me, the most promising compiled and statically-typed language for scientific computing is Nim.
Disclaimer: I am the author of a Numpy/Torch/Tensorflow-like library written from scratch in pure Nim, the look and feel is pretty similar to Python Pytorch + Keras for neural networks: https://github.com/mratsim/Arraymancer
I did similar thing with C# more than once. Not with TensorFlow, but with my only C++ libraries that also used SIMD and therefore required aligned memory buffers.
It worked just fine. There’s IDisposable for deterministic cleanup, and finalizers as a safety net. Unmanaged interop, i.e. [DllImport], is supported on all platforms, e.g. on Linux it imports from *.so libraries.
Sure, but Rust does help here specifically: resources can safely be moved into child or parent scopes.
I would, too. But there’re problems for which C or C++ is just faster because of language/compiler extensions like SIMD intrinsic or OpenMP.
Also Rust can’t be used for GPU code; strictly speaking C can’t either, but practically CUDA, C++ AMP, OpenCL are very close to C and/or C++.
Also if you have to deal with lots of pointer-based data structures (trees, graphs, etc)., and it’s not just on the lowest level you can abstract away behind a safe API, I don’t think Rust is safer. To get performance comparable to C++, it’s necessary to use unsafe rust for raw pointers, and IMO modern C++ is safer that unsafe Rust.
But even if it’ll become available in 2 months, it’ll take some time to develop Rust libraries on top of them.
E.g. in various performance-critical C++ projects I have used https://github.com/Microsoft/DirectXMath and https://eigen.tuxfamily.org
Both are quite large projects with many man-years spent to develop, they saved quite a lot of my development budget.
Of course, more libraries are needed as well, but there’s already some higher level stuff. See https://github.com/AdamNiederer/faster for example. And const generics, coming to nightly near the end of the year, will be another step up. It's true overall that numeric stuff is a weakness, but we'll get there!
In Rust, the borrow check is still active even in unsafe blocks. In C++, however, the complex rules around when destructors are called, combined with references, are a constant use-after-free footgun that you can never eliminate.
Rust projects that use unsafe code are empirically safer than C++ projects. See, for example, the Servo style system as used in Firefox.
When you deal with pointer-based structures (trees, graphs, etc.) you’ll use raw pointers for them. Raw pointers are not borrow checked, and the complete list of pointer-related potential bugs applies to unsafe Rust, you can use after free, double free, mess with pointer arithmetic so you’re out of bounds, etc.
> In C++, however, the complex rules around when destructors are called, combined with references, are a constant use-after-free footgun that you can never eliminate.
C++ ecosystem has lots of stuff that help writing correct code despite unsafe language. Debug builds use special version of heap that fills freed memory with a magic number, this help to catch most UAF bugs very early in the development. There’re runtime tools like valgrind and asan. There’re static analysis tools like PVS studio, clang static analyzer, and coverity.
Unsafe Rust has none of them. Not even a debug heap.
> Rust projects that use unsafe code are empirically safer than C++ projects.
Survival bias: people who need raw pointers and other performance-related features like SIMD and manual RAM layout don’t pick Rust for their projects.
I also think many of your points over generalize. Finite state machines are graphs for example, but I've never needed to use raw pointers to achieve the performance required. Similarly, I definitely need SIMD, and I use Rust for that.
I can, and I do. The trick is to use some other safer language for less performance critical higher level code. C# often works well for me but there’re others, e.g. in data science community Python is quite popular for that role.
> I've never needed to use raw pointers to achieve the performance required.
Very likely, we work on different kinds of projects. One of my ongoing project is a CAD/CAM app for Windows, and I have implemented quite a lot of pointer-based stuff to optimize the performance. Technically they are mostly graphs, logically they’re multi-level hierarchical containers, LRU caches, quad-edge structures, scene trees, external indices, etc.
P.S. Why do you think Mozilla doesn’t use Rust for their DOM tree implementation, and instead relies on the JavaScript runtime and it’s GC? Don’t you think sometimes other people might also want DOM-like structures in their apps, be it for a web page, XML document, objects hierarchy in 3D space, or any other kind of data?
> Very likely, we work on different kinds of projects.
I work heavily with finite automata. Similar libraries written in C or C++ use pointer based stuff quite heavily, but none of it has so far turned out to be necessary in Rust while also achieving comparable performance.
This continues to miss the point that Rust permits building safe abstractions over unsafe internals. Even if you need to use raw pointers in Rust, you're not only no worse off than you are in C++, but you can actually encapsulate that use in a way that is guaranteed by the type system without resorting to using a completely different programming language.
> Why do you think Mozilla doesn’t use Rust for their DOM tree implementation
I'm not involved with that project, so I wouldn't know, and I wouldn't speculate. You shouldn't either.
I’m not pretending that. Just two points.
1. If you really need unsafe code, because raw pointers, or other reasons, C++ is safer than unsafe Rust.
2. If your project has two distinct parts, unsafe lower level, and safer higher level, you don’t need to use C++ for both. Real world software use multiple languages in the same project for decades already, e.g. QuakeC was developed in 1996.
> a regular expression library might use unsafe internally
If a library wants to do that, it probably means that safe language is too slow :-) https://github.com/dotnet/corefx/tree/master/src/System.Text...
> I work heavily with finite automata.
I’m not saying Rust is useless. There are projects where it shines, and where I’d probably picked it myself. Like a web browser CSS engine, or your finite automata, or many other things.
I’m saying that there’re large problem areas out there for which, due to various reasons, other languages are still way better than Rust. I do realize Rust evolves fast; eventually these areas might shrink or disappear. But in its current state, my opinion is the applicability is limited to very narrow areas: no bare metal, no SIMD, no GPU APIs, very limited asynchronous IO, limited embedded options, very limited numeric libraries, no GPGPU…
I’ve only listed the areas where I have recently (last couple of years) developed substantial amount of code in any other language.
It just happened that I didn’t work on either finite automata, nor browser CSS engines.
I don't agree. I've told you why. I find speaking with you very frustrating, and it's not clear to me that you've actually understood my point unfortunately. :-/
> If a library wants to do that, it probably means that safe language is too slow
That doesn't make any sense.
Yeah, same here. You’re insinuating things I didn’t said nor meant.
> it's not clear to me that you've actually understood my point
I disagree with your points, and I've told you why.
Nothing prevents you from writing a DOM tree in Rust, but when in comes to the Web DOM you need to interact with JavaScript and with the JS GC no matter what you do. How is DOM managed in other browsers written in C++ ? I would be surprised if it wasn't also handled by the JavaScript runtime.
Sure it does! I've used most of those features in Rust, except for Address Sanitizer. But even ASan is available now: https://github.com/rust-lang/rust/pull/38699
I remember spending hours using Valgrind on Rust code back in 2011 to track down codegen problems in the compiler :)
I wouldn't say zero-overhead. I would say O(1) overhead in the best case, and equivalent to C-style manual memory management in the typical (which again isn't zero-overhead and isn't even necessarily always O(1) overhead with heavy allocation and deallocation).
But, yes, it isn't GC'd with the attendant challenges GC poses (or benefits it brings).
Many papers exist but often have conflicting results, or weird methodologies, or extreme artificiality. It's really expensive/ hard to generate good research on this topic that controls for the big variables - you basically need a company that's willing to throw away at on of developer time to build two versions of the same product in both languages, then a metric for measuring success, a way to ensure you take dev skill into account, etc.
So instead it seems a lot more worthwhile to not bother looking for objective research as it won't be found.
I think a better approach is to just use what we have - sense and anecdotes.
From a sensibility standpoint, let's take a step back from rust.
Do we expect fewer memory safety vulnerabilities in Java vs C? Probably yes - Java is a memory safe language with few escape hatches.
So what about Rust vs C? It's a bit less clear as rust has a more oft used escape hatch, but I think we can reason that it's likely to have fewer memory safety bugs.
Would we expect fewer bugs in general? That's much harder to discuss with that approach.
You can look at testaments from Servo devs, and other companies using Rust ( https://www.rust-lang.org/en-US/whitepapers.html ) and their experiences.
In C++ land, you'd have to use TSan thread sanitizer and a lot of other tooling (ASan, Valgrind, etc) to achieve even close to the same level of confidence. Not only is this a runtime check but all that tooling has a fairly heavy mental cost too.
I know that the parent post used multi-threading as an argument in favor of Rust. But there are many ways to blow of your foot in single threaded C++. For example, consider this single-threaded code:
std::vector<size_t> v;
v.push_back(1);
size_t *first = &v[0];
std::cout << *first << std::endl;
// The STL vector implementation will probably
// reallocate the backing array at least once.
for (size_t i = 2; i < 1000; i++)
v.push_back(i);
std::cout << *first << std::endl;
However, the mental cost a Rust programmer pays for that trade (due to borrow checker) seems too high for me
The thing is that most of the errors that the borrow checker will point out are potential programming errors in C or C++ as well. However, current C/C++ compilers will put most of the responsibility completely on you (plus tools like Valgrind). So, to me C++ has at least the same cognitive load if you want to program responsibly.
A good example of the challenge is that there's unsafe inner code in rusts own container code. If you try to build your own custom containers (or things like arenas) you'll likely work with unsafe code... Or rely on rusts container code (not a bad thing of course).
My feeling is that "C to Rust" is close to "Java to Haskell" more than anything else. The extra layers are important enough that they need a new thought process. (I say this as a proponent of rust)
> You really need to lay your code in away that will satisfy the checkers.
There are some patterns that are common but problematic but are being addressed by non-lexical lifetimes. Other than that it's almost "program in C++, then fix errors".
But that becomes a second nature after a while. I think all the edge cases that still exist make it more problematic. E.g. I was writing some SIMD code yesterday with a loop somewhat like this (simplified):
let mut v = vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0];
let mut s: &mut [f32] = &mut v;
while s.len() >= 2 {
println!("{:?}", s);
s = &mut s[2..];
}
It makes you scratch your head for a bit.
A good example of the challenge is that there's unsafe inner code in rusts own container code.
I am pretty proud that one of my first small Rust projects was a randomized ternary trie. Apparently, it does not use unsafe anywhere.
https://github.com/danieldk/rtrie/blob/master/src/ternary.rs
Not that I am proud of the code ;).
But in all seriousness, unsafe in container code is somewhat expected, because at some point you need to wrap raw memory (as e.g. in Vec). I think quite a bit of, but probably not all, unsafe use in containers such as Vec comes from that.
They thought it worthwhile to invent a new safer language for massive parallelism and security, rather than to attempt it in C++.
Also coming from Java, the Rust experience is not that painful. It's like learning a completely foreign language.
void Scheduler::schedule(const Job& job, Result* result);
void Scheduler::wait();
The consumer of this API in C++ has an easy job. They can create an array of results and iteratively call schedule(jobs[i], &results[i]) for each job, and then call wait(). A similar API in Rust would be a giant pain in the neck to use, because there's no simple way to collect the results. You'd likely want to scrap this API entirely and have Scheduler expose some kind of queue or something.
My point is that these extra constraints are not just extra typing locally to specify how you're using lifetimes. You'll likely have to rethink the way you write APIs and whole programs in Rust because of the ways it restricts mutability. Which is OK, it's a trade-off, all I'm saying is the safety doesn't come for free.
The mutability thing isn't a problem. You can allocate a vector of results and jobs, and then have a mutable iterator over them, zip them together and then iterate over that. There's no problem passing mutable references to each of the elements.
The right way to implement the API above is for schedule() to take Arc<Mutex<Result>>, and explicitly share mutable ownership over each individual result between some unspecified asynchronous execution mechanism and the caller of schedule(). This makes the calling code significantly more complicated as it needs to incur the overhead of atomic reference counting and a mutex per result, and must try to lock each result before using it, even if it knows there are no other owners of the result due to wait() returning.
https://play.rust-lang.org/?gist=1324a919ca5d0e2f16e445364a7...
Rust also offers some other methods to allow multiple mutable references to containers: .split_mut() comes to mind – that splits a mutable slice into two mutable slices that don't overlap. In the future I'd like to see in the stdlib "container adapters" that take in a normal container and provide an interface to it that allows taking multiple mutable references while ensuring the soundness using dynamic checks (it can keep an array of the pointers or indexes that are borrowed out, and check against that). I created such an interface as an experiment last year: https://github.com/golddranks/multi_mut
i remember reading about the stylo (multithreaded css styling) engine, something that has been tried times and times again to implement it in C++ without success: too complicated. sooner or later you'll get too-hard concurrency bugs and the project would be abandoned. and it wasn't just mozilla with firefox, google also failed to implement concurrent styling with chrome.
the blog post stated that this time they were successful _only because of rust_. the mental overhead of managing the borrow checker is far lower than the mental overhead of dealing with concurrency issues in a complex high performance project.
Better to be charitable, I think. Doing so costs us nothing.
Rust is a fairly conservative language. It doesn't really do anything novel, it mostly just synthesizes patterns that have existed for years. Hopefully Rust truly is mediocre (at least in hindsight). That would mean the things it is attempting to replace are seen for how primitive they truly are.
This is so well said that I agree 100%, and couldn't say it any better.