Why Rust helps even if you have to use a lot of `unsafe`
v5.chriskrycho.com
v5.chriskrycho.com
Ability to isolate high-risk "hold my beer" code from boring glue code has many nice side effects.
In a team, you can agree who can write and review `unsafe` code, and not be afraid that your junior team members will cause some horror heisenbug (there can always be other types of higher-level bugs in safe code, but safe code doesn't cause them, you'd have them anyway, on top of memory issues).
People often move abstractions built from unsafe code (like clever data structures, concurrency primitives, OS or C library wrappers) into separate crates. This makes it easier to thoroughly test and fuzz just the unsafe code, without having to test a larger project as a whole (once you ensure your safe API is indeed upholding safety invariants, the compiler will prove all uses of that code are safe too).
EDIT: I'm rate limited. If we assume an `[[unsafe]]` attribute would only apply to things like pointer arithmetic and unchecked array access (not lifetimes), could such a linter exist?
n3005: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3005.pdf
There's also the issue that I've seen it commonly advocated to keep `unsafe` blocks small - which makes intuitive sense, but brings into question whether `unsafe` is supposed to _just_ mark the unsafe operations being done, or mark all the code under which certain assumptions have to hold for UB to be avoided and it be 'safe'. It's been a little while since I looked into it, but there wasn't much agreement on this the last time I looked and I've seen people do it both ways.
The thing is that once you've managed to do that, the _consumers_ of your API are free from worries. The situation becomes a whole lot better than the equivalent C situation, where both sides of the API need to be careful about what funny abominations the another side might devise.
Yes, I've written a decent amount of `unsafe` code. And I would also point out, C programmers say the same thing about their code :P The problem comes when an assumption you thought was ok turns out to be something the compiler writers thought you shouldn't do, and then your code is broken. Rust doesn't have a defined list of invariants you can't break in `unsafe`, which makes it pretty hard to be confident you didn't break one of them. I've loosely followed the work to pin down some of these things[0], but I don't personally feel like tons of progress has happened and they've been working on it for years.
That said don't get me wrong, I like Rust, I just wish the `unsafe` situation was improved. I also agree that creating safe APIs around `unsafe` code is the way to go, but I've seen a whole bunch of people get it wrong due to bad advice given around `unsafe` or lack of clarity around what `unsafe` is actually supposed to mean.
`unsafe` has also IMO been overloaded to mean both "this code does one of the unsafe operations" or "this code has some invariant you must keep". The latter tends to be the better way to go for creating APIs, but is also harder to do and ends up causing you to mark completely safe code with an `unsafe`.
I find programming in a language without sum types is like rock climbing in ankle weights. I just can’t model many types of data correctly, and I hate using product types for something that is clearly a sum type.
Even if Rust didn’t make any safety guarantees over other languages (although sum types provide safety guarantees through exhaustiveness!) I would still think Rust helps a lot because of that.
I guess Swift and Kotlin are basically “it has sum types” languages too, although they both come with baggage (Apple-only in practice, JVM).
(now of course an automatic transformation into a structure-of-arrays would be pretty cool, I actually wonder what Zig's MultiArrayList does with tagged unions).
PS: in general I agree though, for instance in Typescript they are fantastic, but there I usually don't care much about memory layout and performance.
Some of that could probably be solved with comptime code (like Zig's MultiArrayList container I mentioned above).
Such as ?
I also wasn't thinking about moving parts of a struct out into heap allocations, I think in most situations that would be worse than having to deal with padding. Instead at least have the programmer think about how the internal memory layout of sum types might affect performance in specific situations instead of just using them everywhere and in mass because they are convenient.
And if the type has some illegal values, these can sometimes be used to discriminate the type. When that's possible, the union will have the same size as the largest underlying type with no overhead for a tag. For example Option<&T> matches the size of &T, since it uses the illegal null value to represent None.
The tag can often be stored inline in the data, so doesn't necessarily require extra space.
> because they always take up as much space as the largest wrapped type
Yeah... but what's the alternative?
enum Foo {
Bar,
Qux([i32; 1024]),
}
enum Foo {
Bar,
Qux(Box<[i32; 1024]>),
}
Today, clippy does what I think is the right thing: it teaches the user
warning: large size difference between variants
--> src/main.rs:1:1
|
1 | / enum Foo {
2 | | Bar,
| | --- the second-largest variant carries no data at all
3 | | Qux([i32; 1024]),
| | ---------------- the largest variant contains at least 4096 bytes
4 | | }
| |_^ the entire enum is at least 4100 bytes
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#large_enum_variant
= note: `#[warn(clippy::large_enum_variant)]` on by default
help: consider boxing the large fields to reduce the total size of the enum
|
3 | Qux(Box<[i32; 1024]>),
| ~~~~~~~~~~~~~~~~I'm always surprised to hear people say this, when my perspective on the JVM is that it's a modern technical marvel several decades in the making with advantages like several unparalleled GCs and an ecosystem with extensive off-the-shelf & highly performant code available for any domain.
If I was working on a server thing Kotlin would seem really good. In my work it is obviously applicable to Android, but not so much elsewhere.
I think Rust is nearly perfect for developer CLIs, you don’t really need to deal with the thorny memory and lifetime stuff because most things are allocated on the stack and the entire program is synchronous. clap is very good. Performance, especially startup performance, is incredible.
For instance, you can’t use String on hot paths. Also, NPEs can happen almost anywhere, and the compiler doesn’t enforce thread safely (by which I mean “statically checked to be data race free”)
Also, the 10-50% slower rule of thumb only applies to throughput. Tail latency is generally much worse than that.
There is nothing comparable in C as performant as all of the streaming tooling built up around Kafka for example.
So your can benchmark percentages all you want, but there are a lot of things you just can't do in C that you can on the JVM (without 100 developers and three years)
This is an ill-chosen example.
librdkafka is as or more performant than the Java stuff.
As a server, Redpanda blows Kafka out of the water (but this is rarely any bottleneck with brokers).
I was referring more to the general ecosystem of Flink/Spark/KStreams/KTable etc.
Now if it means I can run fewer smaller brokers that's awesome.
Then it's still a bit of an untruth since all of those use RocksDB for their high-performance storage layer, which isn't in Java. You could maybe (maybe!) argue for some ergonomics (though especially hard to argue for Spark!) but performance still goes plainly to C (/ C++).
Not to mention you still pay a real performance penalty for being unable to make direct use of RocksDB features, because Kafka is shoving a million "abstraction" and cache layers in there and you just want your your damned merge operator to apply.
I don't agree Kafka off the JVM is subpar. Kafka Streams has a lot of sharp edges from "abstractions" that aren't, Spark is a godawful developer experience and tuning it for performance is black magic for 99% of data engineers, and Flink keeps dithering about how consistent/stable baseline features like queries will be. You can stand up very good apps for a huge set of common cases just as quickly in Go or C++, and not have to deal with Kafka Streams's funny opinions about threads/tasks/consumers.
Can't quite comment on the GCs though
I'm not exactly a fan of Java, to put it mildly, but it is pretty efficient these days. To the point that it sometimes amazes me.
From having played around the JVM, and spent quite a bite of time optimizer large spark workload , from the limited Monomorphization, to memory and cache inefficient object model, to the limited inlining depth, absence of code compaction ( for i-cache performance), the poor registor allocator etc... the JVM is really not up there.
I think it only shines when compare to things like go/python/javascript that the JVM can appear to be great.
Many that like to bash Java EE, have no clue how bad it was before it came to be, it wasn't only Sun's marketing pushing it.
It isn't by accident that RMI had interoperability with CORBA.
And again, the success of JAVA in the 90 and 00 can also be explained by better tooling, safer language,better x-platform stories,better libraries etc... etc...
Saying that java/jvm are not particularly performant is not bashing ... java/jvm have other great things going on... performance just isnt one of them
Naturally not having a clue about how to write proper Java code (for loops instead of System.arraycopy()), use profilers, use JIT caches with PGO feedback, not understanding how to allocate off heap, not using JNI to avoid writing everything in C or C++ with their own set of issues, doing background stuff on main threads,...also helps some Java applications not being performant.
There's a reason the entire streaming ecosystem (Kafka/Flink/Spark/Etc.) has built up around the JVM.
Is memory util on servers still really a consideration in 2023 for most workloads if performance is otherwise good?
> If you can, avoid Java for anything that isn't a long running server process
Isn't that the JVM's niche?
Startup time is definitely a concern. Good luck on the throughput front though. Java libs (like Netty) still outpace what's available in Rust. You may end up with your own code being more performant, but the things you rely on externally being slower. This was our experience when trialing re-writes in Rust.
The way that cloud scaling works is that as you increase RAM you also increase CPU, which costs a fortune. You can go for 'memory optimized' instances but even still, there are costs associated with memory that aren't just the RAM modules, unfortunately.
Plus with container orchestration it's increasingly important to pack your services into a single instance, and memory can veryyyy quickly become the constraint - in fact, it might be the one I've seen most often.
For our workloads we seem to need more CPU when we need more ram anyways.
Where have you found so much use for them?
In a language with sum types like Rust, Kotlin, and Swift, if you’re following the default “happy path” of coding, you are not allowed to forget a check. The compiler will not let you.
That's because it comes with a statically-tracked "tag" (so: sometimes called a "tagged union") that the compiler can use to check you. You can write the same thing with a `union` in C or C++, with an `enum` to tag it, but you're doing that by hand rather than having it built in.
Also: Where a struct or class's relationship over its members is the product of all of the types which make it up, a tagged union/enum is the sum of them. A common example: instead of a bunch of optional fields on a single class to model the different possible combinations, modeling something as only the legal set of combinations of fields. I recommend Scott Wlaschin's 'designing with types' series (uses F♯, but applies to any languages with 'sum types') for a great intro and then deep dive on the ideas here: https://fsharpforfunandprofit.com/series/designing-with-type...
The most motivating example is Rust's Result<T, E>. Either you have a "good" type T, or some error E occurred. Want to bubble this error up? Just do `my_result?` to get the underlying T or return E early. This idiom is pervasive and makes error handling via return codes very easy to use.
It also makes it feasible to have types that have no "default" value. In C++ you have std::unique_ptr<T> which, for various reasons, has to have a nullptr state, or else it's frustrating to use. In Rust, owned pointers Box<T> don't have an empty state. If you need a nullable pointer, you use Option<Box<T>>, which is either Some(my_box) or None. The fact that Box<T> is always pointing to a valid object is a stronger guarantee than C++ smart pointers, and is only ergonomically possible because of the Option<T> sum type.
It is just that std::optional and unique_ptr do not have the right semantics out of the box, but it is not hard to roll your own.
But yes, I wish C++ has good builtin sum types and proper destructuring via pattern patching. Maybe one day....
It is still possible to return non-movable, non-copyable types from functions.
But I guess ergonomics would not be good.
However your examples i don't think are particularly convincing.
The examples you gave can mostly be done with std::variant<> and are simply examples are unions. (Result == std::expected)
In term of non nullable smart pointer, it would be easy to enforce the the non nullability in the constructor, and have the same ergonomy as Option<Box<T>>. In fact you can do something like std::expected<std::reference<T>> but why would you instead of just std::unique_ptr and handle the nullable case.
What makes sum type powerfull is to have full Algebraic DataType (tags + sum types + product types) + pattern matching + good compiler errors.
https://doc.rust-lang.org/reference/items/unions.html
The unions of Rust are tagged unions that can hold data:
https://doc.rust-lang.org/book/ch06-01-defining-an-enum.html
which is immensely powerful in data modeling.
Sum types are also quite nice to bridge the gap from a 'dynamically typed mindset' into a 'statically typed mindset'. Sum types basically provide the ability that a variable can assume different types during its lifetime (? although I don't know if such 'shape-shifting' is allowed in Rust), so it almost feels like Python or Javascript, but since the compiler 'knows' sum types it's all still completely typesafe.
In Typescript it's convenient and frankly quite nice, in a programming language like Rust where performance matters I'd be less enthusiastic though.
I think so! You mean this?
let mut x: Result <bool, String> = Ok (false);
dbg! (&x);
// Get a reference to the bool and mutate it
if let Ok (b) = &mut x {
*b = true;
}
dbg! (&x);
// "shape-shift" to the other enum variant
x = Err ("No boolean :(".to_string ());
dbg! (&x);
1. failing to check for sigils, because nobody uses a sigil when you have sum types. This would have prevented a famous exploit in SUDO a couple years back where someone forgot to check for the special negative value on an input parameter.
2. Eliminating null pointer exceptions or anything similar. You can't forget to check, you can still choose to have this kind of error, but you have to do it explicitly.
Consider something like a function to open and give you back a file handle. In many languages such a function will return a pointer to a file, or null if there is a failure, or perhaps throw an exception if there is a failure. If you neglect to check for null, or forget to handle the exception, you have a problem. With sum types, the natural thing is to return something like a Result type, which is EITHER a file, or an error. And you have to check which it is, to get the File out.
// This derive macro expands to a visitor pattern during build
#[derive (serde::Serialize)]
enum Message {
QueryFoo (QueryFoo),
MutateBar (MutateBar),
}
struct QueryFoo {
foo_id: i64,
}
// If we receive a Message::MutateBar from outside our process,
// it's a compile error for us to accidentally access the field `QueryFoo::foo_id`.
struct MutateBar {
bar_id: String,
baz_value: u32,
}
fn send_msg_as_json (tx: &mut Sender, msg: &Message) -> anyhow::Result <()> {
// The JSON serialization / deserialization is generated, too
tx.send (serde_json::to_string (msg))?;
Ok (())
}
I haven't seen anything in C++ that's both as easy and as airtight as a Rust enum with serde's automatic serialization macro on the top.
Also in well designed low level binary protocols message types have a discriminant header already so often common to wrap them in an ad hoc-variant which doesn't have a separate discriminant.
Some languages do that wrapping and checking for you, automatically. Everything from Ada (variant records) to Haskell has them. You cannot mishandle such objects by misinterpreting them as the wrong type; it's a compile-time error in those languages to do so.
Imagine you are parsing something. In my opinion, the most natural way to express this, is a function that takes an array of characters and returns an array of parsed objects or tokens, where each object may be of a different possible type. In C, you'd probably use a union of structs, and set which type of struct as you parse each object. This is error prone and verbose, and the compiler can't double-check it for you. It's all automatic in a language with sum types. Quite efficiently too; most of the runtime checks can be optimized away.
Additionally that is one of the examples on the ongoing C++ reflection papers.
Then there are a couple of examples in D, Common Lisp, Scheme, C# code generators.
Please point to a specific library if you disagree.
Unsafe blocks in rust are the only place you can dereference pointers. There are whole classes of memory bugs which are impossible to code up in safe rust code because of this.
As an example of how this helps, a couple years ago I was working on a complex & clever rope data structure. My code segfaulted - and I can count on one hand the number of times my rust code has ever done that. A segfault like that can only come from memory corruption or dereferencing invalid memory - both of which are only possible in unsafe code. And the bottom of the stack trace was a safe function, so I knew I’d corrupted memory somehow. These bugs are really hard to find in C - basically the worst case for debugging because the problem could be literally anywhere. But in rust I knew the bug was in one of only two unsafe blocks in the ~thousand or so lines of my library. And one of those blocks was executed not long before the crash.
5 minutes later I had found and fixed the bug. In C, I’ve had similar problems take hours to days to fix.
Bugs like this aren’t possible at all in Go or C# because of their memory model. But in those languages you pay for safety by sacrificing performance. The rust I’ve written & benchmarked so far somehow faster than the equivalent C code. Which is pretty wild!
> 5 minutes later I had found and fixed the bug. In C, I’ve had similar problems take hours to days to fix.
Modern tooling like a memory synthesizer and good debugger makes this kind of debugging relatively easy.
In some case, the bug would have been detected even before you run with proper warnings.
> Bugs like this aren’t possible at all in Go or C# because of their memory model.
Except in their respective FFI or unsafe block of code.
> But in those languages you pay for safety by sacrificing performance.
That's a bit of a strong statement. Even if we accept (i don't) the performance aspect, GC languages give more safety,convenience and dev prod. than rust and at a significantly less complexity overload.
> As an example of how this helps, a couple years ago I was working on a complex & clever rope data structure. My code segfaulted - and I can count on one hand the number of times my rust code has ever done that.
This might simply be an indication that you were using the wrong tools in the first language. Most native languages (D, C++ etc...) have facilities to avoid unsafe behavior. Use them.
I'll cede the point that some of Rust's advantages are merely psychological. But I would argue that most of us are tired, frustrated, and working on teams where we have neither the energy nor authority nor teaching experience to make everyone learn the right tools. I've had to teach some coworkers how to use a profiler. And then remind them to use it.
Therefore mere psychological benefits still improve outcomes.
It's easier to memorize "cargo run" than to memorize the vocabulary quizzes for Python or Kotlin tooling. (A gradle is used to serve soup. A wheel is part of a car.) Rust produces one binary by default, and I don't even have to think about how to bundle a runtime. That means I also don't need to teach anyone what Docker is.
I have a hunch this actually matters. Everyone who's learning to program right now will be deep in their work in 20 years. So every software that's succeeded in my lifetime - JavaScript, Python, Unity 3D, etc., has succeeded by appealing to noobies and then lasting long enough that everyone likes it. In the very long term, a low barrier to entry might matter as much as a high skill ceiling.
I assume it's a typo for memory sanitizer (i.e., ASAN).
At the limit, I think the performance benefits of rust are pretty clear. The performance ceiling of rust seems to be pretty unambiguously higher than that of Go, Python, C#, javascript, etc. But yeah - no contest on rust being way harder to learn and use. I’ve been programming rust for years now, but I still reach for typescript when prototyping because I still find it much faster when making prototypes. Especially when async is involved.
> This might simply be an indication that you were using the wrong tools in the first language. Most native languages (D, C++ etc...) have facilities to avoid unsafe behavior. Use them.
Can you give some examples of those tools? The “first language” I mentioned above was C. What equivalent does C have to rust’s memory safety? Or C++ for that matter? Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?
Not to be pedantic, but "at the limit" performance is not a very good metric. Programming is an engineering exercise, so IMO performance delta per unit of development work is better characterization if we are comparing programming language.
At the limit C# supports inline assembly, so at the limit it could be match rust pounds for pounds with enough efforts, but that's hardly a useful comparison.
In general, a lot of the modern performance differences between programming languages is caused (from the programming languages perspective) by the object model each programming language use. Python/Javascript are in a class of their own and i don't think belong in performance conversation against statically type languages. Can't comment on go cuz i don't know much it.
On C#, the memory model can be surprisingly close to rust. The combination of typed IR for efficient monomorphization + the the ability to use stack allocated struct, mean that C# code can be quite close to rust code. And then the differences are mainly a consequence of LLVM backend vs ruygenJIT or AOT.
To say nothing of the advantage that a GC can have by having a global view of memory allocation/deallocations.
Added to that the fact that the time a rust eng. is spending modeling his ownership relationships and converting then into something the borrow checker can/accept could be spend improving C# perf further.
> Can you give some examples of those tools? The “first language” I mentioned above was C. What equivalent does C have to rust’s memory safety? Or C++ for that matter? Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?
I am not too familiar with C, but i don't a think any new tool is needed : wrap all the unsafe operation (naked array accesses, allocation etc...) in a safe API and only use the safe API...
In C++, its trivial to avoid memory unsafe operation between smart_pointer, &reference, for_each for iteration etc... etc... Manipulating raw memory in "modern" C++ is mostly a choice than a necessity these days.
ASAN (address sanitizer), UBSAN (undefined behaviour sanitizer), Valgrind.
All of these only detect runtime violations, so they can find bugs but don't prove their absence. On top of that I got a sufficient number of false negative or even bugs that wouldn't reproduce/wouldn't give interesting information with these tools that I can confidently tell you they don't even approach the guarantees provided by Rust.
There are also static analyzer like cppcheck and clang tidy. They do find some useful issues, but they:
1. Have false positives 2. Are a hassle to add on top of existing projects 3. Have false negatives, so a code that passes is not necessarily devoid of UB.
> Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?
Smart pointers in C++ effectively prevent some classes of double free errors and some classes of leaks (like all RAII, I might add). They do very little against use-after-free in the general sense, because as soon as you take an observer pointer/reference (the equivalent of a Rust borrow), you have to either:
1. Forfeit safety by handling its lifetime on your own 2. Forfeit performance by unnecessarily cloning data (and also forfeit correctness in some cases, as the "sea of shared pointers" approach of some codebases create object identity issues and memory cycles).
The standard library is also full of footguns that can cause UB, such as calling `vector::front()` on an empty vector, that come up a lot in practice. I think cppcheck was able to flag some of these but I'm unsure of this off the top of my head.
Lastly C++ has a slew of very ugly errors like ODR violations that do happen in practice (albeit rarely) and that are completely outside the purview of tools (the last one a former colleague wrote would generate completely garbage information in asan, and would cause valgrind itself to segfault. The error cost the afternoon of two engineers for what was a typo causing a namespace to be early closed in a file... The kind of abhorrent error that causes to avoid these languages as much as I can).
I developed in C++11 for almost 10 years, and I am now a professional Rust developer. Productivity, developer experience, and correctness are night and day, Rust's advantage.
This is what memory safe systems languages offer, there is no way around avoiding such tooling, it is part of the language and needs to be explicitly disabled.
Maybe; rest to see if using modern tooling is easier or harder on the average programmer than learning a new language.
> This is what memory safe systems languages offer, there is no way around avoiding such tooling, it is part of the language and needs to be explicitly disabled.
The programming language is just one part of the chain. The goal is to detect/prevent bad code pattern and address them. There is no rules about this happening at the language level.
As a point of order, such bugs are possible for Go. Eg, hashmap's behaviour becomes undefined if you access from multiple threads at the same time.
Admittedly, that's a Go peculiarity. Making the programmer deal with concurrency on collections undoubtedly gets it a speed boost, but it's not the tradeoff most gc languages take.
The beauty of Rust, is you don't need to trade speed or memory for concurrency.
I think this side note is the most interesting tidbit, because I agree. I think Rust is a beautiful thing, but it's starting to get more competition now, and the "memory safe systems language" space is really starting to heat up. I'm particularly interested in seeing whether Val becomes its own thing, or if Swift and Rust (or another language) just absorb it's good ideas.
Rust is amazing and the community is full of great people, but there are a lot of rough edges and places that feel awkward for new folks. I appreciate seeing all of the constructive criticisms that have come out against Rust lately, and for folks like Tim McNamara who are pointing out all of Rust's rough edges in hopes of improving the language. If I were able to contribute more to the project, that is where I would focus, personally.
Also, Rust needs to get it's leadership issues sorted out. More and more projects are starting to sneak up on Rust, and if the folks in charge of the project keep fumbling the bag and causing rifts in the community (like with the Rust licensing, and driving phantomderp away), it's going to keep folks from wanting to do the hard work to improve the language.
One of my favourite is the split between the abstract/programmer's view of a type, and the machine implementation/representation of the type.
There's no bit twiddling, no masking with 0xFFFF..., no worries about order in a struct or packing. If you have a 4 bit value, and want to store it in bits 27 ... 31 of a word, Ada lets you specify that. From the programmer's perspective, you work on the abstract type, perhaps defined as range 0 .. 15 in this case. (It can be any type that fits into the number of bits available, by the way.) It packs/unpacks transparently behind the scenes. And if you overflow that 0 .. 15 range it will raise an overflow/underflow exception. If you were trying to cram that 0 .. 15 range into only 3 bits, that'll raise a compile time error.
Ever since I've encountered that its absence from the C family seems like a very obvious oversight.
Also I'm not sure the cognitive overhead is really that big. The compiler helps a lot with its useful error messages.
Even heap allocations like `Box` are still just pointers, that have destructors that run when they fall out of scope, but access is just a pointer dereference, the same as you could do with unsafe.
Look at the Firecracker codebase. There are a lot of unsafe calls into libc.
There is also a runtime reference counter which takes up cycles since it has to operate at run time.
I'm not trying to invalidate all claims that unsafe is necessary, nor even that there are cases where unsafe is quite helpful for performance. It's just that a little unsafe used deliberately and thoughtfully can go a long way, and accomplish those goals in a largely safe codebase.
Re: overhead—I think the compiler helps a ton here, but I do think that it makes a lot of memory-related concerns explicit where they are usually implicit, and dealing with lifetimes can be quite burdensome. Now: I also think that those concerns still exist when you're using other systems-level languages, and that the tradeoff therefore often ends up looking like "Deal with it up front, or deal with it on hard mode later."
- From a language point of view, the syntax for dereferencing pointers in rust is awful. I don’t mind the unsafe keyword. But in rust you can’t do a->b. You need to write (*a).b. And that gets increasingly awful as you have more complex pointer expressions. Unsafe code in rust looks significantly uglier than the equivalent C.
- The Rust community - particularly on Reddit - seems to get very snide annd condescending about programmers using any amount of unsafe in their rust code. Even when it’s totally necessary. I’m not convinced the community will is there to fix rust’s bad pointer syntax, or other missteps with unsafe. Apparently having unreadable unsafe code is my atonement for wanting high performance code. Never mind that the harder my code is to read, the more likely I am to make mistakes. And in exactly the part of my codebase where bugs would be the most damaging to safety! Argh.
I love rust. I love the fresh ideas it’s brought to programming as a whole. But I can’t wait for some other, newer languages to iterate on these features & the language syntax some more. It’s not the top of the mountain.
For what is worth, I think that we'll add that at some point.
Cargo stores packages from both sub-communities and we fight over the heart of rust. - And sometimes to our collective detriment, like what happened to actix.
Also: agreed re: syntax.
> But I can’t wait for some other, newer languages to iterate on these features & the language syntax some more. It’s not the top of the mountain.
YEP. That's exactly what I was getting at in my callout at the top of the piece. First word, not the last.
It’s pretty common to need to interact with existing APIs or libraries that only have a c interface.
Fortunately, rust is good at that.
Unfortunately, it’s impossible to do without unsafe code.
You can wrap libraries in safe wrappers, or if you’re lucky someone has a wrapper you can use…
…but I’ve heard it several times for this reason, especially in shops that have existing internal code bases (Ie. 0% chance of an existing safe wrapper) and want to incrementally adopt rust.
Never heard someone say it’s because of perf tho; that sounds like a strawman argument to me; if your code is seriously too slow because of the bounds checking you’re doing something crazy and probably want to be using a gpu instead.
It’s usually io bound, and unsafe code doesn’t fix that.
As someone trying very hard to love Rust this is just simply not the case. It takes the shortest path to most answers. The most trivial example where the compiler becomes less helpful is a self-balancing binary tree. It will constantly suggest explicit lifetimes and other magic when in reality you probably want `Box<Rc<..>>`. It will suggest Boxing for recursive types...but that's about the end of it. In many cases you're better off knowing what to do. But the problem is knowing what to do requires you to have done what you need...and hence the Rust Ouroboros is born.
The nice thing about unsafe code (C, for example) is the cognitive overhead is initially very low. That allows you to get an idea down and then worry about stuff later. Of course, if you have a gun to your head from management you never get to the "worry about stuff later" stage and introduce all sorts of nasty bugs. Rust frontloads all of this making it sometimes very difficult to use without having everything already thought out way ahead of time. Both a benefit and a curse.
There’s only so many variations that work for something like trees (especially with doubly linked nodes) in any language, without either inherently unsafe lifetimes (because the particular language can’t express them as part of the API) or some kind of indirection (such as ECS style indices). Or garbage collection or ref counting, but one probably uses Rust to be able to control that kind of overhead instead of having to use whatever the language creators picked.
If you want to learn about this, instead of using a ‘blessed’ implementation, read “too many linked lists” https://rust-unofficial.github.io/too-many-lists/.
As for the front-loading comment: yes, Rust requires you to get some things right from the start, particularly ownership and lifetimes. If you want to work in Rust like in a scripting language, you can just use Box, Rc and clone() everywhere. But getting it right without those might require a significantly different API. Which isn’t that much different with your rough C implementation. To get any kind of safe API there will require just as many changes (if not more).
If you can provide examples of cases like these, please file a ticket in the issue tracker. Some suggestions have to be overly cautious in order to not lead people astray, so having examples of common (or uncommon!) things people try that could use with some "human touch" in what direction they should take instead is very useful and welcome.
I think now that too many lists is part of the book, maybe the compiler can link to that in some cases, hah.
A lot of the cases I've seen so far require some kind of global analysis, that we don't have today. But if we can have some kind of local analysis that will be reliable for a subset of cases, I'd jump at the opportunity to add it.
> I think now that too many lists is part of the book, maybe the compiler can link to that in some cases, hah.
That's not a bad idea, we already link to the book when it seems appropriate.
It primarily comes up when you're 1) interfacing with hardware 2) interfacing with C APIs 3) doing something weird with pointers.
In fact, had languages like Modula-2 and Ada not lost to UNIX/C/C++ hegemony, this would already be quite common.
It is kind of tragic that we still need to keep discussing the value of this.
I also can not believe that that much code actually needs raw pointer operations if the developer considers alternatives.
Software is getting complicated and supply chain attacks are real.
The sandbox should be part of the language.
https://downloads.haskell.org/~ghc/7.8.4/docs/html/users_gui...
I hope it trickles into a mainstream language before I retire. Sure, it's an extra arg to every non-pure function.
So what? Write more pure functions! If a function shouldn't use the network, pass it an IO that can't do so. If it shouldn't touch disk, pass it an IO that can't do so.
This may complicated printf logging... Or maybe there is an IO that only allows passing messages to a logging module, and the logging module can write to disk. So the logging module gets more auditing attention than pure functions.
I didn't realize languages should have theme songs until now.
Think all languages should have a theme.
https://soundcloud.com/chriskrycho/new-rustacean-theme
Other Languages:
C++, some classic rock?
C#, some re-cover of a classic rock song by a corporate group?
Python, dubstep?
This is an interesting Catch 22.
If you accept this premise as true, then Rust becomes ultimately pointless. Because the prescription would be to become a very good programmer and learn how to write memory safe code, at which point you really don't need any other language except C. You will likely write memorypools that have mechanisms to make sure that you never access memory you are not supposed to.
An alternative prescription is to dedicate effort to making higher level language compilers/transpilers better. That way everyone codes in Python, the compiler recognizes common access patterns, and does iterative optimization (probably ML powered) to get you the most optimal machine code.
If you accept the premise as false, i.e writing memory safe code is easy, then Rust doesn't really offer any benefits.
Yes. Indeed you don't need any other language than C. Every program could be written in it.
The point of rust is to enforce strict rules on code that guarantees certain behaviour, but still allow operations which could violate those behaviours.
>Because the prescription would be to become a very good programmer and learn how to write memory safe code
No. The prescription is to be careful when dealing with memory. This is true for literally any language, the goal of rust is that the compiler can take away some of that workload.
The problem isn't becoming a very good programmer. That's not difficult. The problem is becoming a very good programmer who is never tired or rushed. Any correct code in C will have implicit rules about how the memory is managed that are either written in comments or reside in the programmer's head. This doesn't work because eventually some will make a code change without understanding fully the implicit rule (often this person is the same person who figured out the rule).
So what stops that programmer from being lazy and wrapping all the blocks in unsafe in a Rust code base?
Look at any large enterprise Java project. People take shortcuts and break things all the time, even with fairly strict typing and memory safety.
Of course you can enforce things like "no unsafes ever in codebase" on a project, but if you have to reply on guidance for development, then you can simply say "first we write a complete set of tests that are a contract on correct operation, then we develop code", at which point the language choice becomes irrelevant.