The Rustonomicon | Hacker News Reader

The Rustonomicon

doc.rust-lang.org

191 pointslsllc51 comments

Previous discussions:

2016 https://news.ycombinator.com/item?id=12065912

2015 https://news.ycombinator.com/item?id=10234849

The rustnomicon is a very good resource. It's less of "the unsafe book" and more "the advanced book", covering lower-level details and advanced concepts that aren't touch on by the book. One gripe I have is that it kind of pushes the idea that unsafe is bad and evil, which is not true at all:

> The Dark Arts of Unsafe Rust

> THE KNOWLEDGE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF UNLEASHING INDESCRIBABLE HORRORS THAT SHATTER YOUR PSYCHE AND SET YOUR MIND ADRIFT IN THE UNKNOWABLY INFINITE COSMOS.

I know it's a joke, and I laughed when I first read it. Unsafe is dangerous and requires care, but unsafe == evil is definitely not true. Jon Gjengset said it well in "Demystifying Unsafe Code" [0]:

> One thing I keep observing about the Rust community is how we're all like allergic to unsafe code or we think that unsafe code is is totally fine and not nothing to worry about, and I wanted to take a little bit of time to talk about what unsafe is because I think a lot of where the communities lack of alignment on this particular topic comes from a fact that unsafe is a bit of a mystery to many of us

Learning about unsafe is useful, and at time necessary, it's not inherently evil, nor should it be used too freely.

[0]: https://www.youtube.com/watch?v=QAz-maaH0KM

I do understand the point you are making, but if the goal is to discourage use, the tooling should make it easy for downstream users (other developers) to know that a certain dependency "relies on a smart programmer doing the right thing". My 2 cents - some sort of `unsafe` usage score??

I am suggesting this to prevent the likes of the burn-out that happened with actix-web and its use of unsafe (which is now fixed apparently)

There exist unofficial tools for counting the number of unsafe blocks in a project: https://github.com/rust-secure-code/cargo-geiger

However a sufficiently determined evil crate can use soundness holes (like fake-static) or macros (like plutonium) to misbehave without visible unsafe.

> However a sufficiently determined evil crate

Nothing really can stop a truly determined bad actor completely, but I don’t think that was GP’s point, rather that it’s good to easily know the potential risk you are exposing yourself to with your dependencies in a practical sense.

The issue with actix-we wasn't about people badgering the author to remove unsafe and they got tired of it/burned out and tried to find people to take over? I remember that the unsafe code wasn't really necessary with regards to the performance benefit.

Unsafe is not evil, however as far as teaching beginners what to stay away from until they know better, I can understand why people jokingly say that it is =)

It's just a tool of course, so the perspective that it's neither bad nor good is justified to an extent. But it's not that unsafe is complicated, it's that it's full of unknown unknowns.

If you don't know any better, it's _really_ easy to write something not exception-safe, accidentally trust safe code a little too much to maintain invariants in your in unsafe code, or to break some other subtle assumption that prevented LLVM from merrily miscompiling what's in your head.

Unsafe code is a necessity in Rust, but I actually really like how it's abstracted for a beginner. For the first few programs you write, you should be doing it the way Rust wants you to, so you can get a better understanding of Rust's ergonomics and what it's development workflow looks like. Introducing unsafe code too early could end up with people programming stuff like the Actix engine, which used unsafe code so liberally that it eventually became too difficult for the founder to maintain.

I'm no stranger to unsafe code but in my professional work, I avoid it almost any cost. In my experience the common denominator among average engineers is an aversion to the kind of processes that derisk unsafe code, where average == "I can reliably find at least two engineers at this skill level within the next few months with competitive but not FAANG-level salaries". This is why Rust is so successful in the first place: it's a low level language disguised as a high level language (or the way around, depending on your perspective) where you don't have to worry about fuzzing or vagrant. That `unsafe` is avoided like the plague is a great feature of the Rust ecosystem and IMO an achievement in its own right.

I've actually found the reverse problem to be even more detrimental: people coming from high level languages like Python are getting tripped up because they try to go too low level to make up for tradeoffs they no longer have to make. Rust makes it painfully obvious when wrapping a shared value in a Rc+RefCell/Arc+Mutex or cloning so it seems like an antipattern to newbies. Even though the majority of them are coming from languages where everything is implicitly reference counted and performance is abysmal, they worry about incrementing an atomic integer or acquiring a lock in a program that will only ever run on x64. I can't imagine what they'd come up with using `unsafe` if the community didn't literally refer to it as the dark arts.

I've found I constantly want to build off of an existing c or c++ library, so I need unsafe. I'm much less experienced than the people's code I'm using, and I'm generally fine with "Rust protects against me, if the lib authors screwed up then bad things happen".

Absolutely. I don't base my choice of libraries solely on `unsafe` but at work the number of people who will review internal unsafe code is a fraction the number of people that would review even a moderately popular open source library.

IMO, using `unsafe` in order to do FFI is of a fundamentally different character than pure-Rust `unsafe`. To the compiler they're the same thing ("I can't verify this, be careful"), but to a reader/auditor the former is fairly reasonable ("I can see why it might be pragmatic to not have to rewrite this entire C program in Rust"), whereas the latter should raise eyebrows ("why, exactly, does this need to be `unsafe`?").

(That said, you obviously still have to be cautious when doing FFI.)

I thought the same, but it seems to me the logical conclusion is to mark "I've added no unsoundness compared to if you were using the library I'm binding to" as safe, and add in the docs homepage a general vibe of how sound the bound library is. When I suggested this on the Rust Reddit enough people thought this was a fundamental insult to the principles of what Rust means to them I decided not to do this.

Do you have a link to that discussion?

https://www.reddit.com/r/rust/comments/mxiufr/when_soundness...

I was one of the people who commented on that post. My point was basically that "you need to uphold invariants in order for this to be sound" is exactly what `unsafe` means in Rust. So if you're wrapping a library that doesn't guarantee safety then you should mark it as unsafe (and there's nothing wrong with that).

Rust library users will typically assume that they can do absolutely whatever they want with a safe interface and they cannot possibly cause memory safety issues, undefined behaviour, etc. A large part of the benefit of Rust is not even having to think about that. So it's important that libraries continue to stick to this convention.

I think the discussion on Reddit you're referring to was saying something slightly more nuanced than that. The case I specifically remember was someone arguing that simply passing through a C library function call without making it clear certain invariants needed to be held in the calling Rust code to not cause undefined behavior was somehow "safe". If you're wrapping a C call with some Rust code and you don't ensure all the invariants that could crash the C code are taken care of, then your function is still unsafe. If you do ensure that all the invariants are taken care of that could crash the C code are taken care of, then it's fine to have it be safe.

There are tools to automatically generate "safe" C++-Rust bindings (the intent being to restrict unsafety to trusting C++ code to be sound, without the risk of screwing up the bindings): https://github.com/dtolnay/cxx and https://cxx.rs/.

However, creating bindings without typing out `unsafe` was a controversial issue, discussed at https://www.reddit.com/r/rust/comments/ielvxu/the_cxx_debate... .

Also there are tools to automatically parse C++ headers and turn them into cxx bindings: https://github.com/google/autocxx and https://docs.rs/autocxx/

When you use that sort of thing you pretty much always need to handwrite some bits as well.

Did you mean Valgrind or is there a Vagrant besides the VM manager?

Yes, I got my V* tools mixed up

Overuse of "Rc+RefCell/Arc+Mutex" IS something of an antipattern though. In C programming you try and avoid doing too much of that kind of thing as you can get stuck in "undefined behavior" whereas in Rust you'll instead end up coding yourself into a circular dependency or some other construct that will never compile. So I agree that you should avoid using them other than where needed in Rust.

Definitely this is very common. I've seen static variables with Arc+Mutex and initialised with lazy_static when they're only used once in a single thread, in multiple jobs.

Unsafe code isn't evil, but it is scary, and I think that's what the 'nomicon is trying to get at. The reason it's scary is the risk of undefined behaviour, which is perhaps even more dangerous than it is in C, because you can completely violate the invariants that safe Rust code relies on.

> Unsafe is dangerous and requires care, but unsafe == evil is definitely not true.

The quote doesn't say it will unleash indescribable horrors. Just that it provides no warranty that it won't. That's entirely fair considering what optimizing compilers will do when you break a contract. So it doesn't say unsafe == evil. It just says evil things may or may not happen when you use unsafe.

I think the documentation should be very honest about how bad it actually is to use "unsafe". You don't want people reaching for it every time they think they're stuck without it, but at the same time you don't want people thinking they've written their code wrong because they had to use it.

I think linked lists are a great example of something that causes Rust's ownership model to fall apart. I've seen it done with tradeoffs, but it's something that you're best off implementing with pointers and unsafe blocks (though you probably should just use a battle-tested implementation or consider if you'd be better off with a vector for cache locality).

> I think linked lists are a great example of something that causes Rust's ownership model to fall apart. I've seen it done with tradeoffs, but it's something that you're best off implementing with pointers and unsafe blocks

It's worth checking https://plv.mpi-sws.org/rustbelt/ghostcell/ and https://github.com/matthieu-m/ghost-collections for an alternate approach that's currently being worked on.

Quite non-intuitive and it has yet to be proven 100% safe, plus it doesn't actually obviate everything you might want to do w/ potentially-aliased pointers, meaning that some desirable patterns are still off-limits - but it has the best chance of working out so far.

I think one of the big existential questions for Rust early on was whether the "unsafe is evil" attitude was viable. If avoiding unsafe was very inconvenient, everyone would've kept writing unsafe code all the time, and this part of the culture wouldn't have gotten traction. Today I think it's easy to say "we all know not to use unsafe in 'normal' code", but that wasn't always a given, and it might take some more ongoing effort to keep this culture in the long term.

After too many years doing C/C++, I became allergic to off-by-ones, use-after-free, etc. Oh, I still enjoy an occasional carefully crafted unsafe piece of code, but it really must pay back. Otherwise I'm just lazy and prefer to be protected by the compiler. I do too many mistakes.

Sadly, IIRC the Nomicon hasn't seen a new release in a while (meaning there's unpublished Github commits at https://github.com/rust-lang/nomicon ), and there are some TODOs scattered throughout the document. It has been incomplete since 2016, as mentioned at https://news.ycombinator.com/item?id=12066259 .

For example, https://doc.rust-lang.org/nomicon/send-and-sync.html says:

> TODO: better explain what can or can't be Send or Sync. Sufficient to appeal only to data races?

Coincidentally I happen to have just written about this topic a few months ago: https://nyanpasu64.github.io/blog/an-unsafe-tour-of-rust-s-s...

Yeah I found reading it a bit underwhelming considering it's mythical status. There isn't that much material and interesting things like Arc aren't covered.

Update: Turns out the Send/Sync article did receive an update giving more information/guidance... on GitHub... and not on the actual published website. https://github.com/rust-lang/nomicon/commit/8551afbb2ca6f5ea...

Update 2: The changes to the GitHub repository are only being reflected in the nightly Nomicon, not the main webpage. Link to update: https://doc.rust-lang.org/nightly/nomicon/send-and-sync.html...

Update 3: Perhaps they are being reflected in the main webpage, but with a delay (possibly once per language release?)

I wrote that with a lot of help from the excellent maintainers. They were very welcoming of a PR that needed a lot of work.

If you're interested in this sort of thing John Gjenset's YouTube channel is also a great resource.

Yeah, considering it's mythical status, I found the material rather thin and uninformative :/

Love the introduction for Transmutes

> This is really, truly, the most horribly unsafe thing you can do in Rust. The guardrails here are dental floss.

A typical large C++ code base will have hundreds of reinterpret_cast<T> or C code base with void*. It's not that horrible.

IIUC, transmutes in rust completely destroy lifetimes. Unsafe in rust gives you a few more things you can do, it doesn't do stuff like disable the borrow checker. But transmute basically kills lifetime checking.

Transmuting an & to &mut is UB.

- Transmuting an & to &mut is always UB.

- No you can't do it.

- No you're not special.

I remember reading this exact part of the 'nomicon and being really put off by it. I understand it's a joke, but it's a bit tasteless, and terribly condescending. I was surprised by how much it bothered me.

Goes to show that people can be offended by anything (:

I'm not sure why it's surprising that some things might bother some people but not others. (?)

Anyway, I thought the passage was an interesting reflection of the perspective of the 'nomicon authors. I'm fairly new to rust, and I've observed the culture – or at least, some part of the culture, and only as it's visible to newcomers – to be astonishingly friendly and helpful, but also sometimes, well, condescending. I sometimes get the occasional vibe of "this is bad and you want a bad thing" instead of "let me acknowledge your concern and here're options for addressing it" (with all the usual caveats: this attitude isn't by any means universal or even exemplified by a particular individual; obviously, the rust community doesn't owe anyone anything, so don't interpret the complaint as a demand; &c).

I wonder whether other people have had a similar perception.

I guess I am used to this sort of rhetoric around UB because it is just something that should not be done. I'm assuming you're not coming from C++?

haha C++ is actually my primary language at work ^_^ and UB crosses my mind pretty often: there've been times I've read through the spec to make sure I avoid it.

My feeling isn't really about `unsafe` in particular, though; it's more general. (Although I do see the potential conflation of "safe = correct, unsafe = incorrect" as problematic.)

I view it more as "sound = correct, unsound = incorrect" (where unsound code is code that breaks Rust's rules and/or allows LLVM to generate malfunctioning code). The design goal of the language is that all code without unsafe is sound, and unsafe code

I also feel "moralistic" about soundness, and I do see that in the community. You could argue that such moral thinking is misguided, IDK.

And there are cases where people deliberately use UB to improve performance, like https://internals.rust-lang.org/t/bit-wise-reasoning-for-ato... and https://internals.rust-lang.org/t/unordered-as-a-solution-to.... I'm not sure how the core team, or community, views such situations.

Additionally, async fn desugarings and Tokio's intrusive linked lists (self-referential objects that can create &mut references) are unsound under Stacked Borrows (Rust's currently popular "formal memory model" for deciding what code/optimizations are sound), though I've heard promises that Stacked Borrows will be adjusted to declare these as sound.

I feel that way about the disclaimer on the homepage, but the transmuting to &mut disclaimer is literally true.

Apologies if you're already familiar with this: It doesn't mean it's not sometimes necessary to have two `*mut T` to the same location, it's that the compiler makes specific assumptions about `&mut T` such that two &mut to the same location existing, even if never used, is UB. To work around this you need to stay in raw pointer land and use APIs like ptr::write.

FYI, Carol Nichols mentioned this in a Rust video [0] and I thought it was pretty interesting! (yes, previously submitted to HN, but long ago with little traction so I think this is within HN guidelines).

[0] https://www.youtube.com/watch?v=A3AdN7U24iU

I like that the site turns black with dark mode.

What a great name

Very good write-up. Really good. Thanks OP.

What is the format this page uses?

This is formatted using a program called mdBook, a tool in rust for converting markdown to HTML books: https://github.com/rust-lang/mdBook

Uhm... Yes, Rustonomicon.

Reply on news.ycombinator.com