Visualize Ownership and Lifetimes in Rust | Hacker News Reader

Visualize Ownership and Lifetimes in Rust

github.com

222 pointsljahier59 comments

How many of those advanced concept in Rust such as borrow checker & lifetime could be avoided as a beginner? If utmost performance is not required (i.e., for mortals dealing with Java/Python/JS on a daily basis) would those lifetime and borrow checker concepts be a hinder to move quick?

I read this article https://corrode.dev/blog/prototyping/ and it seems to address almost all of my concerns when I started learning Rust. I think it could take a beginner a long way until they need to get down to the borrow checker/lifetime. I said I agree with the blog "mostly" because there are situations where you have to interact with 3rd party libraries or APIs dealing with those lifetime. The, you need to know about those concept. If you know a better way to handle this, please let me know.

The borrow checker and lifetimes aren't simply a matter of performance, they are a matter of correctness. Languages without them (go, java, etc) allow for bugs that they prevent - dataraces, ConcurrentModificationException, etc. The fact that you can only write through pointers that guarantee they have unique access is what lets the language statically guarantee the absence of a wide category of bugs, and what makes it so easy to reason about rust code as a human (experienced with rust). You can't have that without the borrow checker, and without that you lose what makes rust different (it would still be a fine language, but not a particularly special one).

You could simplify rust slightly by sacrificing performance. For example you could box everything by default (like java) and get rid of `Box` the type as a concept. You could even make everything a reference counted pointer (but only allow mutation when the compiler can guarantee that the reference count is 1). You could ditch the concept of unsized types. Things like that. Rust doesn't strive to be the simplest language that it could be - instead it prefers performance. None of this is really what people complain about with the language though.

I recently came to this realization in a large typescript codebase. It's really important to understand who owns data and who has the right to modify it. Having tools to manage this and make it explicit built into the language is so helpful for code correctness and is especially beneficial for maintaining code you didn't write yourself.

Another great way of handling this if you cannot switch out the language, is to start adopting a more functional approach and also try to keep mutations into one place/less places. So instead of having all the X services/adapter/whatever being able to pass data around that they mutate along the way (like the typical "hiding implementation details in objects/classes"), have all those just do transformation on data and return new data, then have one thing that can mutate things.

Even if you cannot go as extreme as isolating the mutation into just one place, heavily reducing the amount of mutation makes that particular problem a lot easier to handle in larger codebases.

Right this is what I tried to do but unfortunately trying to mark everything immutable in Typescript leads to some very unergonomic type signatures. Hopefully this can improve in the future.

Could you show an example of what you mean? Not sure how not mutating data would lead to more unergonomic type signatures, I'm sure an example would help me understand. Although it wouldn't surprise me TypeScript makes things harder.

You are polluting every variable signature with `readonly`. This also can create cascading effects where making one function accept only readonly variables forces you to declare readonly elsewhere as well. Quite similar, in a way, to Rust.

If I have a complex structure MyStruct that I make recursively readonly it doesn't show up in the IDE as DeepReadOnly<MyStruct> or something like that. It shows up as a huge tree of nested readonly declarations, so the original type is highly obscured.

Couldn't the same guarantees be achieved with immutability? Of course this would be setting aside concerns with performance/resource usage, but the parent is describing an environment where these concerns are not primary.

Personally I find it much easier to grok immutable data, not just understand when concentrating on it, then ownership rules.

Absolutely, with full immutability the borrow checker doesn't give you much at all.

It also doesn't cost you much, the borrow checker just gets out of your way if you just wrap all your immutable data in reference counted pointers (try out imbl [1] for instance). It's not free - there's some syntax overhead compared to a language that was intended to primarily work this way - but it's cheap.

I think it's reasonable to view the borrow checker as a generalization of immutability. Immutability says "no mutation", the borrow checker says "no mutation unless you are the only thing that might be accessing the data". Edit: Worth noting though that the rust standard library and ecosystem is a take on this that doesn't emphasize staying within the fully immutable regime as much as it could, instead preferring to improve performance. A variant of rust that tried to explore keeping more things immutable would be interesting.

Personally my take is that there are some problems which are very naturally represented immutably, and there are others that are very hard to fit in that framework. The borrow checker is general enough to capture almost all of that second category as well. But if you're firmly in the first category, and you aren't worried about every last drop of performance, there's probably some managed language with strong immutability that is a better fit.

[1] https://github.com/jneem/imbl

Ownership isn’t an advanced concept. It is a software engineering problem, not a rust problem. Rust is one of the few languages which make it explicit and even checkable at compile time and the first popular one.

What is hard is designing systems in a way resource ownership can be tracked and controlled without impacting performance. Rust makes it possible, but you can use smart pointers to give up speed and take simplicity instead. Most other languages assume (rightly so) you’re too dumb to do it correctly and give you smart pointers by default; some assume you’re smart enough and are proven wrong all the time (this is assembly and C relatives; actually they say ‘we don’t want smart pointers and we want a simple compiler, sucks to be you’).

It very quickly becomes a special Rust-only software engineering problem. Rust has no partial borrows and this affects many designs where a lot of data needs to access fields of other data. Consequently, you see humongous large, flat structures in many Rust projects. And of-course, the famous "replace references with array indices" and just skip the borrow checking and lifetime rules by simply making your own custom pointer system - which is also common in many Rust projects and famously popularized by the "Object Soup is Made of Indices" Rust post here on HN.

I assume you mean these are bad things; I see it as 'ownership enforcement pushing architecture towards memory safety' thing. Path of least resistance changes for the better - if you don't want to use a Box or a RefCell, that is.

Yes, these are bad things. The extreme burden imposed by lifetimes and the prevention of easy refactoring for changes causes spectacular design bloat via workarounds and safety circumvention mechanisms which are unique to Rust projects. Its a special and necessary Rust skill.

You only need to circumvent safety if you have it...

You can choose to have it in runtime. You don't get that choice pretty much anywhere else. If you don't want to make that choice in a granular way as rust allows, pick a language from the other two groups.

No, Rust definitely does add some additional complexity on top of the inherent complexity of ownership. Despite what some people think, Rust's borrowing rules are actually extremely simple. So simple that they reject a lot of safe programs.

Paradoxically, programmer life would be made simpler if there were some more complex borrowing rules, that would allow (for example) partial borrows of objects, or allow aliasing &mut in single-threaded circumstances where it's known to be safe (i.e. when the data is something primitive like an int, where it doesn't actually matter if it's overwritten while referenced).

But I know there's extra language design complexity that this introduces, and extra codegen complexity (Rust makes certain aliasing promises to LLVM that it isn't allowed to break) so it will take time. But, there are proposals in the works.

> or allow aliasing &mut in single-threaded circumstances where it's known to be safe (i.e. when the data is something primitive like an int, where it doesn't actually matter if it's overwritten while referenced).

Incidentally this is basically what the `Cell` type does. I suspect that making it the default wouldn't make it harder for me to reason about the code I'm working on - but it is an interesting proposal.

Partial borrowing is not that much of a problem for the borrow checker. It is a problem for bikeshedding core language developers, apparently...

I've tried to wrap up my philosophy on how a significant chunk of rust code can be written without lifetimes using shared and sharedmut primitives.

I've shipped three projects on it and they are pretty much as performant as they can be. I've never regretted skipping the lifetime work in application code.

https://github.com/mmastrac/keepcalm

I still dig into lifetimes for a lot of true low-level code but it doesn't need to exist at all at the high level

That article on rapid Rust prototyping matches my experience with using Rust as the backend for a web and iOS app. I used clone() and Vec and String and other shortcuts from that article as much as possible since I was building a backend application versus an operating system. It enabled a lot more velocity and made it fun to add features. And it was still blazing fast.

If anyone is considering using Rust and is nervous about lifetimes and bare metal, check out that article and try its guidance. I learned these things on my own the hard way and would have loved to read this article 18 months ago. It's really quite good.

> How many of those advanced concept in Rust such as borrow checker & lifetime could be avoided as a beginner?

As a beginner, you can avoid references (&) and simply clone() everything when it gives you trouble. If you start off by writing simple Actix/Axum web services instead of manually multithreaded apps, the problem domain is inherently linear and you'll avoid lifetimes and the borrow checker almost entirely. This lets you feel productive while getting a feel for the rest of the language features.

Don't do this once you learn the ropes of the borrow checker, of course. Once you grok it, the borrow checker is almost second nature.

I don’t think the basic usage of references is hard to grok for a beginner. If you aren’t going to mutate data and only access it, then pass a reference. No need for over-complicated semantics when describing it to a new Rust user.

What if you want to add a reference field to a struct? That's the point where I usually get pretty upset with how Rust works.

You probably don't want a reference field in a struct, at least to begin with. It's a lot easier to reason about structs if they contain owned data only, and you take a reference to the entire struct. There are some specific cases where it might be sensible, or even necessary to do that, but for someone who is still learning lifetimes, these cases are unlikely to come up.

I recommend reading Steve Klabnik's "When should I use String vs &Str?" post, which is generally good advice when deciding between owned data and references. The "Level 4" section covers the case of references in structs.

You get a lot more compiler help than when you try and put a reference in a class in C++, and if you want to use smart pointers its even better because you'll never have to learn about move and copy constructors or copy and move assignment operators.

I think you will run into the borrow checker pretty soon, because you have to deal with it whenever you deal with references (which you will inevitably have to do if you are dealing with anything more complex than number types). But that aspect of the borrow checker is not that difficult. You could avoid it entirely by cloning everything but IMO it's not necessary and you would do better to invest a small amount of time in understanding why the borrow checker is complaining.

Lifetimes you can probably get further without having to deal with. Just avoid storing references in structs and you will avoid a lot of lifetime headaches. Cloning can again be helpful here.

An alternative to cloning everything, if you are dealing with simple data types, is to derive copy for your structs so you can pass them around without worrying about ownership. It's not always possible though.

Smart pointers are another workaround, as others have said. But my problem with (some) smart pointers is that they simply move the checks to runtime, meaning now your code has a much higher chance of panicking at runtime.

That's basically being the point of the article I shared

The places where you'll usually see lifetimes creep in where you may not expect are in closures (say your app is one big struct, you call a library function that takes a callback, you reference &self in the callback and get an error that the arg type must be 'static, or some other bound), or if you're spawning async tasks and using locks you may get weird Send/Sync and lifetime errors around await and spawn points.

They all make sense if you know why the program doesn't compile, but it may be surprising to newbies.

You can skip lifetimes in the beginning, and I think that's the sane thing to do, if you come from a GC languages.

Borrow checker, well, this actually includes lifetimes. But let's say "basic ownership and basic borrowing", there is no way around starting with that, and it should be a point to learning rust.

You can Arc<Mutex<T>> your way out of ownership/borrowing. At some point, you'll start to see your program as just data (and functional programming will make sense as the only way to program) moving around. I think it's an under-rated way to get started (I started that way) and at some point a bulb will light up and you'll start seeing programming as data moving around and you'll care about ownership/borrowing at the foundational/prototype level.

Yup, that’s definitely correct in terms of avoiding most of the complexity. The tips in that blog post can apply to third party libraries, but sometimes the complexity of the lifetime can leak past the API boundary in a way you can’t ignore (but often you can).

None. And lifetime elision and other forms of pretending to beginners that they don't exist are why so many people have trouble with Rust.

... just write Java/Python/JS, then? Or Kotlin, Typescript, Python+type annotations if you're feeling more modern.

There's nothing wrong with any of those languages (mostly) and not everything has to be written in Rust. IMHO the real value of Rust is as a systems programming language that's safer than C.

Id like a plugin for rustrover.

Rustrover can do this, although unfortunately it requires cargo check/clippy turned on rather than being native (still faster than LSP, at least).

https://www.jetbrains.com/help/rust/rust-external-linters.ht...

I have found clippy better anyway.

Clippy lints are nice. I'm a particular fan of `while_let_loop` which is basically Clippy looks at a Rust loop you wrote and it says "Hey, this is equivalent to while let SomePatternHere = some_function(blah) { };" and mostly you realise oh yes it is, and that's much clearer than what I wrote.

Once in a while I get that lint and I think no, what I wrote is easier to understand, and I just #[allow(clippy::while_let_loop)] to acknowledge that.

The way I end up with these loops is I realise I need a loop - we're definitely doing something here more than once, so in Rust that's loop - then during further development and refinement of the software I get the exact rules correct and then it's obvious (to clippy) that this is a while-let loop, often it's in the form while let Some(thing) = container.pop() { /* do stuff with the thing, maybe putting more stuff in the container in the process */ }; but the loop { } construction is harder to read if left in that form and the whole point of source code is to be readable to humans.

Many languages don't like to give these diagnostics names. The reality is that these names mean something. If you called it 81402 then now that's just arcane knowledge, like if you insisted on calling all the elements by their atomic number. If we call it "Element 26" rather than "Iron" it's the same except harder to remember. I think software which uses numbers here is trying to avoid the semantic value, but that's never going to work, so just embrace it.

It's slow to call out to external linters. But it's even slower to use LSP (which rustrover saves me from), it's really not a huge deal in practice for me.

A family of these plugins would be a true killer app for Rust beginners.

This would be fantastic in helix

Wow this is super cool. Is this pretty reliable for more advanced scenarios? Simple ones I wouldn't need to see this on, but the more complex things I'd be very interested in perusing code to make certain things click.

It's too bad the lifetime of rustc is only about 3 months before breaking changes (new features, updates, etc) are introduced to the compiler and used by devs.

New features aren't usually considered breaking changes, only modifying existing ones so that old code doesn't work.

The rust compiler never strives to never introduce breaking changes (by the definition I just described). It doesn't quite succeed (because some things like correctness are considered more important), but it fails

a) Very rarely, not once every 3 months.

b) In very small ways, that only break a tiny portion of code.

c) In ways that are very easy to fix.

d) Usually the rustc-devs will go offer patches to the entire open source ecosystem before any such release.

I imagine that is a dev's perception (in shipping rust binaries to users) with their rolling OS and constantly updated rustc. But as a user trying to compile rust code written by rust devs what happens is that it won't compile because the compiler is 3 months out of date and rust devs immediately use new features. I had this happen personally often enough it put me off even trying out rust written applications. This is not things I've heard. It is things I've directly experienced.

rustc is a rolling only compiler and that's not great and it does break often (not be able to compile code) in distros that are not rolling. And no, curl|sh and/or rustup are not solutions. I think the only solution is waiting for rust to become popular enough that the proportion of bleeding edge using devs to normal devs goes down.

Right, that's usually described as "forwards compatibility", and yes, rust the language doesn't attempt to supply it. And yes, rustup is in fact a (the) solution for users. Distros increasingly package it because it is the recommended way to use the rust compiler (e.g. if you're on the latest ubuntu or any of its derivatives you can install it with apt instead of curl|sh).

A decent portion of projects, especially bigger ones, will try to support stables going back a few versions, but at best they're just changing the timeline slightly. This is really intended to help packaging for distributions, not users. Users should just use a compiler at least as up-to-date as the software they are trying to build.

Exactly. rustc has a lifetime of about 3 months. gcc has a lifetime of about 5-10 years. Perl's interpreter has a lifetime of about 20 years.

edit since I can't reply: The difference is that Perl devs don't immediately use the new features (and Perl 6 is not Perl). Bash too constantly gets incompatible (forwards) changes, but no Bash dev is so inconsiderate as to use these as they want their software to actually run on people's machines.

That's my entire point: rust's immature/bleeding edge dev culture causes the problem. rustc could be okay, but the culture is too bleeding edge for using rust software now. Maybe in a decade it'll settle down.

Perl 5 introduced forwards incompatible changes less than a year ago. Perl 6 was so backwards incompatible that after 20 years of work they renamed the language to something else - it last had forwards incompatible changes sometime in january.

Not only are these not meaningfully better, they're also the opposite of what I would hold out as a "successful versioning model".

Rustc lasts until you are using software that depends on a more modern version of rustc, just like any dependency. Then you upgrade it - which just like any dependency with backwards compatibility - is painless (and in fact entirely transparent if you use tooling like rustup).

> And no, curl|sh and/or rustup are not solutions.

I'm confused - you are frustrated that the official release channel of software that is used by almost all users of that software is "not a solution" because you want to use unofficial release channels that have outdated versions of the software that are much more rarely used?

Correct. I avoid rust because it is a language where it's pretty much infeasible to have a compiler from your system repositories unless it's a rolling distro. Having some random application require updates so often (like a browser) is marginally acceptable if distasteful. But a compiler and toolchain that exists entirely outside my distro? No thanks.

What I am confused about is how everyone is pretending this is a normal software situation. It's vastly weird and different from most compilers and toolchains. It may be normal if you're coming from web dev but web dev is not normal and not a healthy ecosystem.

It's totally normal except for C/C++ toolchains on Linux. C/C++ compilers are managed this way on MacOS, Windows, and by most embedded toolchains.

Why don't you want your toolchain managed outside your distro? Don't you want your software to run outside your distro?

Longing for a more stable world doesn't sound very weird to me. Maybe core Rust development will slow down in the future.

I came over to Rust from C++ and I really don't want core Rust development to slowdown. The changes you see are mostly useful APIs in std.

Compare to C++, where using a new std API/type can take a literal decade and might not be backwards compatible on targets even once compiled.

Stability is only useful if it gets in the way of reliability and convenience. That's true in C/C++ because of the compilation and development model. In Rust those problems don't exist, so "stability" of std largely gets in the way.

Well, you're never going to have a stable world when you're constantly pulling in updates to your dependencies. In any language.

And if you stop updating those, you stop needing rustc updates.

Right, but in the rest of the world it's more common for the libraries to support a wider range of compiler and language versions. The culture as of yet of Rust, is that more libraries require the newest compiler.

But that wider range by itself means nothing. It only matters in the context of what it enables and how well it enables it.

Reply on news.ycombinator.com