An alternative introduction to Rust
words.steveklabnik.com
words.steveklabnik.com
Something with no GC for hard real-time systems would fit the bill, also something that I could write performance-critical libraries with a C ABI like Rust to call from Python. I've had my eye on Rust and of course C(++), but mostly waiting for Rust to stabilize before diving in.
Guides like this are going to mean a lot in the coming months for people like me who are Rust-curious but don't have a systems programming background.
I'm not so much looking for yet-another-scripting language or GC'd language but a systems language that perfectly augments my Python/SQL knowledge.
Thanks for doing this, Steve.
C++ has it easier in practical terms, I guess—few projects actually use smart pointers as pervasively as modern C++ guidelines stipulate (and they're hard-wired into the language in a couple places like operator new and this), so C pointers usually have to be taught pretty early anyway. In Rust, by contrast, unsafe pointers are considered evil things you touch only when you know you have to. But the problem is still there: it feels like something that comes with the territory of manual memory management in general.
[1]: http://stackoverflow.com/questions/15384443/doubly-linked-li...
Anything else is unsafe, specially if the pointer is allowed to escape the RC object that contains it.
And this is one of the reasons why usually optimized GCs outperform RC solutions.
Unless approaches like affine types or counting elision are used, but those fail down in the presence of cycles, forcing the developers to use other approaches to represent cycles.
weak_ptr is basically for managing leaks, not for avoiding reference count overhead. Leaks aren't the problem in a doubly-linked list, however.
It just got me thinking into having some kind of data structure examples in Rust as side project, after 1.0 availability.
Now just have to find time for it.
Well, I wrote our existing introduction, this is a different version, that takes a fundamentally different approach: small example, high level of detail, rather than larger examples at a higher level of detail.
Your point about it being an introduction stands, but that's why it's an alternate _introduction_.
Line six has a closing curly brace, and so main, and thus, our program, is over.
Could be rewritten as:
Line six has a closing curly brace, and so main is over. By extension, our program is over as well.
Maybe even with a quick reinforcing of the (hopefully earlier explained) concept that main is the main program block.
It's a fine line to walk, making the text interesting while also making it very clear, but I think there's something to be said for making the English easy to parse, since the people going through the tutorial will have enough on their hands dealing with the code examples.
Are CS majors typically required to take a technical writing class like other engineering disciplines?
We also don't distinguish between CS and Software Engineering degree.
I would then go further than you suggest, saying something like, "The blank lines, 1,3 and 7, do not have any impact on the program." Then "Line 4 binds the value 5 to the variable x.", followed by the explanation of what that is called in rust.
After that, "Lines 2 and 6 are the beginning and end of the program. For this program, they also define the scope that owns 'x'." Followed by the explanation of ownership.
Once 'let' is established, a short explanation of 'mut' can follow.
From someone reading about these details for the first time:
Calling them variable bindings confuses the matter. You say, just moments earlier, that variables are called by that name because they can change over time, they’re mutable. But this variable is immutable? But only sometimes? !
I know you mean that the mutability varies and that it makes sense in the context of the jargon of functional languages, but that's something that will make sense to language developers who have worked on Rust for years and maybe to the HN crowd, and not to new coders trying to grasp difficult, novel (to them) concepts.
How about "multimode bindings" or "multiphase bindings" (as in liquid/solid phase transitions) or dual-mode or something more entertaining like "double-agent bindings". Is it too late to change?
Maybe I'm missing something, but I suspect years from now people will be saying, 'so variable bindings are not always variable? huh?'
One way to view it is that the bindings don't have a value that is fixed at compile time: they vary between runs.
In any case, I'm not sure what you mean by "mutability varies".
> My only concern is that lot of the motivating of the ownership
> system seems to be directed at C/C++ programmers, but at the same
> time you're explaining pointers and memory management as almost new concepts.
Writing is hard.
For example, going through line by line leads to a slew of text that doesn't really provide that much value. I feel you could cut a few paragraphs with a simpler method of explanation with perhaps more code (and explain through comments), visuals (ascii diagrams of the scopes), or something.
I also feel the analogy towards lifetimes relating to segment of lines of code might confuse users.
> Rust knows how to connect the scopes of variables that are pointing to the same thing, as well as how to know the scope of things that are more than just stack-allocated memory.
Too many 'things', knowing Rust I obviously know what you're trying to say but I could imagine it being hard to grok for new comers. However, I don't really have a suggestion on how to explain it better.
After that, you show some code and try to explain it in paragraph form after that. I think it might be easier to just place those explanations inline with the code. That way you can skip the "at line 7, it does this" type of thing.
Referencing the implementation details of the stack and heap, it seems like too many details that are not really needed at that time. Perhaps a difference between the stack vs heap, but which memory address a variable is located doesn't really convey anything useful.
Anyways, I like the where things are headed, but as you said, writing is hard. I've tried writing introductions to lifetimes and ownership and, well, I just gave up. Trying to convey that stuff after you've learned it to people who haven't is hard.
As [1] says: "Rust for C programmers != Rust for functional programmers."
[1] http://science.raphael.poss.name/rust-for-functional-program...
Being one who has "enjoyed" numerous encounters with C pointers (and pointers to pointers, etc.), a lot of the explanation seemed elementary indeed. But I didn't mind the "review", in fact, kind of fun imagining I was a beginner and how amazing it would be to learn the details.
I can see why Rust's concept of ownership is essential to grasp. However, the tutorial was a little murky when it came to the deeper subjects of "mutable borrow" and the cause of the example error.
It took a few readings to get what made the line "let x = &v[0];" so crucial: "...&mut is a promise that there are no other references...", i.e., meaning the promise to "x", since "x" is the recipient (owner?) of the "&mut" of "v".
The final paragraph is the key to the entire exposition, so it would be enormously helpful to take a few extra lines of text to clarify just how the expression "v.push("B");" violates the promise to "x". That's even more vital to your purpose as describing pointers in vivid detail.
People new to Rust are beginners re: ownership, borrowing, et.al., and would benefit from a step-by-step walk through of these less familiar parts.
I spent the last two weeks reading the entirety of the 30 minute intro, TRPL, and Referecne, and Rust By Example. My background is primarily Python / Java / JavaScript. I can read but am uncomfortable writing C.
This article touches on mutability, references, stack versus heap, ownership, and lifetimes. It has listings of C and C++ code interspersed with Rust. At one point there are nine tables representing memory layout, with no clear indication of what's changed, and only the first 3 tables even fit inside my browser in their entirety.
I understand what you're trying to get to, but I don't think this is that. I'd appreciate detailed sidebars on how Rust avoids dangling pointers / data races / etc., but not part of the main path immediately following "Hello, world." Systems language folks will readily grok that stuff without exposition on what a stack and heap are, while those of us from higher level languages would get stopped in our tracks and completely bogged down in that morass.
Lifetimes and ownership were the most difficult concepts for me to grok, but I was able to make it through TRPL relatively quickly and come out knowing exactly where I was unclear. Rust By Example helped immensely on that front. I specifically appreciated the destroy_box function on http://rustbyexample.com/move.html, though the entire snippet was quite illuminating.
The examples of lifetime elision on http://doc.rust-lang.org/book/ownership.html#examples coupled with the attempts as manually annotating lifetimes in http://rustbyexample.com/lifetime/fn.html finally made that click.
I'm just one learner, but in my experience TRPL is fundamentally sound. I wouldn't fix what isn't broken. It may not sell Rust in the sense of putting its "best foot forward," but it's concise and it works. When shooting for first impressions, it might be worth focusing on the 30 minute intro or some other document instead. I didn't touch TRPL until I had been sufficiently sold on the language from other venues to commit to learning it.
Edit: But holy shit, Steve, thank you so much for the work and care you're putting into this. Rust wouldn't be where it is without you.
Thanks, this was literally my hypothesis for the intro, like qualifying a lead: give enough to get you interested, then either give you more or say good day, depending. Maybe this is just better as the intro itself, or many this draft is at the wrong level of abstraction, we'll see.
I don't know of a succinct overview of Rust for experienced system programmers, and this video certainly isn't that.
But I agree it does feel a bit weird to need to explain pointers while also assuming people understand memory layouts. There should probably be a quick guide to memory usage, maybe even using C, and from there introduce Rust's concepts on top. It's certainly a hard problem to introduce a high-level, functional, language that also has very powerful low-level concepts.
Thanks so much Steve for doing this work. Reading the guide made it all click into focus for me when I started, and the rest is _mostly_ syntax.
When you enforce immutability of aliases, as Rust does, aliasing stops being a special case. The underlying value may be the same, but the behaviour is identical to when they are independent.
The only time you have problems thereafter is when you do weird things with the value's address which only coincidentally works. For example, comparing interned strings or Java's Integer objects using == works for other interned strings and small integers, respectively.
Along with that, it's really hard for me to get used to right-hand line numbers. Shoving them along the left hand side like most editors would be much much much easier to read for me, at least.
That said, I do enjoy your writing and find it quite readable.
I do agree better diagrams would help, for sure. And yeah, the right hand numbers were kind of a compromise: they kept the exercises compileable without needing to mess with markup stuff, which is just a weakness of markdown :/
My experience, having started in C++, then gone to scripting languages, then come back again, is that it can be quite helpful to consider what actually happens under the hood in a scripting language. Because Python's references are more limited, you would never exhibit the same bug that you illustrated in C++, but there are still issues around e.g. aliasing which tend to trip up inexperienced Python programmers, and where you could illustrate interesting things about Rust's ownership model. And boxed/unboxed data types plus reference counting (in Python) would give you an excuse to talk about RAII. So I think that you could do this in a way that would actually speak to savvy scripting programmers, while still being definitively systemsy.
Just a thought.
But I do like the idea of focusing on ownership and lifetimes more front and center, since that's rust's raison d'etre.
Can you get a hold of Niko's slides from the NYC meetup this week at Bloomberg? I was there, and his overview of ownership and borrowing, using the examples of a book, were actually really helpful for me. For whatever reason, his approach really resonated with me.
Things to cover:
Basic lifetime concepts in Rust:
1. Scopes are nested.
2. Variable lifetimes are tied to scopes.
3. Unlike other languages, variables can be created
tied to a scope outer to the one where they are
created. The user can control this by indicating
that the lifetime of a variable is the same as that
of some variable from an outer scope. This is Rust's
big innovation.
4. A function can create a new object and return it by
reference if the function explicitly says that the
new object has the same lifetime as some parameter.
1. There are three kinds of ownership - single ownership,
reference count multiple ownership, and thread-safe reference
count ownership.
2. The reference count ones work like you'd expect.
3. Single ownership interacts with the lifetime system in
to make it possible to create data structures such as
trees, with all components single ownership and having the
same lifetime.
4. The "=" operator usually means "move", and when it does,
the right hand side of the "=" becomes inaccessable.
(This is type-dependent; some types copy when "=" is used.)
This preserves single ownership.
1. Rust has enums, which, unlike enums in other languages, can
have data objects associated with them. They're really variant
records.
2. You can declare an enum with a data object as a field in a
structure. This creates a structure that contains (really links
to) another data object.
3. This can be done recursively, so you can create trees of
structures.
4. You can't violate the rules of single-ownership. Trees are OK;
directed acyclic graphs or backpointers are not allowed using
single ownership.
5. There's no "Null". Instead you create enums which have one value
with an associated data item and one without.
6. Everything has to be initialized to a valid value for its type.
1. You can create temporary references to something with an equal
or longer lifetime, because the temporary "borrowed" reference
cannot outlive the thing it is referencing. That's safe.
2. You can have one writer (a mutable reference) or N readers
(immutable references) at a time.
1. It's complicated.
2. There's automatic type inference. Not just for "let", either.
3. Function parameters have to be declared, not inferred,
except for the special syntax for closures that lets you write
concise lambda expressions. This limits how hard the
type inference system has to work.
4. I don't fully understand Rust's type system yet.
1. All of the above is enforced at compile time.
2. The result is memory safety.
- Lifetimes are novel, but not that complex.
- The type system really is complex.
- There are generics and you'll need them more often than in C++.
- The compiler first checks syntax, then types, then borrowing. It does not report
borrow errors until everything else is correct.
- There are no exceptions, and proper error handling requires rather verbose code.
- Have a plan for your variable lifetimes before you start coding, or you end up
fighting with the borrow checker and losing.
- If you think you need "unsafe" in pure Rust code, you're doing it wrong. 3. Unlike other languages, variables can be created
tied to a scope outer to the one where they are
created. The user can control this by indicating
that the lifetime of a variable is the same as that
of some variable from an outer scope. This is Rust's
big innovation.
Given
let x = 1;
{
let y = 1;
}
4. The "=" operator usually means "move", and when it does,
the right hand side of the "=" becomes inaccessable.
(This is type-dependent; some types copy when "=" is used.)
This preserves single ownership.
2. You can declare an enum with a data object as a field in a
structure. This creates a structure that contains (really links
to) another data object.
- The type system really is complex.
Any tutorial on this needs to be illustrated. Talking about line numbers is not enough.
As I said above, there's no lifetime annotations you can use to ensure that y lives as long(aka is in the same scope) as x in the following
let x = 1;
{
let y = 2;
}
Lifetimes are a purely passive thing, designed to allow the compiler to check the programmer's use of references, not to allow the compiler to massage things around to ensure that whatever use work.
That is, if a Rust program compiles, then it will also compile (modulo needing extra `unsafe` annotation) and, importantly, do the same thing, if all the references are replaced with raw pointers, which don't have lifetimes to describe their relationships.
struct Foo<'a, P>
where P: std::ops::Neg + 'a,
<P as std::ops::Neg>::Output: 'a {
foo: &'a Bar<P>,
}
In any case, half the "complexity" in the example you give is the noise of using the fully-qualified std::ops::Neg path in the bounds, rather than importing it and just writing `Neg`. :)
use std::ops::Neg;
struct Foo<'a,P> where P: Neg + 'a, <P::Neg>::Output: 'a {
foo: &'a Bar<P>,
}
use std::ops::Neg;
struct Foo<'a,P> where P: Neg + 'a, P::Output: 'a {
foo: &'a Bar<P>,
}
`<P::Neg>::Output: 'a` gives me eye cancer and my intuition is if `P: Neg + 'a` then `P::Output: 'a` is implied (why would it not have that bound if P does?)
the Neg trait's Output IS Self (as Neg)! Why is the bound not transitive?
What if the output of the unary minus method was say, a constant 'static? This is a reasonable thing for a function to want to do in general. (Probably not for Neg in particular, but obviously we don't want to add special cases in the inference for specific traits.) The function needs to know about it because you can do many more things with a 'static than a 'a (as 'static references can never become dangling).
But please make it clear that the tables don't accurately describe the real layout of memory in a compiled Rust program. Heap mappings and thread stacks are typically at random (thanks to ASLR) addresses, rather than the start and end of memory, and the locations of specific heap allocations are usually based on complicated rules depending on the size class of the allocation (differently sized objects never get allocated together in most allocators), thread caches, ordering of buckets within a page, etc. (Some architectures also grow the stack the other way.)
Now, I'm in no way a writer. English isn't even my main language.. So take this next point accordingly:
Unfortunately, I get an even more condescending vibe from this draft than I got from the book. Not as in "I'm an expert and you're treating me like a newbie," but as if I was a kid instead of an aspiring Rust programmer (beginner or otherwise). I wish I could point out -exactly why- I get this feeling, but I'd be lying.
I do think I'm starting to see a pattern, though, that perhaps contribute to this gut reaction. The pace between concept introductions is odd, somewhat erratic. At times, in one line you're talking to a complete novice programmer, and the very next you assume they know other languages and general concepts [1]. Other times, you take some basic knowledge for granted and explains it right after [2]. I know it must be very hard to find the right balance for your target audience, but I found these jumps very jarring. For what it's worth, once you get past the introductions you pick up a more to-the-point-explanations rhythm and that reaction goes away.
[1]: For example, after several paragraphs and two code snippets to introduce the concept of 'variable bindings' in a novice-friendly way, even going as far as (somewhat) explaining what system and functional languages are, the text suddenly assumes I know about program execution and what a control flow structure is?
> As you know, a computer program is executed line by line. At least, until you hit a control flow structure, anyway.
This line was specially off-putting. After all that explanation, the assumption I'd know this made me feel like a kid. But I can only imagine how inaproppriate I'd feel if I didn't know. Given the novice level of the text up to then, the 'as you know' sounds very out of place.
[2]: The very next paragraph, we go over a program with one variable binding line by line. And then, it's assumed I know about scopes. But after that they are explained anyway. I realize this is a way to go over every concept, even if one already knows them, but it feels odd! Both beginner and experts would understand a plain explanation. But this back-and-forth about how to present concepts doesn't work -- for me. It'd make more sense to read straight-forward explanations of what I already know than these "as a programmer, you know that x," mixed with even simpler concepts being introduced as new.
This sounds harsher than I planned -- I'm no writer afterall! But, really, despite these gut reactions, the Rust book is turning into one of my favorites. Thank you for all your work!
It's hard, for sure. There are certain sentence structures that strike certain people this way, and my style sometimes triggers them. For example, "basically" has no negative connotation for me, but really gets some people going, so I've been eliminating it wherever possible. I wonder which aspect does it for you... thanks for the brain dump, this is helpful.
I'd love to see this kind of explanation done in a non-text medium. For example, a slideshow or interactive webpage (ex: http://www.redblobgames.com/pathfinding/a-star/introduction....)
I'm not sure jumping so much into what's going on in the C++ side is a great idea. The section as it stands now covers the important points (with out lifetimes invalidating pointers is easy) without getting too lost. However, I'm already pretty familiar with C++, so the more brief explanation may not be enough for everyone.
1) There are many memory schemas for the push_back example, maybe you want to organize them side-by-side with an arrow between them showing which statement caused the change. Saving some vertical space may do some good.
2) I'm not a fan of the sentence structure in many places. Needs more pith. Perhaps apply some Strunk & White and some K&R.
The only quibble is that I'd make the middle slog a little bit shorter, maybe laying out the vectors horizontally instead of vertically, so people wouldn't forget what the text was about by the time they get to the end of the diagram.
Yeah, the layout could be much better, and it'd be neat if I could use real diagrams with arrows as the pointers, maybe.
Also, is it standard to talk about memory growing "down" as the memory locations numerically increase? That's the opposite of my impulse (I would have said that 0xff is the top of memory, and 0xfe is "down" from there) but TBH it's been a while since I took the relevant CS class.
I'm sorry, but this implication enrages me.
If I'm just trying to print out "Hello World", why would I use some kind of list and a pointer?
You touched on an example of where Rust simplifies non-GC memory management.
Its nice that there are various ways to track when you are done with pointers (and memory referenced by them) across the code.
I think some of the techniques are very useful. But its taken to an extreme in Rust where you will do anything to avoid GC.
Some of that stuff involving tracking and matching ownership or correspondence across the code would work better outside of a purely sort of one-dimensional textual representation. This is an example that illustrates why textual source code is a legacy.
But it IS text, and often, with Rust, its too complicated, and often not easily applied to typical "systems" programming (which is quite often used to actually mean some ordinary types of user applications, or programs that do the same thing).
Personally I think Rust could maintain a good portion of its core strength while losing about half of its complexity.
Also, in terms of an introduction it would be nice to see some real useful "systems programming" examples where the core memory tracking and other capabilities make Rust stand out. Hello World isn't a good example of this.
We want the compiler to be smart and be able to avoid collecting things at runtime when possible and practical, and for the programmer to be able to effectively indicate corresponding data so the compiler will know when it should be deallocated. However, we also need the code to be clear and concise and to avoid making the programmer do routine work when its not necessary.
> while avoiding about half of its complexity.
How?
However, since it's a GC, it's probably not possible to have a pointer that refers to a memory location that doesn't exist, because having a pointer to a memory location is by definition what keeps it from being free'd by the GC.
I guess there is a typo and should be "four through six" instead.
becuase
langauges
ownerhsip
I 'kudos'ed your post though - because I thought it was very nicely written!