An Intern's Experience with Rust
msrc-blog.microsoft.com
msrc-blog.microsoft.com
We're going to see a lot more of this. As the language has matured, the Rust folks have paid great attention to easing the learning curve. Even with the borrow checker, it's not that far from Java or a scripting language these days. Most people won't hit lifetimes early on in their use.
Depends on what you're doing, doesn't it? If you're using Rust for its original design-goal purpose—as a systems language—then you're probably writing some kind of daemon that keeps persistent in-memory state between RPC requests from clients; or some kind of UNIX-ish CLI tool that reduces over IO streams. And as soon as a request, or a stream event, wants to persist some state into a longer-lived scope, lifetimes rear their head.
Someone with the knowledge of how to write code like this is already familiar with the problem domain and the vocabulary. They'll know enough to be able to find out how to get this performance out of Rust.
New programmers aren't going to look for this or need it right off the bat. They'll still be able to leverage a modern, safe language with lots of features and an extremely compelling package manager.
This turned out to be quite challenging in Rust, even when starting from a working single threaded server. Threading the needle on borrows was quite a challenge.
It ends up being pretty short, though we also build a threadpool by hand. If you used a package, it'd be even smaller. (I like scoped_threadpool myself) (And the FnBox stuff will go away once I write an update for it, it's no longer needed.)
It does the same: grab a connection, spawn the thread. There's virtually no borrowing.
The Rust book actually ends by showing you how to do that, but... there are many reasons why that's the last chapter in the book. Starting there is definitely not the most efficient way of learning Rust.
Because if you did just use Go, in Rust you can just `cargo add actix-web` (or `tokio`, `hyper`, `async_std`, or any other goroutine-like run-time) and write a multi-threaded web-server in approx. one line of code.
What the Rust book example shows is how to implement what the Go language uses inside its run-time for running goroutines, which is a much harder thing to do, and very different, even though that's something that every good web-server uses somewhere in its stack.
Learning Rust by using `actix-web` to build a simple web-server is ok, but learning Rust (or any other programming language, really) by implementing `actix-web`, is not a very efficient way of learning the language.
The "crossbeam" library helps a lot: https://docs.rs/crossbeam/0.7.2/crossbeam/ It's the only way I know of to have thread closures with a less-than-static lifetime, which helps a ton with borrowing/moving and also general encapsulation: https://docs.rs/crossbeam/0.7.2/crossbeam/fn.scope.html.
Once you get the hang of it you can move forward pretty well, but the ergonomics around Rust concurrency, especially in the standard library, really need some work if they're going to continue focusing on it in their marketing materials.
In my experience, this gets ugly rather quick.
But yeah, I hope the pipeline operator comes soon, chaining is nice, but nesting can get quite messy.
This is one of the hardest parts about writing examples: you want something that's real, but also not long. Very difficult.
Worst thing ever. I was trying a gfx-rs example to do some drawing. One smallish function took another function as a parameter (a closure actually) so they just wrote an entire drawing function inline. The pipes helped clue me in that something strange was going on.
My own code pulled that function out into a clean definition eventually. As it should have been in example code.
It's not in the name of conciseness. Not sure what it is, but it's not that.
And what I realized was that, as fashionable as it is to say these days that imperative code is the devil, Rust's (deeply) const-by-default local variables remove one of the main reasons people have for nesting their expressions inline: avoiding the creation of state that has the chance to be mutated. And I also found that giving names to those intermediate steps can really help improve legibility. For me, Rust has made imperative code cool again, while also allowing for maximally expressions-based code (even more than JS in some ways!) where it makes sense.
I get annoyed when my C builds take more than a few seconds and a major reason i avoid C++ for my own stuff (C++'s complexity is another reason) and i hear that Rust has worse compile times than C++. This is enough for me to stay away from it.
But if Rust manages to get an implementation that feels at home at Windows with a full IDE and debugger and also gets around C-like compilation speeds, then i'd like to check it out.
We do support back a little bit, to help distros.
Isn't that possibly asking for a bit much? The Rust compiler is doing significantly more than a C compiler, so why would be expect it to have C-like speeds? Should we not also expect C to have compile speed improvements in that period, which might leave them relatively unchanged with respect to each other?
It feels sort of like you're comparing a Mercedes sedan and Land Rover. Both very capably vehicles, but targeted to excel in slightly different circumstances, and when people suggest a Land Rover for rougher terrain, you're noting that when they can provide the same acceleration and gas mileage as the sedan then you'll be interested, which is both unlikely to happen and missing the point, since an off-road vehicle can do things a sedan just can't. Preferring one over the other is fine, expecting one to be superior to the other in every way is a tall order.
So then lets pick D, Delphi, .NET Native, Eiffel, Ada as examples of toolchains from complex languages that compile faster than rustc is currently capable of.
There was a time when rustc could be called a relatively slow compiler for what it does, but not anymore. It's actually a reasonably fast compiler now. I expect small wins to continue in the future, but the only remaining foreseeable large win is Cranelift.
> but the only remaining foreseeable large win is Cranelift.
I'm not so sure about that, given the huge work going on to make rustc truly incremental.
I think the issue is more that not many people are using those languages in anger (except for Delphi I suppose, but Object Pascal is a comparatively simple language with no H-M inference that also does very little optimization). People don't hit the limitations.
So here go my couch coach ideas.
You already mentioned one possible improvement, I don't need to compile with all optimizations turned on on debug builds.
Secondly, combine JIT for development with AOT compilation for release deployment, e.g. Eiffel / .NET Native.
Thirdly, make use of a build cache across crates, including third party dependencies instead of building the whole world from scratch.
Combine that with an incremental compiler/linker, like Visual C++ does, where the team use Unreal, CryEngine, Windows, Office and other similar sized projects to measure build improvements.
This already exists and is easy to use: https://github.com/mozilla/sccache
As Steve mentioned, there is a global build cache already.
We have an incremental compiler. It's not as incremental as it could be, however.
We already have an extensive system, crater, that can measure build time improvements.
My point is not so much that the language is complex, but that the compiler does a lot more stuff along the lines of correctness checking. If you take a C or C++ compiler and add in the equivalent level of checks that Rust is doing, can we expect it to be faster than the Rust compiler?
Are any of the other examples you're putting forth doing things of the same complexity? I know many of them have their own ways of providing assurances, but I know some of them are not compile-time (or at least not more than inserting instructions for run-time checks). You would know better than me, since you've expressed previously that you have experience with many of those, at least at a cursory level. Can you comment on what they're doing at a compile time level that's approaching or surpassing the complexity of the borrow checking in Rust?
https://devblogs.microsoft.com/cppblog/improved-linker-funda...
https://devblogs.microsoft.com/cppblog/msvc-backend-updates-...
Another ways of improving compilation times:
D has fast compiler as reference implementation, leaving the hard work for gcc and llvm backends.
Eiffel uses a JIT for development, with AOT compilation via the C or C++ system compiler for production releases.
.NET Native, which now also supports F# (so H-N comes into play) takes advantage of MSIL, the development builds are a bit lightweight in optimizations and only the deployment via the store does the heavy crunching of optimizations, based on Visual C++ backend.
I do not expect it, i'd just want it to be fast for me to want to use it. I mean, i do not really care why it is slow (outside of curiosity, but not as an for accepting it), i care that it is slow.
One of the main reason we stopped using C++ is because Rust compiles much faster than clang. That was 3 years ago, and in this time frame Rust has improved compile-times significantly (e.g. reductions of ~30% per year).
Developers are users of their development tools.
> when building complex/efficient software expecting lightning builds isn't just unwarranted, it's selfish.
This is wrong. There is no rule that says that you cannot write fast applications with fast builds. It is just that we got used to C++'s slow build times.
This sort of thinking even falls flat on its face when you consider that C provides similar optimization opportunities as C++ (and in pretty much every modern compiler it uses the same backend and most - if not all - optimizations) while compiles much faster.
But beyond that most of the codebase of a decently sized application does not need more than the most basic of optimizations - it is only a tiny tiny fraction that needs that, yet with modern C/C++ (and most other) compilers, the entire codebase suffers for it (i think only Visual C++ allows you to control optimization settings on a per-function basis, but i haven't encountered any codebase that actually uses this outside of temporarily disabling optimizations for debugging and/or working around bugs).
There are only three (or is it four?) rules about when you can elide. Personally? I never write lifetimes, and then the compiler yells at me, and then I fix it by adding them. It's not something that you need to actively think about. You see
error[E0106]: missing lifetime specifier
--> src/lib.rs:2:8
|
2 | x: &i32,
| ^ expected lifetime parameter
and you add it in.It's the same with Rust ownership and lifetimes. C programmers know the concept of ownership, but don't have to be rigorous about it. They can use `char *` for both owned and borrowed stings, and make a mess of them (using `need_to_free_that_pointer: bool` as a crutch). When Rust has owned String and borrowed str, it's a surprise.
Sure, there's syntax to learn, and it has a few obscure bits if you're into generic programming. But the fighting with the borrow checker Rust is infamous for comes from using a C mindset of "will it crash or leak?" rather than "is this owning or borrowing?".
I teach Rust workshops and I find that novice users struggle with this, because they think of references as pointers, so they use them in places where they don't make sense (e.g. in owned structs or returned from constructors), and then desperately try every possible lifetime syntax to force through an impossible borrow.
In practice, once you "get" ownership, it's rare to use any lifetime syntax at all! In a codebase I have open right now (which is in production in 194 datecenters) only 3% of uses of references have a lifetime annotation, and it's just `<'a>` and `&'a`, nothing more complex.
Writing I'm not sure, reading it's just awful, everytime someone show his project on HN I check github and I'm baffled how the code looks bad as a human reader. (){}<>''-_
Yes, Rust is a very symbol rich language, but it also is very information dense which makes it easier to read than many other languages once you learn it.
APL-likes (and Forth-likes, and Lisps when you use the macro features) are a bit too far on the "power when fluent" axis, at the expense of learnability, and so the total area of the product is small. Might be good for one-man projects, but not for large ones.
Minimal languages like ASM or Java, where there are just a few primitives and everything else is design patterns, trade high "learnability" for low "power when fluent", and so also have a reduced optimization product.
Most languages aim somewhere in the middle, though often with a bias toward the side where they think the optimum might truly lie; for example, Go is slightly on the "learnable" side, and Rust is slightly on the "power when fluent" side, but both seem to be generally more productive per average-programmer-man-hour invested into FOSS projects than languages that more heavily favour just one axis.
(Side-note: I'd love to see some real numbers crunched on this. A comparison of rival FOSS projects to implement some shared standard in different languages would make for a pretty good "natural experiment" to look at productivity over. First thing that springs to mind for me personally are the rival Ethereum clients of Geth (= Go) and Parity (= Rust), but I'm sure you can think of your own examples.)
These are mostly orthogonal issues to how easy and/or powerful a language is.
By this standard, most programmers never achieve 100% fluency in even their favorite programming language, unless their favorite language is one of the dead-simple ones; and there are some programming languages (esolangs, certainly) that nobody is 100% fluent in; and many (the APLs, Forths, etc.) that only a few human beings on earth are 100% fluent in. (That's not to say that there aren't many people who can read and write them—just, not fluently. If you need to look at a programming-language reference more often than you need a dictionary to parse out the meaning of a sentence in your native spoken language, you're not fluent.)
A language that's less "readable", by your terming, is just one that takes longer to become fluent in. Given that the programmers working on any given project are going to be in a mixture of phases in their careers, and thus have invested different numbers of man-hours into language fluency, there's going to be a mixture of fluency levels on any project (where less-fluent readers experience the language as "less readable.") Longer time-to-fluency means that the same number of invested man-hours get you less fluency, and so on projects in languages with longer time-to-fluency, but the same distribution of programmer career-levels, you'll see lower average fluency (or a skew toward the bottom of the distribution, really) and thus more complaints of low readability. Mind you, this isn't really a fact about the language itself, but a fact about the project—if everyone on a project are 20-year veterans of the language, any language can be "readable."
I would note that we can measure time-to-fluency; it's objective. We can teach people a programming language, test them along the way, and see how many man-hours of study it takes to get 100% on the tests. We can't measure a language's "readability"; it's subjective. But it likely has a heavy correlation to time-to-fluency, so time-to-fluency is the measure you should care about if you care about A-B testing programming language syntax features for their effect on "readability."
C++ is design-patterns heavy and symbol heavy at the same time.
The heaviest design-pattern language champ would be JavaScript 1.5 with the lack of construct (assuming people work in super-huge JS project with hundreds of developers, not a simple web-app).
I think it looks awesome. Pattern matching, short keywords, types on the right... It accomplishes this beauty while also carrying lifetimes, pointer types, and traits. I personally like it much better then Java or similar static typed languages.
You can't compare it to a scripting language because it has type information.
What do you mean by this?
A statically typed language would be hard to reason about without types and it would dramatically impact compilation times.
Why would you include lifetime annotations in a GC'd scripting language? These are features for the compiler and the human to reason about. The language and compiler are intertwined.
While you could divorce the language from the compiler, it would be misguided to do so for many languages and problem domains.
You can hold up TypeScript as an example of a scripting language with types that upholds your philosophy, but its advantage is primarily for humans. I'm unaware of any performance benefits or optimizations it provides. It wasn't carefully spec'd out with a compiler in mind. It's a well thought out cross-compiled superset language. This isn't the case with languages designed for bare metal.
edit: I can't reply with a comment thread this nested. Compilers absolutely inform language design and vice-versa. Bad language design leads to horrible compilers (C++). A good language design process works in lockstep with its compiler design, with careful attention to the AST, the number of passes, analysis, etc. If we didn't think about compilers, we couldn't design languages that could be compiled.
Otherwise, you've conflated a bunch of things. Interepreted languages don't need to have a GC, or lifetime annotations, or anything else, really. Static typing doesn't require explicit type annotations, although it tends to have them, and neither is the domain of exclusively interpreted or compiled languages. And so on.
I also argue there's no meaningful distinction between "designed for bare metal" and "designed for application programming" other than a language that is designed to allow deployment on bare metal has a way for you to poke memory locations and at minimum turn off a GC. Everything above and beyond that is language features or libraries and by no means limits your ability to operate quickly and efficiently in either domain.
Like, to me, the human language of Vietnamese looks like a mess of accents, and I'm baffled by why they're necessary. But I'm not a fluent speaker of Vietnamese; and I can imagine that, for someone who is fluent in Vietnamese, and another sibling language like Chinese, maybe the extra accents actually compare very well and help readability compared to e.g. the reduced set of accents in Chinese pinyin.
Likewise, I would suspect that for someone fluent in both C++ and Rust (i.e. someone who knows how to write code that takes advantage of arcane language features in both), Rust might be more readable than C++ because of some of the initially-offputting syntax, whereas C++ might seem less readable because it eschews extra syntax in favor of using template constructors for everything.
So like you said - same thing with languages. You could probably substitute "Japanese text" or "Arabic text" into the original comment and see a little more clearly that fluency does matter.
Your mental model will affect how you feel about a language.
Once I finally took the plunge and committed a full project to learning TypeScript, I grokked it. I no-longer saw noise, I saw signal. I saw details about interfaces that I used to have to infer, document, or reverse engineer if I forgot.
I had (still have!) trouble reading JS, because the vast majority of things I've worked on have been in statically typed languages. JS was baffling because I always have to try to infer/reverse engineer things that I can just read off in the languages I'm most familiar with.
The only way I really could handle dynamic stuff was for things like scripts, where I could mostly remember the entire set of types.
I've gotten better with JS after using some TS, but that difficulty still exists. Correspondingly, I'm delighted by Rust - sure, it's a verbose language, but that puts everything I want to know right there for me to read.
In other words, there's beauty, and there's acquired taste and it's legitimate to point out this difference.
Most humans frankly don't understand what's needed to capture complex mathematical concepts, so that's not at all surprising. The same argument can be applied to your objection to syntax, and this merely reiterates the OP's point: until you're fluent in a particular language, you have no idea whether the syntax is necessitated by some complex but necessary semantics that you simply don't know about.
I think this is more a problem with what you're used to. You see the same criticisms of every language: Basic dialects are too wordy, Lisp dialects too bracketty, C dialects too symbolly, etc. etc. It's hardly the measure of a language.
Java was the language I learned during CS101 in school and I always hated the “don’t worry about this part right now just write static main void... or the equivalent”. Python code is cleaner too look at but Java has it’s reasons for the verbosity.
The braces allow you to be explicit with what you want. Rust's {} are a feature. They let you see exactly which scope you're in. With Rust's memory management, you then know exactly when a variable will go out of scope and be cleaned up.
You peaked my interest. Guido van Rossum on the subject:
> But the complexity of any proposed solution for this puzzle is immense, to me: it requires the parser (or more precisely, the lexer) to be able to switch back and forth between indent-sensitive and indent-insensitive modes, keeping a stack of previous modes and indentation level. Technically that can all be solved (there's already a stack of indentation levels that could be generalized). But none of that takes away my gut feeling that it is all an elaborate Rube Goldberg contraption.
https://www.artima.com/weblogs/viewpost.jsp?thread=147358
As a side note, there are more than 100x the number of jobs with Python in the description vs rust. Out of the 30 or so jobs with Rust in the description most are for automotive techs.
Rust 1.0 was released May 2015 [1]
Python 1.0 was released January 1994 [2]
That's over 21 years that Python has had a chance to work its way into the industry and into almost every field imaginable. I'd wager that many careers have revolved around the Python language at this point.
Rust is just getting started. The volume of press is the first sign that it's about to take off. It can take years for existing enterprises to approve a new language for use. Give it time.
[1] https://blog.rust-lang.org/2015/05/15/Rust-1.0.html
[2] https://en.wikipedia.org/wiki/History_of_Python#Version_1
like Haskell ?
Objective-C in particular was verbose, but the symbols made sense and their meaning could be understood from the context. In Rust you either know what the symbols mean, or you do not understand the code.
This is just part of knowing a programming language. Readability to people who don't know a programming language might be considered an interesting property in itself, perhaps especially for cognitive psychology or education, but it has little to do with programming. It's only accidentally related to a concept of 'readability' relevant to software development.
Python on the other hand is awesome, and I also begun looking at Go which seem to be quite okay.
The only exception is the method::<Type>() that's sometimes necessary.
When I write or read rust code the symbols just sort of disappear.
Nowadays every time I have to go back to C++ and read `method<Type>()` I need to parse whether this is `(method < Type) > ())` or something else. Luckily for me, C++ introduced the `method.template <Type>()` syntax... but over that I prefer `::<Type>`.
I also prefer ::<Type> over Scala and D's syntaxes for some reason. I find Scala syntax for this more complex, and I find D syntax too simple.
I find Rust readability a mixed picture. Being a Rust novice I sometimes find the details hard to follow precisely (as in: I often wouldn't be able to reproduce the code without error myself). But the type system, stdlib & generics make it sufficiently expressive that it's usually pretty easy to understand broadly how things work. In my journey so far I've been able to follow large library codebases more easily than in other languages I've used.
The "before" example is C, or bad C++. You are allowed to write bad C++, as you are allowed to write bad Rust. But if you are to compare langues honestly, you should be comparing good code in one language to good code in the other.
The alternative is tendentious but typical, and nothing to be proud of.
Basically, even without the STL, the students would get our department C++ library with the usual stuff (strings, arrays, vectors, bounds checking enabled), and the teaching would be similar to the way that Kate Gregory preaches how to teach modern C++.
Rust seems anecdotally better at onboarding new engineers than C/++. We're going to start seeing more articles like this.
Rust has many great qualities and I've been looking to find a project for it, but it just doesn't seem like the model for long-running processes with immutable data-structures and complex/dynamic/interlinked lifetimes. Seems like dev. effort is better spent on functionality than managing lifetimes, especially given that modern GC is so fast. (And I've seen C++ ports to C# become faster, due to unneeded calling of copy c'tors, and other goodness.)
I can totally see the case for Rust in systems code where lifetime is gated by a system/API call and you want to safely pass buffers around during the duration of the call, for inner-loop gaming, for embedded, etc. But seeing it's en vogue, there are many examples of Rust code where it doesn't seem like the right tool -- but then again, I feel that way about much code written in C++ as well.
These are not mutually exclusive.
(1) A GC'd language can still do static analysis at compile-time or JIT-time to avoid GC all-together when possible (2) GC'd languages can have RAII/static-scoping capabilities such as immutable value-typed (non-ref) structs which require initialization and using{}/after blocks for scoped lifetime (3) GC'd languages can have owned-object semantics like C#'s Span<> to avoid GC (4) You can use unsafe and pointers and GC'd languages a well
So you can still program with an eye toward efficiency for any code that shows perf issues using many of the same constructs you do in Rust, in a GC'd language -- but be able to fall back on GC when lifetime issues are complex or hard real-time perf is not an issue.
Again, I'm not bad-mouthing Rust at all -- I'd far prefer it to C++ in the context where the latter is appropriate.
Rust can express ownership and lifetime of objects statically at compile time, as opposed to dynamically at run time. For beginners this is a new aspect of programming they need to learn, and it adds to the learning curve (same as you need to think about types in C++, instead of winging it in JavaScript), but it has very nice side benefits:
• Deterministic destruction. You don't need to remember about closing handles/cleanups, and all things are freed immediately when their scope ends (rather than "finalized" sometime later). Some languages have `defer` or `withFoo(callback)` to handle these. In Rust it's automatic and more flexible.
• Ownership adds useful information to the program. You know which args will be kept or shared, so you don't get shared mutable state by accident. Libraries don't have to copy their args just in case the caller changes them unexpectedly later. And as a caller you know what each function does to the data you pass in.
The last point is especially useful for passing buffers around, because you control at all times when you give exclusive or shared access, and how long each part of the program can keep them.
This is a completely bogus comparison.
First, static typing is mostly about finding bugs. GC-induced overhead is not a _bug_.
Second, it is theoretically impossible to create an algorithm that checks almost any interesting property of a program in a Turing-complete language in finite time, that doesn't reject valid programs. In most (non-academic) programming languages with static typing compiler checks assumptions about the _data_, and doesn't wade in control flow at all. Data is more or less "static" by itself (types of variables remain the same in most algorithms, with exceptions covered by ADTs), that's why static typing is practically useful.
Lifetime control is a problem on a whole different level. Roughly speaking, lifetime of an object _easily_ depends on whether or not some part of the program terminates, so, from the get-go, your static checker has to solve halting problem, which it can't.
I went through several Rust projects to see how they deal with this. And there are two solutions: 1) sprinkling code with unsafe's 2) resorting to reference counting. GC is safer than both alternatives, and its amortized cost may be less than same for ARC.
Absolute majority of the code written doesn't have hard real-time requirements, and should just rely GC as a safer and more convenient option.
Yes.
Most Rust code deals with "trivial" lifetime issues -- not trivial in the the sense of how it gets used (usage can still be in a "serious" context, such as systems programming) but how the lifetime is controlled. Only certain models lend themselves to be expressed as "borrowing from an owner", and ref-counting will break with general object graphs.
Most garbage is gen-0, which gets recycled with little overhead, and when scavenging is needed its fairly quick and tunable these days and can optionally take advantage of multiple cores.
Another "trick" I see in Rust code is to use a symbol-table of sorts to represent object graphs using names and lookups, but you're just inventing your own, slower version of object references, and then there's the issue of who owns the graph symbol table, so you're only passing the buck in the general case.
For a long time languages were stuck with the false dichotomy of either trying to be 100% safe (and pay the cost of a VM/sandbox/GC) or said "halting problem" and gave up. Rust solved this dilemma by allowing safe abstractions around unsafe blocks. It works like a charm — I get speed I got from C, but don't have to use gdb.
There's option 3) use program architecture that has clearer ownership and less shared mutable state. Borrow checker doesn't like doubly linked lists or mutual parent-child relationships. Instead you use different containers and DAG structures.
And when you still have to have shared ownership without a clear scope, then you use refcounting. The nice thing in Rust is that (unlike C++/ObjC/Swift) it doesn't always have to be atomic in multi-threaded programs, and objects can be borrowed and used without touching the refcount.
Modula-3, Mesa/Cedar, Component Pascal, System C#, ...
You get the productivity of automatic memory management, RAII, stack and global memory allocation and unsafe blocks just like Rust, with C's speed as well.
It's also possible that you're reading a version of the book before the ..= syntax was introduced; it hasn't been there since 1.0.
Now if only the images with code actually had the snippets in code form I would be able to share the blog post to other people.