John Carmack: writing Rust code feels wholesome | Hacker News Reader

John Carmack: writing Rust code feels wholesome

twitter.com

211 pointsjimyl81 comments

Man reading Twitter is so awful. It feels like sitting in one of those 2 hour meetings where everyone is allowed to give their 30 seconds and all that's left is your throbbing headache.

I looked seriously at rust about 2 years ago. I seem to have tried the language at the wrong time .. they were transitioning between versions and this made learning it hard. I grew up with C so am very comfy with pointers. Even reference counting feels natural to me. That said, the borrowing/ownership semantics of rust (at the time I looked at it) felt needlessly over complicated. Has this got better? Is there a K&R equivalent that an experienced programmer can get pick up to get productive with the language soon?

>That said, the borrowing/ownership semantics of rust (at the time I looked at it) felt needlessly over complicated.

Isn't it just “one mutator at a time”?

>Is there a K&R equivalent that an experienced programmer can get pick up to get productive with the language soon?

(Opinion incoming.) The Rust Book[1] is great iff you are a complete beginner. If you are an experienced programmer, who knows his way around concepts like resource management and ownership, it's kind of a drag. Rust by Example[2] is much better. The Nomicon[3] is interesting, but it's mostly about unsafe.

[1] https://doc.rust-lang.org/book/

[2] https://doc.rust-lang.org/stable/rust-by-example/

[3] https://doc.rust-lang.org/nomicon/index.html

>the borrowing/ownership semantics of rust (at the time I looked at it) felt needlessly over complicated.

Taking a wild guess here, but it sounds like you maybe referring to issues addressed by Non-Lexical Lifetimes (NLL)?

Reference: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll....

NLL are a part of Rust 2018 Edition.

Three complementary books are the best resources:

- The Official book: https://doc.rust-lang.org/book/

- The Rust By Example book : https://doc.rust-lang.org/rust-by-example/

- Programming Rust (O'Reilly) : https://www.amazon.com/Programming-Rust-Fast-Systems-Develop...

NLL only modes aren't available on stable Rust atm. What edition 2018 has right now is migrate mode NLL. In order to fully enjoy NLL, you'll have to use Rust nightly and do #![feature(nll)] or -Z borrowck=mir -Z two-phase-borrows, both flags being independent of the edition you're using.

Edit: removed outdated behaviour of migration mode.

> What edition 2018 has right now is migrate mode NLL which emits an error if either NLL borrowck or AST borrowck think there is an issue with your code.

If that were true, then this example would fail to compile, since it fails with the old AST borrowck: https://play.rust-lang.org/?version=stable&mode=debug&editio...

See: https://rust-lang-nursery.github.io/edition-guide/rust-2018/...

Oh right, seems it got a bit more tolerant. The mode's still called migration mode though, full NLL is to be unleashed later on: https://github.com/rust-lang/rust/issues/57895

This is incorrect. NLL is on for Rust 2018 code.

The transition mode is for Rust 2015 code.

Please see the code: https://github.com/rust-lang/rust/blob/68650cacf23c60ee2f346...

In edition 2018, migration mode is turned on.

I learned myself about this today btw, after I saw this PR: https://github.com/rust-lang/rust/pull/58342

I think I was mistaken about one point though which is that migration mode is erroring if one of the two errors. IIRC migration mode was introduced that way but it seems to have been changed to be a bit more tolerant than AST borrowck.

Anyways, more NLL fun is to come, and it's to come to both editions.

The only difference between migration mode and NLL-only mode is that errors produced by the NLL checker are downgraded to warnings if the old borrow checker doesn't produce any errors on that code[1]. This is strictly more permissive than NLL-only and completely equivalent if you treat warnings as errors.

You can completely take advantage of any NLL feature, you just might be able to write code that should be an error but is instead a warning if you hit an edge case (I think this would be considered a bug in one of the borrow checkers since NLL is supposed to allow anything that the old borrow checker allows).

[1] https://github.com/rust-lang/rust/issues/46908

NLL still doesn't cover all use cases, for example common GUI patterns with callbacks having references to internal fields.

I've been programming in C and C++ since sometime around 1992 (and Pascal before that). For the last 3 years, I've been writing Rust professionally.

Here's my Rust advice for C and C++ programmers:

- Rust is a higher-level language than C. A C/C++ programmer could think of Rust as "C++ without most of the footguns and magic."

- Certain types of C/C++ code will translate very easily to Rust. For example, if your code mostly transforms bytes (or data structures) into other bytes (or data structures), then Rust will usually be fairly easy to learn. We could call this "mostly functional" C code, with only localized mutability, and with clear, hierarchical data structures. Examples: Most typical Unix CLI tools.

- Certain kinds of C and C++ will translate very badly to Rust. If you have a big web of mutable objects, all of which point to each other and update each other, then your first experience of Rust will probably be frustrating. Examples: doubly-linked lists, traditional GUIs, typical video games. You can do all these things in Rust, but you'll need to either re-architect them or use more advanced features. (See http://cglab.ca/~abeinges/blah/too-many-lists/book/ for doubly-linked lists, or https://kyren.github.io/2018/09/14/rustconf-talk.html for video games.)

- Async Rust is not ready for prime-time. We make limited use of async Rust at work. It's rock-solid and powerful, but it's missing critical ergonomic features and it demands a surprisingly high level of Rust knowledge. See https://areweasyncyet.rs/.

Two good Rust books are https://doc.rust-lang.org/stable/book/ and http://shop.oreilly.com/product/0636920040385.do. "The Rust Programming Language" seems to work best for people coming from Ruby or JavaScript, but I've known several C/C++ programmers who preferred the O'Reilly book.

Honestly, if you're happy with C, there's no reason to switch to Rust. (Except, maybe, preventing exploits, if you worry about that.) If, however, you love certain parts of C++ and hate other parts of C++, then Rust might be worth a look. Anyway, that's my personal take. :-) Overall, I've been really enjoying Rust. It generates fast, reliable code and it fits the way I think.

What are you building with Rust if you don’t mind sharing? I’m curious to know what sorts of companies are using it out there in production.

Here are the major categories of stuff we've built in Rust at work:

1. Lots of little CLI tools, statically linked using https://github.com/emk/rust-musl-builder. These are fast, portable, easy to share, and easy to add to Docker containers. For a fun open source example, see https://github.com/faradayio/geochunk, which groups US zip codes into evenly-sized chunks.

2. Code where we need to handle all the corner cases very carefully. Rust's enum and match are wonderful for this. An older open source example would be https://github.com/faradayio/cage, which performed all sorts of fiddly transformations on docker-compose.yml files. (I need to massively overhaul this to work with https://kustomize.io/ someday.)

3. Code which moves around large amounts of data, and maybe processes some of it. This might involve REST APIs, AMQP queues, cloud buckets or databases. Sometimes we wind up writing a little extra glue code, but we're pretty happy overall.

This morning, I'm working on an unreleased open source command-line tool that moves large database tables between different local and cloud databases. This actually uses async Rust (specifically https://tokio.rs/blog/2018-08-async-await/), which is definitely bleeding edge. But this particular code benefits enormously from Tokio, so we're taking the plunge.

Overall, our Rust code is fast and reliable. I enjoy working on it, and my boss likes the fact that our Rust code tends to be very solid.

Sentry is using it to help their Python backend to process crash reports:

https://github.com/getsentry/symbolic

In my previous job I wrote a push notification service[0] and a high traffic web server[1] with Rust. Both were rock solid and fast.

Now at https://www.prisma.io/ we're converting our Scala codebase one module at a time to Rust, plugging them to the Scala codebase with JNA until all the code is converted. Plan is to have better portability between different languages and smaller resource usage. With Rust, interfacing other languages such as Java/Scala, Javascript and so on is quite ergonomic.

I also saw Kraken cryptocurrency exchange looking for Rust developers to help with transition, so it seems there is some work opportunities already in the field.

[0] https://github.com/xray-tech/xorc-notifications

[1] https://github.com/xray-tech/xorc-gateway

I'll probably hold off on learning Rust deeply until they have async/await as keywords. Most of the programming I do involves async, and I currently do my day programming in Dart, which is an absolute joy to do async programming in. They had Future-based APIs from day one, so the whole ecosystem was built around futures starting out. They've had async/await as keywords for over 5 years now. It's so ergonomic it's painful to go back.

Interesting, what kind of job requires you to program in Dart?

Job didn't "require" it, I was the tech lead of a small startup, and chose Dart for client (WebGL-based game), server side, and supporting command-line tools. It's worked out great and the team loves Dart.

the borrow checker will accept more correct programs than it did 2 years ago so yes it has gotten better, but it is still a big paradigm shift.

Rust's ownership/borrowing model is very close to the (often implicit) model of proper pointer handling in a large C codebase. One medium-sized codebase I've worked with tried to be very disciplined about const pointers as a means of passing shared reference vs. ownership, with a style guide about when to use const that strongly preferred const wherever possible. I tried translating the style guide to Rust and found that's it's basically automatic once you add Rust's reference model.

Of course the big distinction is that Rust mandates it. There's lots of C code out there that happens to work fine with shared data structures that never collide in practice (or at least never collide under non-hostile input...) - see for instance every standard libc API that returns a pointer to a static buffer. Our codebase merely had a style guide, and lots of older code was less disciplined. Rust forces you to care up front about everything.

A few weeks ago I tried to implement a lisp in Rust. Things were going fine until I decided that instead of parsing a string, I'd like to parse some kind of input stream, like one would have in python with a "with open()"ing a file, or in C with a file pointer. The guys on #rust were very helpful with figuring that out (using a mutable iterator). Only after implementing did I find out that Rust dosen't seem to have support for reading files as iterators, and I'd have to write that myself. Okay, not so much of a problem.

Then came what I thought should be a simple task; since my code was mostly recursive, I wanted to add and remove from a map containing the program's current binding state, so when the evaluator sees a 'let' block:

    (let ((a 1) (b 2))
       (+ a b))

for instance, it evaluates the addition with a Map like: {'a':1, 'b':2}, and once that's finished evaluating, the evaluator returns from that scope and we're back to having nothing bound. After all, I don't want to access 'a' and 'b' outside my let.

As it turns out, Rust doesn't support using maps (or hash maps) in this way. You need to allocate and borrow them, and you can't borrow them in multiple places, so I couldn't have a function that checks whether something is bound, since I'm already borrowing the map in my evaluator function.

I was left asking: why can I do this in C (or another 'low level' language, passing the bindings as a pointer to array of a k-v struct that I simply swap with another and remember to free()) and do it in Python (or another 'high level' language like Haskell, creating the map anonymously when calling eval recursively), but not in Rust? The language seems to be somewhat focused on ideas of immutability, but I can't do the things that immutability lets me.

Other things seemed to get in the way too. Once you've 'match'ed a variable you can't actually deal with the thing you matched, because you've already borrowed it when you did the matching. In Haskell (case statement) this is no problem at all:

    is_cons e =
      case (car e) of
        Nothing -> False
        Just _ -> case (cdr e) of -- still matching on e
          Nothing -> False
          Just d -> ...

Maybe I had the wrong use case for Rust, or I was too stupid to figure things out. Either way, I'm a little happier having decided to literally learn Haskell instead, which turned out to be easier than dealing with Rust, a language which is supposed to be more like languages I was already familiar with (C/C++/imperative).

Rust very much supports reading files as iterators, there are even multiple ways to iterate. By line, by byte...

Your HashMap issues may have been solved by the Entry API.

Rust is better suited for use cases where having a GC around is a no-go like OS drivers doing DMA like stuff and high integrity systems, for anything else it is more productive to enjoy having a GC around.

No, it is still bad.

Rust is not for good c/c++ programmer. It is for those who is not familiar with c/c++ but have some knowledge with functional programming.

As a C++ programmer, you can try golang instead of for productive

Many comments in twitter mentions rust still not being ready for server code ( which i start to feel strange since concurrency is the only major issue on the server side, and supposed to be rust strength).

Anyone can confirm ?

Perhaps they meant webapps:

http://www.arewewebyet.org/

Latest news: 11 Aug 2017

Wargroove, a recent game by Chucklefish, has its matchmaking server in Rust.

Many people are waiting for async/await before saying “please come try this”, as it makes the enterprise far easier. We’re quite close!

I have used rust but I think it has another year before it really shines. 2020 is the year of Rust - async/await, tool ecosystem, various crate stability, etc is happening fast.

While you'll hear Rust community often say "just wait until this new hotness", I'd like to clarify it's not like a perpetual "it will be the year of Linux Desktop".

Rust has already shipped everything it needed to be production ready. We're just always excited for the next big thing. Two years ago it was incremental compilation, last year it was a new borrow checker, this year it's async/await, etc.

This is also true. It really is about what you’re doing. Async/await in particular enables you to write idiomatic code, right now you have to twist it a bit and can’t do certain things. Other features? I agree with you.

There's a new async/await syntax coming, which will make working with network-heavy code much more pleasant.

Currently, it's ready in the sense you can write robust, high-performance network code. It's still easier than in C or C++, but it's not noob-friendly compared to Node.js or Go yet. Code based on Futures uses owned closures a lot, so you need to "get" ownership in order not to fight the compiler.

With the async/await syntax, the compiler will be able to relax some of the memory ownership restrictions, and you'll be able to write "normal" code instead of chains of callbacks.

Concurrency, as in multiple OS threads communicating is nicely done, with the borrow checker being able prevent some fairly common multi-thread pitfalls. So for things like data parallelism etc. Rust is already looking pretty good.

For server-side stuff, what's not yet done is some infrastructure and improved ergonomics for writing asynchronous or non-blocking code. See https://areweasyncyet.rs/ . And then when the infrastructure is there, it'll take a while before the higher-level frameworks take advantage of it.

Microsoft is using actix libraries and creating many new rust libraries for their Azure IOT platform. Azure IOT is all about safety at very high speeds, making Rust a great fit for their work.

I don't work there.

Multithreading and multiprocessing work fine, however they may be referring to m:n (aka "async"), which is a story still being written and which the borrow checker currently can make much more frustrating than expected by other languages with m:n threading support (e.g. you can't just close over a lexical variable then use it in multiple steps of an asychronous processing pipeline).

This is because of the removal of the fat runtime and green threads about 5 years ago. Since there was no blessed way to do high concurrency in the language any more, some code started using normal threads, some code (tokio) started using normal callbacks. Callbacks in rust are tricky because of the type system. Around a year ago, async/await became possible due to the development of the Pin idea. async with the await! macro is usable now, but the rust community has been bikeshedding over built-in await syntax and last time I looked that discussion was still raging.

So you can write scalable servers using normal threads, normal callbacks, and async with the await! macro, but not the final async/await syntax because the community hasn’t even finished deciding what it is. Porting code between these approaches is also not particularily easy. I have seen example code for rust networking libraries that uses all of the different approaches, but mixing the different approaches is also difficult. So that’s probably why people say it’s not ready even though you can write code right now that will work just fine. The “final” way of writing it isn’t ready yet, and if you write one of the current forms, porting to the final form will be a pain.

Porting from futures to async shouldn’t be too hard based on my experience doing it in JS.

Sure, but js doesn’t have rust’s type system. It will probably be straightforward once everything is settled down, but right now Future 0.1 (tokio) and Future 0.3 (async/await) aren’t comptatible and you’ll find examples that use one or the other, or plain threads, or plain callbacks. It’s possible to get up and running quickly with the examples from various libraries, but when you try to mix them, if they use different styles, it becomes very challenging. Once async/await is finalized and everything moves to it this won’t be a problem, but that’s probably going to take at least a year or possibly years.

I did a test and converted one of my libraries to async/await and it was not that hard. There are easy converters between the future versions for the transition period.

https://github.com/pimeys/a2/pull/30/files

https://www.ageofascent.com/2019/02/04/asp-net-core-saturati...

That actix you see appearing in most lists and mostly nearly top is written in Rust btw.

Fun fact- frameworks managed by microsoft dominate the space ;)

IIRC actix was gettin some flack for using very unsafe code to hit those benchmarks. Still cool.

There is absolutely no shame in using unsafe!

The issue was that the author uses unsafe code unsafely and often for questionable performance or ergonomics gains that didn't actually need unsafe anyway. Fortunately, he cleaned up pretty much all of it, as I understand it, with the help of some involved community members when people freaked out about it all.

It's pretty nice for server code, but still has one major shortcoming which is that's it's kind of tricky to write high-performance IO-heavy code (it's easy if you want to have a thread blocked on every IO request, but that's bad for performance if you have many concurrent requests).

This will improve a lot once async/await stabilizes. It already works great in nightly if all of the libraries you want to use support it.

When I explored rust I found the exact same feeling too, but couldn't find that word.

My only issue is how surprisingly weak Rust support is in my favourite editor. Rust seems to feel more popular than it is. Or maybe more people write Rust in a "no frills notepad" kind of way.

vscode has pretty good support for rust.

If you need something more advanced, you can try clion from jetbrains with the intellij-rust plug-in.

Vscode is what I tried. It was critically flawed where the autocomplete would crash or be missing half the options you'd expect.

I will need to try again. It's been six months.

I can vouch for CLion with intellij-rust, it is by far the best Rust editor/IDE available at the moment. RLS still suffers from slow/no auto completion and crashes too much for my liking. The pace and features that are coming out of the team working on intellij-rust is amazing: https://intellij-rust.github.io/thisweek/.

Rust is so exciting to me that I'll probably do this. But I absolutely HATE having to maintain multiple editors. It's like playing two pianos where the keys are all in different places. I will spend hours trying to make one like the other but it'll just never quite be right so my coding rhythm. will be discordant and janky.

I wish there was a trivial way to say to intellij, "steal all my config from vscode"

Same experience here. Was learning rust with vscode, and it would only show half of the compiler error output (e.g. the part that recommends how to fix the error wasn't shown).

In the latest vscode release they now support multiline error messages so maybe the situation will improve.

As a fan and user of both Rust and VS Code, it's still pretty bad. It crashes less than it used to six months ago, but the autocompletion is still near useless. It doesn't support traits as far as I can tell, for example.

As an Emacs nerd I did have to find and download a Rust mode. I imagine in the future Rust will get more support in editors.

I've also just started writing Rust in the domain in which I write code, low level networking. I think I agree with Carmack in that it feels good. I can get hungup on syntax when learning a new language and I don't have that problem so much in Rust. I've been writing code in many different languages for close to 30 years and learning Rust feels mostly right. It feels like when I learned Python or LISP and I could guess at syntax and just get it right. As opposed to writing /bin/sh for over twenty years and still needing to constantly look up syntax :(

It doesn't feel like some languages I've learned in the past where everytime I investigate something I face-palm. It seems like the more I investigate it the right choices were made. Even when I'm frustrated with it because the compiler won't let me do something simple, I understand that maybe the bad habit I learned in C should end. Rust forces you to deal with things upfront that may become problems later. It's sometimes annoying when you want to just get shit done, but it's the 'right thing' to do.

It's so nice hearing this from someone with as much experience as you. I have far less but one feeling I had lately when switching to typescript was, "instead of turning off this warning, maybe they're right. Maybe this is a bad habit."

It forced me to think harder about my code's structure. I think that's what I like about Rust too. Not being able to do anything means I have to be thoughtful.

Which also reminds me of how to be creative with games or writing or music: artificially limit yourself and you're forced to be more thoughtful.

CLion/IntelliJ with the Rust plugin works nicely, but I write 99% of rust in (neo)vim. I hate opening up IDEs and keeping them open.

emacs with elpa-rust-mode (ubuntu package) works well for me.

Also posted in the twitter thread, a game dev wish-list for Rust

https://users.rust-lang.org/t/my-gamedever-wishlist-for-rust...

Does anyone know, is https://github.com/rust-lang/rfcs/blob/master/text/0066-bett... ever going to be resolved? I really do not like the idea of having to create a new variable for something so simple as this.

Seems to be unanimously agreed to be a good idea, but is a bit tricky to actually implement[1]. Hopefully some day.

[1] https://github.com/rust-lang/rust/issues/15023

JC: hey this language is nice

Twitter: yes but have you tried my favorite language?!

I've been playing with Tokio and Actix for a few days. That's the same feeling I get.

I decided to learn Go instead of Rust over the last 6 months (very limited time so couldn’t learn both well).

Rust has gotten so much on HN lately that I think it may be time for me to make the switch.

Go is okay too. It's not as hyped as Rust but it has a lot going for it. So don't regret your choice.

I started with Go and am now learning Rust. I liked Go, but it felt incomplete (currently there's a discussion on generics) and I'm tired of Java never settling down and now trying to be cool. Also, method declarations are weird (method over variable or pointer).

Rust seems complete. The ownership paradigm is a bit difficult at first, but the compiler is good at pointing out exactly what the issue is.

Interesting remark of Java not settling down with Rust being complete, while some Rust libraries depend on nightly and async/await is still not fully baked.

Languages are products, either they change with times or they die.

C++ continues to lose mind-share. When will the community acknowledge that their tooling is lacking?

A tool is a tool. It's immaterial if a particular tool like C++ and its ecosystem is no longer the best available tool for some applications. Workmen pick the tool that best suits their tasks, and if a better tool appears it makes no sense to assume that the old tool requires more attention.

C++ has many users that love the language but are in denial that other languages have advanced beyond it (not in all areas, of course, but enough areas that matter).

Do you use C++? I do, and I don't love the language. What I like about C++ is that it's a high level low level language. Anything that isn't is not a competitor in C++'s core application areas. D and OCaml come kinda close but they have (in practice) mandatory garbage collection. The only real alternatives are Ada (which somehow never achieved much mindshare) and Rust.

I do, as native companion to Java and .NET, and I still kind of like it.

However for my own use cases, my actual alternative is the newly founded love for low level programming support by Java and .NET designers. Mandatory GC is nice to have around, while only specific hotspots require a more machine friendly coding.

> Workmen pick the tool that best suits their tasks

Workmen also carry the burden of maintaining old (legacy) products.

No one would say no to a shiny new feature that makes tedious stuff easier.

> A tool is a tool.

Oh yes it is. Which means, a tool is an extension of oneself. We tend to identify ourselves with our tools, especially if they required extensive learning. The simplest tools (hammer) are easily replaced, but the more complex ones (my cello) are highly personal, and seldom swapped.

Programming languages fall on the complex end of the spectrum. They require learning, and to some extent, what you learn is what you are. Throwing away a programming language (and its ecosystem) is throwing away a good chunk of the knowledge you acquired there —a part of yourself. A new language needs to have one hell of an advantage to get significant adoption under those conditions.

And I didn't even talk about network effects…

Tools are generally independent. Using a particular screwdriver doesn't change the effectiveness of your hammer.

Changing between programming languages is a bit more complicated than exchanging tools in a belt.

Where? On HN and Reddit? I see new code bases in C++ springing up like mushrooms. I'd prefer Haskell/ML, but I can't deny reality.

It seems the market for people who can convert systems from other languages to Rust is starting to appear. At least in the last couple of months there has been quite a few offers in Berlin for people who can write and teach Rust.

If it does, than not to Rust. Comparing tooling support for c++ and rust... In that comparison c++ completely overwhelms rust

I like C++ but I think he meant in terms of how sane and usable the tools are, in which case Rust easily wins. Compare Cargo to CMake for example.

Sure it might not have as many IDEs, static analysers, debuggers and so on, but I'd easily take Rust's build system over C++'s tool ecosystem.

When we compare eco-systems, we need to consider everything.

For cargo to win over cmake, it needs to finally support binary libraries.

For Rust to matter to .NET devs, it needs to support COM/UWP and mixed mode debugging on IDEs.

On Android, mixed mode debugging and just fitting in with the NDK/AS would also be quite productive.

C++ is actually doing very well, and the community is very well aware that C++ tooling sucks. It's just not an easy problem to solve because it has such a huge amount of existing code and projects.

Absolutely!

I think the important question is, what will happen first:

(a) C++ gets better tooling around build systems and package management; or

(b) Rust libraries are written for all of the things that we care about?

Certainly in the HN bubble it feels like (b).

It's not just package management. C++ is hugely overcomplicated with features. Understanding C++ is a life project. And the language is unsafe-by-default so you will spend time tracking down segfaults or other UB stuff. Rust allows me to focus on the problem domain, less on the language.

Reply on news.ycombinator.com