Hacker News Reader

Comment by gnulinux | Hacker News Reader

I want to note here that throughout the internet and HN there is this ongoing wisdom that although Rust is very useful, valuable etc writing code in it is very hard because it constantly feels like you're wrestling the compiler.

Well, maybe there is some truth to it, but I want to say that having started learning Rust in 2022, I found it's multi-threadedness model very refreshing. I thought it was such a delight to write code in, and personally loved it at least one order of magnitude than Go's goroutines. It's safe, embedded in type system (Send etc), has tons of standard tools (Arc, Mutex, mspci channel etc...) that are hard to shoot your foot with.

When people talk about Rust's complicated nature, I wish more emphasis is made on how it completely eliminates entire categories of bugs even within a multi-threaded context. As a corrolary, it really makes some parts of programming hard in other languages, nearly trivial in Rust.

View on HN

Concurrency is hard. Before I started using Rust I was totally on the immutability train, and I still think it's a great strategy. But immutability everywhere also involves jumping through hoops and fighting with the compiler. Though for some reason no one talks about that strategy the same way they talk about Rust's. I think that, no matter what strategy you settle on, it's a bit more work to guarantee correct concurrency.

> Concurrency is hard.

Absolutely true. This is the submerged part of the iceberg for any programming language. Whenever I see a new language, I skip the hello worlds and the syntax sugar and go straight to see how they solved concurrency and parallelism. Chances are you're gonna need concurrency. If you do, it'll likely be the largest source of complexity in your program at large. Sound concurrency is paramount, and a huge challenge for PL designers.

Rust has solved some major problems and deserves recognition for that. It is already influencing new languages, and we will be better off in the long term. That said, in order to achieve their extremely ambitious goals, they needed to push the type system to its absolute limits, and in a painfully user-facing way. When you are writing async Rust today, you're spending a large part of your limited attention/complexity budget trying to satisfy the compiler, even when your mental model of the business logic is sound. This is what people mean when they say async Rust is hard.

On the flip side, having used C++, python+javascript (ahhhh) and Rust concurrency, I found the assurance and debuggabilty the type system gives you very relaxing. I can simply break down the problem into "model business logic" and "make the compiler happy" and if I have solved both problems without escape hatches it's not easy to mess up concurrency in Rust (in my opinion). And the ecosystem might be complex, but it's not unnecessarily complicated: basically everything is there for a reason and allow you to hack as much as possible while staying in the promises that rust gives you.

> having used C++, python+javascript (ahhhh) and Rust concurrency, I found the assurance and debuggabilty the type system gives you very relaxing

Not to say anything about C++, python and pre-async JS, but if you compare with async in modern languages and especially if you compare with Go, async Rust leaves a lot to desire imo, mostly in terms of ergonomics and total system complexity.

> but it's not unnecessarily complicated: basically everything is there for a reason

Yes and no. Let me be careful with my words here. Everything is there for a reason, which is always locally coherent. However, globally speaking, these reasons can get more and more detached from the end goal, including the core principles of Rust such as zero-cost and fearless concurrency. I believe this is exactly what happened to async Rust.

A simple example would be lack of async traits. Traits are one of Rust's best features, and everyone has to work around them, meaning that you have to think differently about the same problem depending on if you're in async vs not.

Another example is implementing the Future trait (implementing a trait is standard practice in Rust). Only this often requires awareness around auto traits like Send, HRTBs and pin projection, (which happens to be one of the most complex things I've done), and all I'm trying to do is wrap some user provided future. These deep topics are more of a convoluted type system puzzle (and if you like those it can be fun). But it has very little to do with concurrency and much less with whatever business logic we're trying to manage.

> async

Concurrency does automatically not imply async outside of Javascript.

I am in the camp that async/await in Rust is a mistake.

There are certain things like "Concurrent Data Structures" that don't even have well-defined semantics in a non-GC language. I think "async" is in a similar category.

I agree. I've been writing a lot of javascript for the past few years. I've come to love async in javascript - async makes my javascript code fast, readable and reliable.

Moving to rust, a lot of the struggle I've experienced has come from trying to write rust the way I would javascript - by using rust's async primitives. Async support in rust is complex, immature and incomplete. And its been this way for years.

For a project a few years ago, I tried making an HTTP server which supported a custom SSE-style endpoint. The resulting code was atrocious - it was long, complicated, and I needed to invoke deep lifetime magic that I still don't fully understand to make it compile. I just want to implement both sides of an async stream - but no; thats apparently too hard. I got hundreds of lines in, and the code still wasn't finished. I rewrote it in about 20 lines of javascript and it works great.

I've decided to stay away from async rust until the language & ecosystem mature. Synchronous rust is a delight. I'm going to need to do more IO soon, and I'm dreading it. I don't really want to use threads, but I think it might be the right choice given the current state of async rust.

Does anyone know what the final state of Rust’s async will look like? It seems like everyone is dreading it now but will it ever get to a point that people like writing it?

async is final unfortunately. My experience with rust so far is that there is a small set of program structures which Rusts type system works well for, and a large set of programs and structures where it becomes an unholy mess to make work e.g. deep lifetime magic, complex async/traits scope, etc.

If you stick on the golden path then the compiler has your back, deviate, and then you are stuck in a deep hell of compiler suggestions that don't pan out. One observation I've had is that complex lifetime problems are usually solved by copy's or the use of unsafe in most production code. I would love to see a future equivalent of "clean code" for Rust, as the current situation is reminiscent of early java programming.

Async is final, and I think the design is fine.

The problem is that async code doesn't interact properly with the rest of the rust language. You can't make async traits, async iterators or streams, or use async closures. The problem isn't async. Its that the rest of the language is apparently (still!) incompatible with async code.

All of this stuff is "coming" - and by that I mean, there have been RFCs kicking around since 2017 aiming to fix these problems. Blergh.

Eg:

GATs: https://github.com/rust-lang/rust/issues/44265 (This one might land soon!)

Async closures: https://github.com/rust-lang/rust/issues/62290 (seems to have stalled)

TAIT: https://github.com/rust-lang/rust/issues/63063 (seems to have stalled)

And so on.

> async is final unfortunately

This is wrong. Async traits and iterators will be huge improvements to the language, and there is more in progress.

Can you provide examples of copy/unsafe being required for complex problems?

https://rust-lang.github.io/wg-async/vision/roadmap.html

In my experience I quite enjoyed async Rust, there was definitely quite a bit of yak shaving required to get tokio and various other async/sync libraries to play nicely together, but once that was done I haven't run into any significant issues with it. Are there problems with it I'm unaware of or is the somewhat fractured async ecosystem still a too high barrier to entry?

I agree. It's not like there aren't plenty of improvements to be made, but I write a lot of async Rust and the limitations (which are well enumerated elsewhere in the comments) just aren't really an issue for what I do.

Lunatic (https://docs.rs/lunatic/0.7.0/lunatic/) is an Erlang inspired alternative to async rust and manual threads.

Maybe I just got lucky, but I just built an http to irc relay using Rocket (tokio based) and the irc client crate (also tokio based) and the implementation came out super clean. I made some small wrong turns along the way like breaking graceful shutdown by putting a non terminating future into a join! with the main Rocket future, but with enough time in the various docs I got it right and it's pretty elegant. The Anyhow crate made error handling easy, and I'm happy to see Rocket building on stable since last time I used it. There wasn't really any borrow checker pain or messing with lifetimes. The compiler was very helpful, as usual.

The first step of any Rust project is to decide if you really need to use Rust.

If you control the hardware, then you need to compare the dollar cost of using a GC language and just throwing more hardware at the problem, vs the dollar cost of using Rust and throwing more developer time at the problem.

> writing code in it is very hard because it constantly feels like you're wrestling the compiler

I've been doing Rust for about two years. This difficulty eventually subsides and writing "safe" code becomes second-nature. I rarely wrestle with the compiler any more.

Agree. For other languages it would be Stockholm syndrome but in Rust's case it is enlightenment.

For me I'd describe it as having transitioned from "fighting with the compiler" to "having a conversation with the compiler." If I'm doing anything at all non trivial there's a back and forth there still but I appreciate and understand the feedback more now.

It took me about a month of free time coding to quit fighting the compiler. I'm not sure who won, but we work together now.

It's a real good feeling when you can finally write 100 lines or so that rustc takes on the first try. Even if it was all comments...

To add my own few cents to this very common discussion Rust is hard is the same way that most programming languages that rely "non-standard" paradigms are hard because they challenge your current mental model of programming. However "different", instead of "hard", would be a better way to describe these sort of languages like Rust, Prolog, and Haskell (as well as a ton of other languages).

If you look at Rust from this perspective then it becomes much more valuable to consider learning because it teach you a different way to modelling your data, and how to break-up your code; and while a language like Java may not force you to consider the borrow checker you will probably be a better overall Engineer knowing how to make it happy even when working in other languages.

On the topic discussed in the actual article: I can't wait for scoped threads to finally go stable since being forced for throw an Arc around almost everything you want to share between threads does not really suit Rust which really emphasise zero-copy in their standard library.

I think a lot of the complaints about rust's difficulty come from people trying to use async in rust. I don't think the problem there is that rust is "different". The problem is that async rust isn't ready yet.

Rust's ecosystem pushes users toward using async for IO, and async support in the language & compiler is wildly incomplete. And it has been for years - the last big async feature (the async keyword) landed 2.5 years ago [1]. But its barely usable as-is. Building software with async today feels like trying to build a house out of daggers. Traits? Nope. Closures? Nope. Regular types? Only if they're Pinned. Your types are pinned, right? Iterators over async objects? Nope that doesn't work. Library support? Without traits, libraries are limited in what they can provide. Async rust? Batteries are not included.

We need TAIT, GAT, async closures, async iterators / streams, and a bunch of other little quality of life improvements before I'd consider it "done". I don't know where async's momentum went, but its a bit disappointing how halfbaked async is given how nice the rest of the language is.

[1] https://areweasyncyet.rs/

I agree wholeheartedly on your points regarding async, which is a topic I've tried to avoid so far in all of my Rust projects because of all the reasons you've already raised above and you really don't need it for most use-cases.

How are people writing high throughput web/network services without async? Is there a async library to use. Like what do you reach for if you wanted to write redis or nginx with rust?

I'm not a rust expert by any means, but redis and nginx are both written in C, which doesn't have async as a language feature. Is there a reason the techniques used there wouldn't translate to rust, regardless of the state of rusts native async support?

yeah, rusts async/await is just making it nicer to use the same ideas. like is there a libuv or libevent in the rust world thats being used?

https://crates.io/crates/mio is the de-facto standard package that's similar to libuv/libevent. The most popular async executor, Tokio, is built on top of this. They're maintained by the same organization.

I thought this library had alot less features then the popular c event loop libraries?

I haven't written anything against it directly in years at this point, so I can't speak to that, it's just the best example of a library at the same level of abstraction.

You can, it's just not as nice. Writing code this way can also require unsafe. Async in Rust is primarily about achieving zero-cost async code in a safe way.

I totally agree. I've been working with Rust since before 1.0, and I love the language, and I never want to work on async code in Rust. The mental model is too hard for the payoff. I've also found debugging async code to be more nightmarish.

Most Rust applications I've worked on which have used some kind of parallelism/concurrency would be fine with a thread pool of workers and good old sync channels.

> Rust is hard is the same way that most programming languages that rely "non-standard" paradigms are hard

Agree.

And the problem is not (mainly) async, the borrow checker, etc.

Rust is truly different to other "imperative looking, functional looking" languages. I suspect the choice of make it look too close to C is part of the issue: You come to it and quickly start producing code as if it were another C/python/C# and that is a major mistake.

In specially, because Rust make SO MUCH SO EASIER... but not the things that other langs make easier (use of `String` anyone?), so is a major shock to see that supposedly "simple" things "not work".

Is only after getting that the core of Rust is `Struct Enum Pattern Matching Moves & Borrows`, and pls, stick all together with this `Traits` & composition, that are part of this important `Paradigms` is when Rust click.

This is the shock: Rust is not about the use of some `types` and then apply algorithms. Is the MODELING of behaviours (traits) and the INTERACTION of the types/traits.

That is the mind-bending. Is like see programming flipped!

> I can't wait for scoped threads to finally go stable since being forced for throw an Arc around almost everything you want to share between threads does not really suit Rust which really emphasise zero-copy in their standard library.

Note that this is only for lexical scopes so each set of scoped threads you create must be destroyed in the same lexical scope.

In practice, this means that thread pooling (and in particular async runtimes) cannot use them.

In fact, we used to have the closure passed to thread and join handle had a lifetime param 'a (not 'static), but this was unsound in combination with Arc etc that didn't require those. If you're interested, it's related to the drop guarantees and safety of `mem::leak`. There was a big debacle right before 1.0 and, imo, the team rushed this decision and we're paying for it with Arcs - any async code base is riddled with it.

A likely large contributor to others' experience not matching yours is async: Newcomers often refuse to try rust's excellent native threading support and jump right to the badly tacked-on, lets-write-javascript-in-rust monstrosity called async. Async "won" in the rust ecosystem because of superior branding but it is by far the worst solution to parallelism in rust. It has, by necessity, 100% of the problems and limitations of threading, such as they are, plus many more of its own. It's impossible that async could ever be easier than threading and yet countless newcomers are lured in and convinced they don't understand threads therefore need async. "I don't understand how to fly a Cessna therefore I need to fly an F-15."

Async has been in development for a few years during which the experience wasn't great. The popularity of Rust made it so many people wanted to try it anyway, with a plethora of frameworks being developed and replaced and not working when the next version of futures/tokio/hyper came out.

The experience now that futures are in the standard library and tokio etc are actually stable is much better. Most async libraries are thread-safe, contrary to the JavaScript or Python asyncio worlds, so you are free to call call block_on(func(...)) in whatever thread if that is your preferred coding style and the library doesn't already expose a blocking interface. The low-level "zero-cost" future composition is available to you if you need to squeeze a lot of performance but you absolutely don't have to wrestle with the "monstrosity" if you don't want to.

Well, users are obsessed with Async because Rust authors themselves are obsessed with it. The amount of discussions around design and development of Async feature is endless. The common theme is how a normal crappy Go app will become super app with Rust's zero cost async support.

I can understand they chose this path to gain market share by having tons of web services written in Rust. And async might be great fit for it. But now one can't just go around and say 'Oh, Rust async is not for plebs they can continue with something much simpler to fulfill their needs'.

Yep and the worst is that it seems that now 3rd party dependencies basically force it because they've also made that choice.

And one thing in your stack is async, everything else ends up following.

I'd love an async filter on `crates.io` and `docs.rs`.

Async works great in other languages. I adore being able to get high performance out of a single thread in javascript thanks to its async IO model.

Rust's problem with async isn't that async itself is a bad idea. The problem is that rust got halfway through adding async support and then didn't finish the job.

If you have half a house, your problem isn't that houses are a bad idea. You just need a builder. (Or, a wrecking ball. Which I'd be more than happy with at this point.)

> (Or, a wrecking ball. Which I'd be more than happy with at this point.)

I doubt that would be necessary or appropriate. A lot of thought, discussion, and work has already gone into the design, and I'm inclined to assume that the people involved knew what they were doing. Rust's zero-overhead approach to async is much more ambitious than the async support in other languages. More work on the interactions with other language features would certainly be useful, but I think suggesting that the existing implementation be discarded is more likely to frustrate the people who have been doing all of this hard work than to accomplish anything useful.

> Newcomers often refuse to try rust's excellent native threading support and jump right to the badly tacked-on, lets-write-javascript-in-rust monstrosity called async.

To be fair, many of those newcomers have heard that threaded code is very hard to get right (which is repeated by so many people these days), and/or come from the JS world where async is pretty much the only choice you have. This is something that may be partially fixed by more communication. For example, the Rust book currently doesn't cover async at all. Adding a chapter on async and when to use it might help. Another issue is that lots of people come to Rust from/for webdev. The most popular web frameworks (actix-web, rocket, ...) are all using async. I don't have any idea on how to fix this.

If what you say is true, that "It's impossible that async could ever be easier than threading", then I think that's something that people should know and tell other people.

Threading is not an alternative to "async" and can't replace it.

The alternative to "async" is non-blocking IO and an event loop written by hand, possibly with some threading thrown into the mix for certain tasks. This is exactly what the async runtime is doing behind the scenes, and what languages without first class async support have to do to support this form of concurrency.

Doing an event loop manually is probably going to end up being more difficult and less safe than using the language level async abstractions.

Sometimes just spawning threads is the best solution, but it's fairly wasteful to spawn threads for something like IO, especially if there are many operations in flight (1000+).

How are Rust's concurrency tools compared with those of Java? In Java we rarely need to use mutex at all. Instead, programmers use high-level synchronizers (latches, barriers, phasers, exchanges, and etc) and containers (concurrent collections, executors, various pools and flows, and etc). I don't have extensive experience in Rust, and somehow I saw Mutex everywhere.

Totally!

I return to Swift/Kotlin for new UIs and wrestle with their errors.

Man, "wrestling the compiler" in Rust is SOOOO much better!

... and this is even considering very simple things (like see how they choke in simple syntax error with so small help provided)

It's great until you touch async to which Go is way better and easier to use.

Generally, I've found that wrestling the compiler involves doing something that should really be done in another way.

What resources did you use to get up to speed, both with the language and which libraries to use?

Reply on news.ycombinator.com