Hacker News Reader

Comment by laurencerowe | Hacker News Reader

For backend web development I was somewhat disappointed to see that many of the new web frameworks (all async) allocate extensively on the heap (for example lots of Strings in their http request types.)

I mean it works but I don't understand why I would go to the bother of thinking about lifetimes when it seems performance would be similar to Kotlin / Swift / F#.

View on HN

The short answer is that because async is not done yet in Rust.

The long answer probably includes GATs (generic associated types), and the even longer answer starts with the amazing work that Nico et al. does to reinvent the internals of the Rust type checker. (Basically - if I remember correctly - the compiler team is currently refactoring big parts of rustc, to librarify the type checker, and replace it with a PROLOG-ish library called "chalk", which is able to prove more things, so it allows better handling of GATs, so it allows better handling of async, where you get "impl Future<Output = T>"-s everywhere.

But also somewhere there is that the ergonomics of async/.await are still not very much in progress too. With better error messages people will be able to forego Box<>-ing and figure out what to use instead, and only heap allocate where they must.

It's possible to write low-level async code with hand rolled-polled Futures, and push/manage as much stuff on the stack as possible. But ... that takes time, and performance is already "good enough". (And/or probably there are bigger gains in performance in other areas, such as scheduling.)

And, finally, boxing helps with compile times. (Because it's basically trivial for the compile to prove that a heap allocated simple trait object behaves well compared to a stack allocated concrete, but usually highly complex (plus unnameable) type.)

Can I ask what you mean when you say that async isn't done? async/await and Futures are on stable now. There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that. Tokio and async-std are both fairly usable as well.

The team itself described the stabilization of async/await as being in an "MVP" state. Stuff that's been fixed since then:

* error messages (I think there may be more to do, but they're better and better all the time)

* size of tasks (Still more to do, but they've shrunk a lot since it was first implemented

* async/await on no_std (initial implementation used TLS to manage some state, that's since been fixed)

Stuff still to do:

* Async functions in traits (this has a library that lets you work around this with one allocation, having this requires GATs)

* More improvements to the stuff above

* other stuff I'm sure I'm forgetting

> There is no runtime in the core/std but that's by design, there is no short term intent from the Rust team to do that.

This is true but with an asterick; https://github.com/rust-lang/rust/pull/65875 ended up being closed with "please write an RFC," rather than "no."

Huh! Thanks for chiming in and that's good to know!

Side note, I just want to say that I have mad respect for you always finding a way to stay so positive while continuing to be so heavily involved with the community, even with job changes and all of that stuff going on. You're an absolute role model for how to participate in community management and a godsend for Rust.

Thanks. It has taken a ton of work, frankly. I made a conscious decision to try and improve in this way a while back, and while I'm not perfect at it, I'm glad that I'm moving in that direction.

As a complete noob to rust, I found attempting to manage a memory pool that is handed out per request to cause all kinds of borrow checking headaches.

My bet is people choose to allocate heavily rather than figure out how to share memory correctly with Rust. I need to learn more rust to get a hang of it so I don't allocate so much.

Are there any rust folks who have a great "list of memory management models for rust"?

Rust does not yet have local custom allocators ala C++. (You can change the global allocator, but this doesn't help the "memory arena" use case.) The feature is very much on the roadmap, though.

While this is true, this doesn't inherently mean you can't do this pattern. https://crates.io/crates/typed_arena for example. It depends on exactly how allocations and your data structure are tied together.

It would be amazingly handy if someday it was possible to direct Rust to make all heap allocations under a certain block within an arena. At the moment it looks like you have to rewrite any third party code that makes heap allocations to use the arena explicitly at each allocation.

Yep, it would. There’s certainly a bunch of work to do here still.

I often write my code to allocate at first just to get a feel for the API and business logic and then go through it later to remove them. Given that all of those frameworks are still fairly new it wouldn't shock me to hear that that's the case.

I'd be hard pressed if a backend in Rust gives the same perf as Kotlin/F#. Especially if you write the Rust with an async stack you will likely come out significantly faster than the other options: small binaries, short startup time, less memory usage, more throughput, slightly shorter request cycles, ...

The only place that the langs you mention probably win is: learning curve (F# maybe not so much) a.k.a. getting language illiterate people up to speed, and compile times.

If you allocate and free small memory chunks at high frequency in a non-GC language, then it's quite likely that similar code in a GC'ed language will come out faster. The point of manual memory management isn't that it is "automatically" faster than a GC, but that it gives you a lot of control to reduce the allocation frequency (e.g. by grouping many small items into few large allocations, or moving allocations out of the hot code path).

Also, async itself doesn't magically make code run faster (on the contrary), it just lets you (ideally) do something else instead of waiting for a blocking operation to complete, or at least give you the illusion of sequential control flow (unlike nested callbacks).

Modern allocators (jemalloc, mimalloc) do pooling optimization just as well as GC. The point of non-GC language is usually about pointing directly to the source data instead of copy and having value type that nicely fit together in the same memory page instead of jumping around in heap space. I know JIT can do all that, but reliably? We'd need Sufficiently Smart Compiler.

Whether to use an async stack or not is very application dependent. For something closer to an application server than a proxy server I’m unconvinced async would be superior to threads.

I feel that Rust should be competing with C/C++ here though. I want a modern language with the same speed and control of memory usage. What’s A little frustrating (and don’t get me wrong, I’m rooting for Rust to get there) is that it feels like it should be possible to offer this, at least for threaded web servers.

There are other interesting languages in this space that I really enjoy playing with. Both Nim and Zig generate native code and are really fun, albeit very different, languages to work with.

I'm not sure. These reasons sound reasonable, but in webapps most of the time is spent IO handling anyway. As long as you're using async implementations in any language with performant "threads", I'm not sure of there would be a large difference in response times unless computation was involved in the request path.

You forgot development times.

I broke it down into:

> a.k.a. getting language illiterate people up to speed, and compile times.

With equally skilled people I dont think these languages yield different dev speed, apart from the damage done by compile times.

Rust never has to run a garbage collector, because it already figured out the lifetimes of all the values in your program statically, with relatively minor help from the programmer. This may make your code faster or more likely to be correct than code written in some of those languages.

It didn't figure them out at all, the programmer had to figure them out the hard way and then code them in so that the compiler understands them.

GC is figuring out things mostly correctly and the programmer can focus on other things, at least until they hit a performance problem :)

For the most part, lifetime is inferred from scope, so there isn't anything special to figure out; objects live exactly for as long as you can see and use them. This also includes such things as lock handles, or references to refcounted objects.

Only in the most trivial cases. Tons of real Rust uses copying or RC/ARC or even unsafe to avoid dealing with complex lifetimes because it's... complex.

Copying, refcounting and locking work within the exact same semantics. They're among the most typical tools that are used to transmit or share data, so I'm not sure why it should be strange to see them used in most real programs?

That doesn't change the reality of allocation though. Programmers in languages like C++ and Rust choose when and where to allocate memory, and if there's one thing we know about performant code, it's that we shouldn't be allocating and deallocating repeatedly. Nothing about lifetimes helps or hurts about this programming model. Lifetimes (Rust) and RAII (C++) are just semantic models to help reign in the cognitive load of memory allocation.

I don't think you've understood the nature of the borrow checker in Rust. It doesn't add any extra runtime behaviour, in fact you could compile Rust code without running these checks (though I don't think rustc provides you with a way to do so). Rust's memory management tools are no different from C++: you have stack allocated structures with destructors, which the standard library uses to implement heap-allocated smart pointers, shared pointers, mutex guards, etc, and you can hand out pointers to any of these. The only difference in rust is it checks how these pointers get handed out so you don't get use-after-free, data races, etc.

I doubt that you'll write a large Rust program without relying on refcounting at least for some of your objects. Refcounting is not fundamentally different than a GC.

Reference counting is a kind of garbage collection (taken broadly), but it's quite different from tracing GC.

Naive use of RC doesn't look like tracing much but as soon as you need cycle-aware RC like the CactusRef the algo looks a lot like tracing GC https://github.com/artichoke/cactusref#cycle-detection

That's a fair point, I should distinguish heap allocation from reference counting. It still seems somewhat tricky to replicate the well proven arena allocation approach of Apache and Nginx which avoids memory fragmentation in current stable Rust though.

For general areana allocation, you can use something like the Bumpalo crate: https://docs.rs/bumpalo/3.4.0/bumpalo/

Reply on news.ycombinator.com