Hacker News Reader

Comment by igorkraw | Hacker News Reader

As a rust fanboy, I was very happy to see the 99% and memory numbers. Still, very impressive for Java, I wonder whether this is just trading off the 99% percentile peak vs higher common case throughput? I guess rust actually allocates/deallocates all the memory on each request while java batches the deallocation?

View on HN

Looking at the memory column, it looks to me like Java benchmark uses a large heap to avoid deallocations for most of requests. This means less but larger garbage collections and reduces average time while increasing 99% where the GC shows up.

A long time ago I had to implement a real time service with Java. The best solution was to use whopping 16 megabytes of heap so that it would do a full GC multiple times per second but each of the GCs lasted less than a millisecond.

Was there a theoretical upper bound for how long collecting those 16 MB could take?

No idea. It was much faster than the service really needed to be, so we just set the heap to 128M in production and never heard about any latency problems.

Just curious, did you experiment with different GCs?

At the time CMS was the only option.

In server applications you really shouldn't use the default allocator, but have one allocator per request which can then allocate and batch-deallocate very quickly. Unless you want to keep objects after the request ends, of course.

Is that possible in Rust these days?

Kind of. You can use a bump allocator like bumpalo[0], but any code that allocates needs to be aware of it (or it won't be able to take advantage of it). It also needs to reinvent all collection types (bumpalo provides strings and vectors atm).

The plan is to parametrize all std collections on the allocator eventually, but that's not stable yet.

[0]: https://docs.rs/bumpalo/3.6.1/bumpalo/

Hopefully this project could allow allocations inside memory-mapped files (which can then later be loaded at a different base address) in one quick load operation (instead of serializing everything).

Or you could just use jemallocator with background threads.

https://crates.io/crates/jemalloc-sys

"background_threads (disabled by default): enables background threads by default at run-time. When set to true, background threads are created on demand (the number of background threads will be no more than the number of CPUs or active arenas). Threads run periodically, and handle purging asynchronously. [...]"

What about an app server same process in memory cache, if using per request allocators?

> keep objects after the request ends

Yes. Can that be made in a way compatible with per request allocators?

Don't use the per-request allocator for those objects. You don't need to use the same allocator for everything.

That sounds great :-) Thanks

With optimized memory handling Rust should easily beat Java. Reusing allocations or at least using jemalloc/mimalloc.

Reply on news.ycombinator.com