Hacker News Reader

Comment by gleenn | Hacker News Reader

As a guy who lives in the JVM most days and mostly ignores allocation optimizations, what are some examples of things that are actually newer features in such a project? Isn't allocation a mostly solved problem? Is something about WebAssembly or no_std actually requiring different features?

View on HN

In a no_std Rust environment, there is no allocator. There is no heap. So an allocator is needed to use things like Vectors or Strings.

This is very common in embedded contexts, where you can take nothing for granted.

What's the point of using `no_std` if you're just going to add an allocator anyway? You may as well just use `std` at that point no? (You can use `std` on embedded devices with a small amount of work.)

To get std you need some kind of libc replacement. You don't need that if you just want to use alloc.

Yeah, just adding, in Rust, there's three main levels:

1. no_std (like no libc + no malloc in C)

2. no_std + alloc (like no libc + malloc in C)

3. std (like a full libc + malloc in C)

The difference between 1 + 2 is like three lines. The difference between 2 + 3 is a change to the entire standard library. ATM only ESP32 devices support option 3 (they build a standard library implementation on top of FreeRTOS/ESP-IDF).

The `core` library is still available on `no_std` and contains a lot of useful stuff, so it’s not exactly like no libc on C. That would be `no_core` which is pretty hardcore (heh). The big things missing in `no_std` are

* file and other I/O, including filesystem ops

* access to system time

* threads

* collections and some other things that require an allocator (not many things actually do in Rust’s stdlib!)

* floating-point functions (the types themselves and builtin operators work fine)

`alloc` gives you `Vec`, `String`, `Box`, `BTreeMap/Set`, ref-counted pointers, and a few `Vec`-derived collections like `VecDeque`. Very annoyingly not `HashSet/Map` though, due to a literally single-line dependence on a system entropy source which happens to not be easily factorable out because reasons.

You don't. It currently requires `-Z build-std=std,panic_abort` and some nightly flags (e.g. `#![feature(restricted_std)]`) but you can build `std` programs on bare metal targets. I can't remember exactly what it does if you try to open files or start threads or whatever (probably panics?) but you can compile and run it. If you don't do any of those things it works fine.

Currently the `sys` crate implementation is hard-coded into the compiler but eventually you will be able to provide it without modifying the compiler so you can e.g. target a RTOS or whatever.

It looks like that work started really recently actually:

https://github.com/rust-lang/rust/commit/99128b7e45f8b95d962...

Maybe you're implementing your own "std".

Small binary size (and thus a simpler implementation) are the min things you want for wasm and embedded targets. For wasm this is because its being sent over the network, for embedded because the device may have low amounts of available ram and storage.

It's also the case that such targets often can't take advantage of the advanced features (like multithreading optimisations) that "full fat" allocator provide.

An allocator and deallocator go hand in hand. For the JVM, most of the GCs are compacting so they can use bump allocators. In that case, yes, it is a solved problem. However, it depends on the alloator/deallocator pair and traditional malloc/free implementations involve many trade offs, so allocation work continues.

It's only a solved problem if you can live with the downsides of automatic memory management. For instance, do you know exactly how much time your garbage collector spends with memory management and when exactly that time is spent, where objects are located in memory relative to each other, and how much time is wasted on cache misses when accessing those objects? If those questions are not important, then automatic memory management is good enough. For other applications those questions may be more important, and automatic memory management is then usually harder to optimise than coming up with specialised manual allocation strategies.

I am also living with GC (mostly .net and go) but recently took a look at Zig.

Zig having allocators totally opaque is interesting and you get to learn a lot if you dig deeper. Tons of different designs. Allocators on top of allocators. Tiny ones for small bundles. Non deallocating ones for one shot apps. Stuff you never care about in a GC environment.

I suggest checking it out if it interests you.

I’d say it’s solved if you want a garbage collector. If you don’t, then there is plenty of room for innovation

Reply on news.ycombinator.com