Writing an OS in Rust: Allocating Frames | Hacker News Reader

Writing an OS in Rust: Allocating Frames

os.phil-opp.com

128 pointsingve14 comments

This series has been really great. It's significantly more accessible than many other OS dev tutorials I've read in the past. I spent all day yesterday with a friend and the first few posts, it was such a good time. Nothing like seeing a machine boot up and print something, and know that you wrote 100% of the source code that makes it happen. Even with a tutorial, it's magic.

In case anyone is interested, the best OS tutorial I've ever found is here: http://www.cs.bham.ac.uk/~exr/lectures/opsys/10_11/lectures/...

I've never been able to find a finished copy, but this one definitely gives you a good start, if only for 32 bit archs.

In any case, I'm pumped to try out this Rust one- looks very well done, and it's finally an excuse to learn some Rust!

It also takes you through the 32 -> 64 bit trampoline, which is nice, given your previous source. :)

So basically you can't use standard library memory logic for that?

Somewhat related topic I found: https://www.reddit.com/r/rust/comments/2mthq2/how_would_a_ru...

Right, because the stdlib calls out to C/C++'s malloc/calloc/realloc/free implementations. Their allocators ask the kernel for memory which, in this case, doesn't exist yet because we're starting from bare metal.

This may not be universally true. The Rust docs[1] point out that as of today, compiled binaries use either jemalloc[2] or the system allocator:

> Binaries generated by the compiler will use `alloc_jemalloc` by default (where available). [...] Dynamic and static libraries, however, will use `alloc_system` by default.

[1]: https://doc.rust-lang.org/nightly/book/custom-allocators.htm...

[2]: http://www.canonware.com/jemalloc/

Rust has used jemalloc for a long time (since Rust 0.8, I think). The new feature is the ability to configure it to use the system allocator instead.

But whichever malloc implementation it uses, it still relies on the kernel. For example, jemalloc uses the operating system to map memory, by calling library functions like VirtualAlloc (win32) or mmap (POSIX).

https://github.com/jemalloc/jemalloc/blob/3a92319ddc5610b755...

That sounds like a deployment nightmare! Does that mean if you return a pointer from a dynamic library you can't free it in a static library, because they were actually using two allocation systems?

At the moment Rust doesn't have a stable ABI (aside from things you mark as C-compatible), so dynamic libaries that stick around on your system to do things aren't used much. I bet there will be a solution for this once we stabilize the ABI, though I suspect it may already be solved (I think you can tell rust to dynamically link to the allocator too, or something).

Rust binaries essentially never dynamically link Rust libraries, mostly for the reason Manishearth said in the sibling. The decision to have different default allocators is because there were problems linking to Rust from C libraries which used the system allocator, because Rust previously was exclusively using jemalloc.

Because Rust libraries being used in Rust projects will be statically linked currently, they will be built using jemalloc. Deploying a pure Rust project is as simple as pushing out a binary (std links to libc, so very few Rust projects are technically pure Rust, but its as good as because the system probably has libc).

Also, as a rule, you don't "free" in Rust at all, and code which defines how the memory is deallocated will be defined in the same library it was allocated in. I don't know the details of what it means when two libraries using two different allocators are interacting, but I don't think its as total a conflict as if you were to try to free a pointer with the wrong allocator's free function.

Rust automatically chooses the "best" strategy based on how you're building the source, per https://github.com/rust-lang/rfcs/blob/master/text/1183-swap...

If you're building a dynamic or static library, we use the system allocator because Rust is "subordinate". If you're building a binary, we use jemalloc because Rust is in control and jemalloc is basically the best allocator in town. These behaviors can be manually overridden, and you can also provide your own custom global allocator (see linked RFC for details).

A Rust project using Rust libraries will generally link them in as rlibs, which is basically the Rust equivalent of an object file. rlibs just inherit whatever allocator from what they're linked into.

It it nothing new. There are OS which have an heap manager by dynamic library, Windows being one of them.

It is never a good idea to try to release memory in a module that didn't allocate it in first place, specially if one doesn't know how it was allocated.

That's generally the rule in C/C++-land. If you have two shared objects/dlls linked into your program that are linked with different versions of the C++ runtime, it's not safe for an object allocated by one to be freed by the other.

Yes, but using the core library [1] is always possible. And as soon as you have an allocator, you can additionally link the alloc [2] and collections [3] crates. Then you can use things like Vec or even BTree.

[1]: https://doc.rust-lang.org/nightly/core/index.html

[2]: https://doc.rust-lang.org/nightly/alloc/index.html

[3]: https://doc.rust-lang.org/nightly/collections/index.html

Reply on news.ycombinator.com