Hacker News Reader

Comment by boustrophedon | Hacker News Reader

Without specifying a particular representation[0] on the struct you want to put in "permanent memory", this seems liable to break any time you recompile the code with different optimization flags, or with a newer or older compiler that may optimize differently, or even if you just change seemingly unrelated code that then triggers or de-triggers some optimization.

[0] https://doc.rust-lang.org/reference/type-layout.html#represe...

View on HN

The rkyv crate has stable in-memory representations for many types in the Rust standard library: https://rkyv.org/

I feel like it is the missing piece here because you could do hash table lookups in shared memory, too!

Yup, at a minimum they should apply #[repr(C)] to avoid this optimization.

Worse, it breaks all pointers since you can't guarantee `mmap` can give you the same address.

You absolutely can, with `MAP_FIXED`. But it's not trivial to do because you'd have to handle the case where the rtld and/or the kernel has already put something in memory where you want to put down your map. So it'd be tricky to implement, but definitely possible.

That can be fixed with a linker script, to reserve the range using ELF headers.

But then you're going to hit other problems. This isn't a new idea. Presumably one of the reasons you're restarting your server is to upgrade it ...

I think you could fixup the pointers if you tried. It's more or less the same problem as compacting GC.

Certainly there are solutions, but they aren't transparent, and at some point you have to ask "would serialization be easier?"

This is exactly what I thought: why not just save state to a file?

You can detect when it doesn't, though, I think. If memory serves you can request a specific memory address to mount it at and it'll do it if it's not already mapped. It should (in theory?) return an error in such a case, no?

I believe on conflict it will just fall back to placing it where it can by default. It will return the address it actually used, so you can catch the mismatch.

Good point, thank you! I added `repr(C)`.

Reply on news.ycombinator.com