Hacker News Reader

Mmaped files are already available it seems (https://crates.io/crates/mmap/), although I've never tried it. You'd have to implement the interesting structures on top of that yourself however.

View on HN

Relative pointers are great in memory map data structures. All pointers are relative to current offset. That way it's almost always also possible to get away with just 32-bit pointers -- you get +- 8 GB range.

A library that can do that in Rust would be great.

A signed 32 bit "offset" value gives you 2GB in each direction, not 4GB in each direction.

Only if you use byte aligned relative pointers. For data structures you don't need better than 32-bit aligned pointers. So you get +- 8GB relative offsets. It's not all that unreasonable to use 64-bit or even 128-bit alignment, for +-16 GB and +-32 GB respectively.

Hmm, it seems this library only provides the basic I/O.

The main problem is not opening the mmapped file. The problem is having a kind of pointer that allows arbitrary offsetting, and using this pointer inside standard data structures.

Rust has typesafe arenas, when one allocates small objects from a bigger chunk of memory and typesystem ensures that small objects do not outlive the chunk. I suppose mmap files could be treated similarly when assembling a pointer from an offset stored in the mmaped file is treated as an allocation.

Reply on news.ycombinator.com