Hacker News Reader

Comment by haydnv | Hacker News Reader

One advantage is consistency across host platforms, but the main advantage is that the file data can be accessed (and mutated) in memory in a deserialized format. If you let the OS take care of it, you would still have the overhead of serializing & deserializing a file every time it's accessed.

View on HN

That's what mmap is for.

It might be possible to replace freqfs with mmap on a POSIX OS, but a) you would still have to implement your own read-write lock, and b) you would (I think probably?) lose some consistency in behavior across different host operating systems.

mmaps for read, explicit API for writing, a-la LMDB. Buggy readers can read inconsistent data but cannot corrupt the os.

Corrupt the OS? How might that happen?

Sorry, I meant the DB!

Which OSes does this run on that doesn’t have some kind of mmap operation?

It should work on Windows (because tokio::fs works on Windows) although I have not personally tested this

You can do mmap on Windows, eg. https://github.com/danburkert/memmap-rs

Personally I don’t see a scenario for myself, but I can imagine that there are some where this might be useful. But isn’t there a extremely high risk of data loss an inconsistency when adding an extra layer on top of OS file system handling?

If there is any concurrent access to cached files not through freqfs, there is a risk of inconsistency and crashes.

Freqfs seems like a shim you'd add to an existing project for a quick optimization. Whereas mmap et al. are "better" the same way any specific, built-to-purpose code will be "better" than just bolting a framework on. Sometimes it's the right call to do the extra work; sometimes it's 100% more effort (both development and maintenance) for an extra 10% gain.

Reply on news.ycombinator.com