Hacker News Reader

Comment by tyingq | Hacker News Reader

Cool. On this, though:

"/tmpfs_mount/ must be a ram disk of some sort"

Curious if it really must be, or if that's just recommended for speed.

Presumably it works by mapping that file underneath a regular memory allocator, that means extremely small and random IO patterns. I think "must" is appropriate as a real backing file would likely mean a slowdown of 1-2 orders of magnitude! Compare to a typical DB system where the allocation structure always has some kind of large block IO locality by design

The alternative to 'must' is 'should', and people ignore that more easily, resulting in bug reports like 'memcached performance abysmal when running with disk', 'why does memcache cause 1MB of IO when I only read 256 4-byte keys', etc

Not every memory access to a memory mapped file is automatically paged out immediately, so depending on vm dirty bytes, etc, you may get away with it for a while.

But the worst case will be dismal and not unlikely.

You can do it at really low load levels, but you'll lose performance consistency. mmap'ing files over a real filesystem of any kind is super complicated.

The access pattern isn't optimized at all for flash or HDD or etc... however it does work super well if that mount happens to be a DAX mount over persistent memory.

Can't you just pass in a raw block device, get rid of any fsyncs and you got some very volatile backing store? The OS will write things out as best as it can, and if the whole thing fits in memory (and it's a memcached instance, so it should after all), then read-write will be fast. And if you want to restart, just stop the memcached process, do a sync, wait for it ... and reboot. (This seems simpler than copying the tmpfs, but a lot less safe/deterministic?)

You can try, but it's not going to be very consistent. You're at the mercy of whenever the OS decides to start flushing pages.. head of line blocking, mmap_sem locks, file descriptor locks, etc. My old job had an mmap-on-disk storage engine at scale and it wasn't any fun.

I think the worst part is mmap-on-disk looks fine at first, and only comes out as a problem after you scale up a while. False sense of security. :/

Reply on news.ycombinator.com