Hacker News Reader

Comment by nextaccountic | Hacker News Reader

> Well, today computers really only have one kind of storage, and it is usually some sort of disk, the operating system and the virtual memory management hardware has converted the RAM to a cache for the disk storage.

> So what happens with squids elaborate memory management is that it gets into fights with the kernels elaborate memory management, and like any civil war, that never gets anything done.

ctrl+f mmap on this article doesn't return anything so.. is it talking it's better to mmap files and having files store in-memory data structures like cap'n'proto does?

(if it's entirely in-memory with no file in disk, then it's surely comparing apples to oranges? if a program needs the persistent memory, it needs the disk)

Also: is mmaping a file and using it directly from memory what databases should do today to avoid fighting the OS disk cache?

View on HN

Mmaping directly readable data is certainly a way of avoiding the problems PHK is talking about, and can be reasonably efficient. Sendfile can be even better if you can structure things to have a metadata index that's in memory or mmaped, and never need to read the actual sent data.

But mmap is no panacea. It's big flaw is that the kernel scheduler often doesn't know as much as your app about what you're doing. So when you touch an out of core page and it page faults, your thread just gets locked up. Worse, the kernel's read ahead logic may make the wrong guess about how many pages you're going to read, or when you want data written back out. You can try to mitigate this using madvise and msync, but in practice it's kinda brittle.

Some systems use blocking io with a dedicated thread pool to wait on the block. This way the main threads can move on to other useful work rather than sitting stalled on a page fault. Golang's runtime does something like this for you automatically.

Databases for the most part do their own scheduling, and may even use async io. This is because they know far better what they're actually doing than the kernel can guess. Another reason is the database's buffer cache is usually tightly coupled with the consistency control algorithm, so again, you don't want the kernel guessing, you want total control. Databases often use direct io and take on the burden of scheduling everything themselves precisely for this reason.

LMDB uses mmap, but should not really be considered a database. It's just an embedded CoW btree, though a fine one.

Andy Pavlo's openly published course materials are a great resource for learning about database internels. He used to be emphatically against mmap for databases, but in the last couple years has moderated that position somewhat.

io_uring is looking like it will be the best overall solution moving forward, at least for linux only.

Thanks for the explanation.

> io_uring is looking like it will be the best overall solution moving forward, at least for linux only.

But io_uring means duplicating memory, right? There will be both my in-memory representation and the kernel cache for the file and they will be fighting each other (or at least, consuming more memory than necessary.

You can use io_uring with O_DIRECT, with the usual alignment restrictions. io_uring itself is carefully designed to eliminate copying even request/completion event structures.

Thank you for the pointer, this seems very interesting. do you know of an example of open source code using io_uring and O_DIRECT to bypass the fs cache? What about posix_fadvise?

I'm reading here https://unix.stackexchange.com/questions/6467/use-of-o-direc... that some people don't think it's a good idea to bypass the fs cache like that. Perhaps mmap could have been salvageable if it had a better error handling API.

I don't know an example of that sadly.

O_DIRECT is definitely doing things the hard way for sure. There's a very real risk you won't be as smart as the kernel, even knowing your app's patterns well. Linus famously has ranted against this for a long time. But like mentioned above, with databases, taking on the complexity is often warranted.

I've been tinkering off and on with a database design, and one of the things I want to explore is a hybrid approach where I still have an on heap buffer cache, but misses to that cache get filled out of mmap. The buffer cache is a lock free hash table, greatly simplified by using Copy on Write pages as a fundamental part of the design (don't have to worry about racing updates or duplication races during resizing other than as performance degradations). It's still not finished and totally untested, but the way I imagine it working is the buffer cache is smaller than what you'd typically reserve for a database, and the OS can manage its page cache as a sort of L2 cache.

Thanks for sharing those ideas. This sounds interesting. Is it on github?

Nope but I'm trying to herd my own cats to get a prototype shipped and up. It'll get shared here when I do for sure.

Nice to hear, I hope it does well!

LMDB is an embedded database engine, the same as Berkeley DB, gdbm, ndbm, etc. (except gdbm/ndbm/etc don't support multithreading, ACID transactions, various other stuff that are essential to DB operation). You use them to build other data management systems. Thus, there was a Berkeley DB backend for MySQL, just as there was a Berkeley DB backend for OpenLDAP. As storage engines go though, LMDB outclasses all the others for compactness, robustness, and efficiency.

The eternal debate over app-level caching is always based on the myth that the DB application knows better about the data access patterns than the underlying kernel ever can. This may be true for some data structures, but is irrelevant in the modern computing landscape. The fact is that applications today are running on multi-tenant VMs/containers/etc and only the hypervisor and kernels have a comprehensive view of the resource demands of the entire system. Back in the days when Oracle servers ran on dedicated hardware, it was practical for the DBMS to take all decision making away from the OS, but that is not how computing is done today.

Today your apps are sharing resources with countless other unknown apps, and any cache/memory optimizations your app may attempt are futile and instantly nullified by the actions of other unrelated apps in other unrelated VMs/containers.

Also, as I mentioned above, the data structures make a difference. LMDB relies solely on OS page caches because only the underlying virtual memory manager really knows what's going on in the system, and also, B+trees are inherently the most cache friendly. Your app-level data access patterns just aren't going to outperform them. The tree structure inherently optimizes for LRU cache mgmt; the tree root is always hit on every access, so it is always most-often-and-recently-used. As accesses navigate down the tree to the leaf nodes, each node is successively less frequently used. The net result is that the root and interior branches naturally stay hot in a cache, and the leaf nodes naturally age out fastest, resulting in optimal cache hit rates for any access pattern.

In a modern multi-tenant environment there is no scenario where an app has enough knowledge to make optimal resource mgmt decisions and outsmart the underlying OS.

Yes, I understood that to mean mmap().

> is mmaping a file and using it directly from memory what databases should do today

Some do; for example, SQLite: https://www.sqlite.org/mmap.html (the page has pros and cons, and explains how SQLite does memory mapped I/O).

I thought the fact that mongodb data files were essentially mmapped was used as a critique against its durability as a primary database. Is that not true?

Or was that not a relevant critique?

Welp, I don't know!

From the link of the other comment here, Sqlite disables mmap by default because Linux provides a poor API regarding I/O errors (they are sent as a signal instead)

A database like mongodb is a very different animal from a http cache. The mongodb authors likely read this article (it's from 2006) when they were designing the first version of their database, and due to their almost complete unfamiliarity with what they were doing didn't understand that these two things are not alike.

Databases have different classes of objects (think index vs data, though there are more than just these two), different types of access (e.g. full table scan vs single record select) and a whole lot more complexity than I can cover here. You don't really ever want to page out your indicies, but paging out data is probably fine. You likely want to avoid replacing your entire cache for full table scan of a big table. You probably want to have some concept of ensuring a single user doesn't starve out all your other 1000+ users. Etc etc.

In other words, you can't get away with just opting out of memory management for a database - you need to treat different things differently. I understand that a few years in, the mongo devs eventually caught up to 1990s era mysql and realised this, leading to a new DB engine that was a bit less bad.

MongoDB released the Wired Tiger storage engine in 2015 which replaced the MMAP storage engine. So whatever your read about MMAP is obsolete.

https://www.mongodb.com/presentations/a-technical-introducti...

Yep I know that. I’m saying mmap used to be a critique.

I left the place I introduced mongo to, a couple months before they bought WiredTiger

It's a valid critique, but A cache has no durability requirements.

Reply on news.ycombinator.com