Symas Lightning Memory-Mapped Database (LMDB)
symas.com
symas.com
http://en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion...
edit: that page refers to a deletion that happened a year ago but lmdb had a page much more recently than that - even a month or two ago, I think - so apparently deleting the lmdb page is a hobby for someone. I'm not familiar enough with the tools to dig in and figure out what's going on there.
This is a problem with wikipedia in general. I think archive.org is/should be archiving dumps.
We also forked and improved some Go bindings to lmdb: https://github.com/armon/gomdb (which is the lib we use in Consul).
When building Consul, we were specifically looking for an in-process DB that supports MVCC. The reason is because while Consul is doing a snapshot, we wanted to be able to INSERT/UPDATE without affecting the integrity of the snapshot. LMDB fit this role nicely and the performance has been fantastic for our use case.
And still very immature.
He can also rock the violin.
http://symas.com/mdb/microbench/
Multi-threaded, vs memcache
PS and read the benchmark description carefully - some of the benchmarks are performed in memory, without actual disk i/o.
Similarly "not too fast for write-intensive apps" is only half the story - LMDB write transactions have a fixed cost related to the tree depth, so while many tiny updates may incur a noticeable fixed penalty in some cases, that cost becomes less noticeable e.g. with larger transactions performing reasonably localized updates (say, to partially sequential key ranges).
Also note even for huge databases, the write overhead of shadow paging is usually somewhere south of 64kb per transaction. To reiterate, the fixed cost is essentially per-transaction rather than per-update.
Depends a lot on your key and value sizes - the larger the values, the faster LMDB is vs any other solution, due to the zero-copy behavior.
The write pattern in the HyperDex benchmark is pure random but it still obliterates HyperLevelDB (and don't even think of vanilla LevelDB there). Keep in mind that LMDB uses free pages in sorted order, so even for a random write workload, pages are allocated in ascending order, which generally translates to unidirectional seeks on an HDD. This is why even for a purely random write load, LMDB is still faster than all of the traditional update-in-place B-trees out there - they really have to do random seeks, LMDB doesn't. (And you pay the highest in seek time when the drive head has to reverse direction.)
That's why LMDB write performance remains uniform under load, while all LSMs suffer from GC/compaction pauses.
But ultimately, storage is moving to solid state, and seek time will be irrelevant.