sghemawat

Hi Kenton! No worries at all. I tend to be quieter than Jeff anyway (less public speaking etc.) and I am happy to not have a dedicated website. :-). -Sanjay

Not your fault. We just plain forgot to update the documentation, so the freshest available document is a few years out of date.

About [1] Sorry about that: the document you linked to is amazingly stale. tcmalloc has been releasing memory to the system for many years. See for example the IncrementalScavenge routine in a version of page_heap.cc from Dec 2008:

https://code.google.com/p/gperftools/source/browse/trunk/src...

One caveat: physical memory and swap space is released, but the process's virtual size will not decrease since tcmalloc uses madvise(MNONE) to release memory.

About [2], code using tcmalloc-specific features/symbols is definitely a problem. I would strongly advise against doing that and sticking to the libc interfaces instead for the reason you pointed out.

We have no such plans. The library is designed so that concurrency control is outside its scope. If I needed it for an application, I might consider using some application level code that looks like:

    bool CAS(db, key, oldvalue, newvalue) {
      lock some mutex;
      read key's value from db;
      bool result = (value == oldvalue);
      if (result) write key=>newvalue to db;
      unlock;
      return result;
    }

This should be not much slower than any hard-wired CAS we could provide from inside leveldb.

The use of c++0x should be limited to the small AtomicPointer class in port_posix.h. The class uses <cstdatomic> for its implementation. If you can provide a different implementation of "acquire store" and "release load" for your architecture, you should be all set without the need for c++0x.

I suspect that you may find suitable code in the chromium source code. See leveldb's port_chromium.h.

Merely an incomplete one. Leveldb never writes in place: it always appends to a log file, or merges existing files together to produce new ones. So an OS crash will cause a partially written log record (or a few partially written log records). Leveldb recovery code uses checksums to detect this and will skip the incomplete records.

leveldb is a persistent ordered map; bitcask is a persistent hash table (no ordered iteration).

bitcask stores a fixed size record in memory for every key. So for databases with large number of keys, it may use too much memory for some applications.

bitcask can guarantee at most one disk seek per lookup I think. leveldb may have to do a small handful of disk seeks. To clarify, leveldb stores data in a sequence of levels. Each level stores approximately ten times as much data as the level before it. A read needs one disk seek per level. So if 10% of the db fits in memory, leveldb will need to do one seek (for the last level since all of the earlier levels should end up cached in the OS buffer cache). If 1% fits in memory, leveldb will need two seeks.

One of the leveldb authors here. TokyoCabinet is something we seriously considered using instead of writing leveldb. TokyoCabinet has great performance usually. I haven't done a careful head-to-head comparison, but it wouldn't surprise me if it was somewhat faster than leveldb for many workloads. Plus TokyoCabinet is more mature, has matching server code etc. and may therefore be a better fit for many projects.

However because of a fundamental difference in data structures (TokyoCabinet uses btrees for ordered storage; leveldb uses log structured merge trees), random write performance (which is important for our needs) is significantly better in leveldb. This part we did measure. IIRC, we could fill TokyoCabinet with a million 100-byte writes in less than two seconds if writing sequentially, but the time ballooned to ~2000 seconds if we wrote randomly. The corresponding slowdown for leveldb is from ~1.5 seconds (sequential) to ~2.5 seconds (random).

It is durable from an OS perspective. I started typing in a long explanation, but I think the comment for the "sync" field in the options.h file captures things well:

  struct WriteOptions {
    // If true, the write will be flushed from the operating system
    // buffer cache (by calling WritableFile::Sync()) before the write
    // is considered complete.  If this flag is true, writes will be
    // slower.
    //
    // If this flag is false, and the machine crashes, some recent
    // writes may be lost.  Note that if it is just the process that
    // crashes (i.e., the machine does not reboot), no writes will be
    // lost even if sync==false.
    //
    // In other words, a DB write with sync==false has similar
    // crash semantics as the "write()" system call.  A DB write
    // with sync==true has similar crash semantics to a "write()"
    // system call followed by "fsync()".
    //
    // Default: false
    bool sync;

We don't have much experience with scaling to larger databases yet. There are known problems which will cause a (smallish) constant factor slowdown in write performance after the database becomes a few (10?) GB in size, but I don't recall the details all that well, and the implementation has changed somewhat since that experiment. I would like to characterize this better and fix things so we can support somewhere between 100GB-1TB databases well. It just hasn't become a priority yet.

The benchmark numbers on the linked page were from a small million entry database that easily fits in the OS buffer cache.