Hacker News Reader

Comment by folex | Hacker News Reader

Is it possible to make Badger completely in-memory? I.e., disable persistence.

You can keep both the value log and LSM tree in memory, exposed by badger.Options. That'd still give you persistence and crash-resilience while allowing your reads to benefit from direct memory access. If persistence is a non-goal, yeah, tmpfs would work.

If you use it as an in-memory DB, then I'd recommend using the LSM options, instead of Default options.

Related issue discussion: https://github.com/dgraph-io/badger/issues/377

How about using tmpfs?

You might as well use a simple hash-table then.

It seems that most of the design effort of this project was focused on fast disk access.

Fast disk access on linux means async io+direct io, which badger does not use.

Files are essentially mmap'd.

Regarding the hash table suggestion, you would lose compression/compaction and indexing/prefix scans.

> Files are essentially mmap'd.

Yes, but using LSM tree has a performance penalty over plain hash tables.

Yes, you would lose compression. But you could still have prefix scans by replacing the hash table by e.g. a red/black tree.

Mmaping is optional. Badger doesn't require it, it gives you that option. Async IO in Go has been debated vigorously over the years, but there're little benefits in building that library. Because Goroutines do an equivalent job. Which is what we're doing, random reads (for value log access) spread over many goroutines.

My bad, I just glanced at the code. Are you using O_DIRECT?

Goroutines do an equivalent job but write is synchronous and thus blocks one thread of the go scheduler, as far as I can tell.

Also the benefit of the async io API is that it allows sending multiple requests in ones call. Syscalls in go do have an overhead which can be mitigated this way, I guess.

You'd also lose notions of transactions with a simple hash map - it can be nice to have some built-in locking / ACID-ish features, even in-memory (I use SQLite with :memory: for just that purpose sometimes).

Reply on news.ycombinator.com