Hacker News Reader

Comment by mrjn | Hacker News Reader

(co-author here) Thanks for sharing this post! We've elaborated the various issues we encountered trying to use a concurrent LRU cache in Dgraph and our dissatisfaction at existing choices which fail to provide memory management, high concurrency and hit ratios.

With guidance from Ben Manes, we're planning to work on a new concurrent Go cache based on Caffeine (Java). If you're interested in helping (or already have something similar), do reach out to us!

Meanwhile, check out Dgraph, our distributed graph database which is what keeps us busy: https://github.com/dgraph-io/dgraph

View on HN

What are you thinking about for the API signatures? Without beating a dead horse this is where go’s lack of generics really becomes burdensome.

If it were an on disk cache []byte would likely make sense but given it’s in memory I’m not sure. If you use interface{} you’ll need to measure that cost as well.

Our current plan is to store []byte given that badger stores blobs of bytes. Using interface{} may have some overhead, but we are no so worried about that right now.

So the []byte solution requires running through a serielization step which for most in memory uses will be expensive. Did you choose that because in you use case it eventually ends up setialized in any case?

correct!

I'm a go novice, and don't have any particular knowledge about dgraph, but is this a case where auto-generated code could be a good solution? (Assuming it's a bounded set of types)

I don't see why it's a problem, caching solutions like Redis also store bytes. If you cache a JSON blob you don't have much choice.

For an in memory cache (ie one not going to disk/network) you have to unnecessarily serialize your in memory representation to bytes. This is very frequently the most expensive (computational) step in a system.

If you are just going to send that stuff to nic/disk eventually anyway its perfectly fine (which is why something like Redis which is an out of memory cache can get away with it).

As mentioned down in this thread, the key-val would be bytes-bytes or string-bytes. While this is more expensive in terms of CPU, it has the advantage that the size of elements in the cache are carefully understood and do not need the cost of GC sweeps.

When de-serialized, in-memory structs take up more memory on heap and it gets hard to compute that -- we've tried that in vain in the past.

Having said that, there's still a general need to store pointers as values and that's something we'll look at once this byte-based implementation is ready.

Very cool, I have missed Caffeine when making stuff in go!

You might want to take a look at https://github.com/LiveAsynchronousVisualizedArchitecture/si...

It is a completely lock free key value store, though does not have eviction built in.

Its memory management is done using a free list of blocks, which is part of how it maintains lock free concurrency.

Reply on news.ycombinator.com