Distributed cache and K/V store in Go. Embeddable or independent service | Hacker News Reader

Distributed cache and K/V store in Go. Embeddable or independent service

github.com

23 pointsmastabadtomm13 comments

Didn't Discord have issues when they did caching in Go due to the garbage collector having to run over all objects ever 2 minutes? Was that resolved in a newer version of Go or does this project get around that problem in another way?

Brad Fitzpatrick and others pointed out that in that article they were using Go 1.6, but the garbage collector got way better since then and it would be possible that just upgrading Go would have solved the issue for them.

Seems like this has been merged:

https://github.com/golang/go/issues/35112

https://github.com/golang/proposal/blob/master/design/35112-...

As far as I know, without a generational or regional GC, golang will always have issues with large heaps.

I don't know the Discord case but Olric implements an append-only log file indexed with a built-in map(uint64 > uint64) to store data.

It was discussed here https://news.ycombinator.com/item?id=22238335

Ah, as I understood it the Discord issues only shows up when you do a large long term map(? -> object) but maps with primitive values don't trigger it.

What data structure does it use - LSM tree or hash table or something else?

Also if I'm using it as an embedded library does it do anything to help me to replicate the data?

https://github.com/boltdb/bolt seems very similar.

Olric is an in-memory and distributed key/value store. Bolt is an b-tree based on-disk key/value store and it's not distributed. Olric can be compared with https://github.com/golang/groupcache and it's highly inspired by Amazon DynamoDB paper and Hazelcast.

Why not patch/fork redis and include that as a cache that can be used in different languages and has ~all redis features ?

Is Redis embeddable? If not, it is sometimes building a whole new project is much easier and maintainable than a fork.

Olric is not Redis. Redis implements different data structures and has different persistence options. Olric only provides a key/value store and it's suitable to be used as a distributed cache. README.md file contains enough information to compare those tools.

Reply on news.ycombinator.com