Hacker News Reader

Comment by mrjn | Hacker News Reader

Author of https://dgraph.io here. I'd say this is one of the best collaborations we've done. Due to Jepsen tests, we were able to proactively identify and fix issues in a complex real time distributed graph DB, that is Dgraph. Dgraph's graph sharding, replication and transaction implementation is unique and it took more work to make it all work correctly, while still achieving good performance, and without utilizing any third party system.

I'd like to thank Kyle for his work on Dgraph. We'll continue to expand Jepsen tests to try and identify more issues and fix the remaining ones.

I'm around if you have questions! Cheers, Manish

View on HN

> Dgraph has made significant progress, but 4 of the 23 issues we identified remain unresolved, including the corruption of data in healthy clusters. This work was funded by Dgraph,

Between this and your comment above, it restores a little faith in humanity that there are people out there who honestly want to make their work as good as possible and not paper over cracks.

If you're talking about funding, many of the recent analyses were funded by the company whose product is being tested. Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.

> Most companies do this kind of testing internally but it's good to pay for the recognized expertise and thorough process of jepsen.

Yes, but having it published so that everyone sees the actual results without sugar coating or outright ignoring problems is what I meant. I didn't mean that the testing itself was unique, but people funding research and letting the researcher publish it is.

(OK, "unique" is much too strong as that's what much of Jepsen does, but I feel like whenever I'm watching the news and there's a public health study talking about how good or not bad X is, it was often funded by an X industry group and reading more about it find it subject to a lot of issues.)

Thank you! This is the best acknowledgment of our work I've seen so far. A lot of blood, sweat, hairs, frustrations, and tears has gone into building Dgraph -- and this comment by you makes me feel proud of what we're building, and the community of people using it.

I'll echo that as well. You all are doing it right!

I assume more testing will occur after the four bugs are addressed. Will that be published on the Jepsen blog?

No man, the bugs are open from long time. The team is no more, don't know why. Only the founder is doing commits from months. I had so much hopes on dgraph, it was way better than other graph dbs but immature

Could that be the nature of Github? Doesn't it give credit to whomever merges pull requests?

GitHub differentiates author and the one who committed if they are not same.

Well, that's unfortunate. DGraph seemed like it was pretty cool. I wish them the best of luck.

Yeah, we're working on fixing the existing bugs and working with Kit Patella (https://github.com/mkcp) to further extend Jepsen's Dgraph tests.

The Snapshot Isolation bank test bug, in particular, is proving a lot harder to reproduce now -- it takes many hours (even up to 8h) to show a violation; which makes debugging harder obviously. The rest 2-3 are sort of related and easier to fix.

I doubt Jepsen blog post would be updated, but once all the remaining bugs are fixed, we'll write a blog post about it on https://blog.dgraph.io.

Were there any badger related issues? There doesn't seem to be reading the article.

None were found based on Jepsen testing done.

We had done a separate testing for Badger before Jepsen, where we had identified and fixed a few issues related to crashes. You can read that blog post here:

https://blog.dgraph.io/post/alice/

Yeah, we did talk about Badger a bit, but I think Dgraph's internal testing was likely going to stress it more than I could over the network. We have occasionally found storage-engine level bugs with Jepsen (Tendermint, for instance), but usually the low request rates mask race conditions in storage libraries. :)

Are these fixes also applicable for OpenCypher engine in Dgraph ?

Dgraph currently does not support OpenCypher.

Reply on news.ycombinator.com