Riak 1.4 Released | Hacker News Reader

Riak 1.4 Released

basho.com

88 pointsmoonboots13 comments

http://basho.com/basho-announces-availability-of-riak-1-4/

Have the weak defaults that caused 91% data loss when recovering from network partitioning been fixed yet? http://aphyr.com/posts/285-call-me-maybe-riak

Last write wins is certainly a horrible default behavior. It seems like Basho has worked on a CRDT cookbook to make it easier to deal with conflict resolution. With that in place for critical mutable values, it's a lot easier to swallow the immutable values approach for the rest of your data (similar to Datomic), which completely sidesteps this problem.

(Disclaimer: I work for Basho.)

Last write wins is indeed a bad default for data integrity; I believe we received feedback early on from customers that siblings were too confusing for developers.

The good news is that Kyle was deliberately testing things that were effectively guaranteed to break, thus the high rate of data loss. Most applications won't be simultaneously writing the same keys on both sides of a network partition; there will typically be a healthier mix of new and old keys being written.

Definitely all true. I don't mind putting pressure on Basho for better defaults though. Maybe that will be a 2.0 milestone. Knowing how other databases compared in similar partition tests, Riak is doing just fine.

I don't think you understood aphyr's post. You might want to try reading it again.

As a longtime follower, let me say, nice work guys! It's amazing to see how far Riak has come. I have a question.

> Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics

So the results can be sorted but are not stored in sorted order in the secondary indices?

> Introducing Counters in Riak: Counters, Riak’s first distributed data type, provide automatic conflict resolution after a network partition

Awesome!

P.S. more info here: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md

Secondary Indexing Improvements: Query results are now sorted and paginated, offering developers much richer semantics

So the results can be sorted but are not stored in sorted order in the secondary indices?

Secondary indexes are stored sorted on disk, but segmented per vnode. Previously, they wouldn't be sorted before being returned to a client.

Suppose you're querying for "Bananas" through "Bavaria"; the node that contains "Bassoon" could return its first result before the node containing "Bananas" and "Barons", which would, in old Riak versions, result in out-of-order results.

Disclosure: I work at Basho, and have been working on riak-ruby-client updates to support the 2i improvements.

Sorting used to be done through M/R queries. Does Riak perform a M/R sort on the results from each node from this release ? Just wondering how this sort is achieved and what it means for performance, if it is requested.

It does a merge sort, and I can't see that you can opt out of it: https://github.com/basho/riak_kv/blob/master/src/sms.erl

2i doesn't use mapreduce in normal operation.

Is there a Lua/openresty client on the horizon?

Turner broadcasting uses Lua/openresty and riak: https://github.com/bakins/lua-resty-riak

I have been looking through http://docs.basho.com/riak/latest/ but I can't find any doc about the counters.

We're working on expanding the official counters docs. We'll also have a blog post up about it next week.

Basho Hacker Sam Elliott started a CRDT cookbook that will walk you through using the counters in 1.4 (and eventually the new data types we add). That should get you started.

https://github.com/lenary/riak_crdt_cookbook/tree/counter_ex...

Enjoy. Also, come to ricon.io/west.html. We'll be talking plenty about Counters and all the huge stuff coming to Riak in the near future...

Reply on news.ycombinator.com