Hacker News Reader

As far as "Atomic Datasets", he's referring to the ability to send off an atomic operation against a data structure. Yes, you can fake this by normalizing and storing it in a table (or any number of other permutations), but for counters, the ability to fire off an atomic increment is very useful.

View on HN

Your 'atomic operation' sounds a lot like a transaction. Or am I misunderstanding?

Not really.

If you have a bunch of counters, you don't want a hot counter to require a transaction every time you want to increment a page view. I'm sure there's a solution in high end DBs, but in mysql/postgres UPDATE somecounter SET value = value + 1 typically ends up being very painful for any significant write load. With Redis and Mongo, there are "atomic" increment operations that do a +X on a given field of a list or hash that is very fast.

see twitter's Rainbird for what a production scale solution looks like. 100k writes per second, with a huge skew in which rows get hit.

Reply on news.ycombinator.com