Comment by api | Hacker News Reader

Comment by api | Hacker News Reader

Parent Full thread

We use it here (ZeroTier).

Pros:

- Very easy and robust clustering (Raft-based, automatic fail-over). This is huge for us.

- Streaming change feeds. This one is also huge. Makes any kind of real-time, reactive, or event driven programming very easy and IMHO is something that should exist in every database.

- It's kind of half SQL. It's a NoSQL document store but encourages a relational design and supports many relational queries.

- Rational and pretty easy to understand query language. It's much cleaner than Mongo.

- Easy to deploy and configure.

- It passed the Jepsen tests before Mongo did and overall has a solid history of not losing data.

Cons:

- It's a CPU hog, at least when compared with PostgreSQL.

- It's also an I/O hog, though we sponsored some improvements that are getting merged in the next version that will reduce this and also make table commit a configurable parameter. You'll be able to have fully and partially (long flush delay) in-memory tables for highly ephemeral data.

Hey Adam - I'm very interested in your sponsored contributions towards reducing IOPS in RethinkDb. It's our biggest challenge, even though we're on SSDs. Our backfills after outages are especially long and painful. Are these updates something you're running safely in production today?

How do you handle the lack of transactions / atomic updates affecting more than one document?

Our data model generally doesn't require this. We're actually okay with less guarantees than RethinkDB provides. AFIAK NoSQL stores in general are a bad choice if you need this. You should use a SQL database.

To be clear, nosql vs sql doesnt mean much - use the right type of database for the scenario: relational, document, graph, key/value, etc.

They all have various support for transactions with relational usually the most comprehensive.

Not OP, but I admit that's not a problem I've ever had. I've done a lot of webapps supporting data-science pipelines, and I've built some major components of those pipelines. It's not something I've felt the need for when I've used postgres.

What do you use that for?

Doing destructive data rollups (combining multiple rows and deleting the original rows) are made much simpler with transactions, especially if you do hybrid aggregations of short term data and long term data.

For example, you might store data with minute level granularity for the past 24 hours but only hourly for the past 30 days. If someone queries the past two days, you need to look at both those datasets. Then, every hour or so, you need to summarize an hour of minute level data, insert it into the hourly granularity table and then remove it from the minute granularity table. Meanwhile, you want to make sure any queries aren't going to double count that data after insertion but before removal.

This can be done without transactions in a few ways, but they require putting your replication and rollup logic and constraints into your reading code, rather than having it isolated to your roll up code. And your data model has to be tweaked to allow for some of these operations. And the complexity often results in double counting bugs (or bugs where the data is not counted at all).

There are solutions though. They just require a lot more hoops than starting a transaction, moving the data, committing the transaction.

If you're going to use a database that does not support these features then you should come up with a data model that does not rely on them.

For example, instead of applying a bank account transfer as a database transaction that debits one account record and credits another, you create a new transaction record (account transaction, not database transaction.) Then account balances are a sum over these transaction records.

Reply on news.ycombinator.com