Hacker News Reader

Comment by halayli | Hacker News Reader

This looks like a big hack to compensate for using the wrong tool. Cassandra would have been a better solution IMO.

With Cassandra, you can set replication factors, speed up the writes, and automatically shard the data without having to manage your own "mapping tables".

View on HN

Cassandra was immature when this work was started (late 2011). The team was also much more familiar with the ins and outs of operating mysql.

Technical debt is not an excuse to use the wrong tool. This is how you end up having a hammer.

Cassandra was mature by late 2011 and we were using it in production back then (billions of records).

The mapping tables are used to store relationships between objects, not to define shard locations. The shard mapping is encoded in the object id.

In any case, I would have a knee jerk reaction to not trust any database system that hit 1.0 a month before I wanted to start using it, especially for mission critical core data.

Doubly so when everything is catching fire around you from how overloaded it all is.

For this very reason many projects nowadays jump directly from 0.x to 2.x :) 2 is the new 1.

Technical debt doesn't necessarily arise from using the wrong (a.k.a pretty similar but with a few features less than what you need) tool. In my experience, it's more likely that technical debt arises from trying to use thew newest tool (which looks to be a silver bullet) rather than trusting your own experiences. It's surprising to me that people are willing to move to a completely different technology rather than extend and repurpose what your engineers are already familiar with.

Although, by using an existing tool, as long as it fits your needs, you'll avoid reinventing the proverbial wheel, ofcourse at the expense of learning something new.. which might not be a bad thing..

But if you also need to use MySql anyway, Cassandra would be the "wrong Tool". Because you would have to maintain two DBMS instead of one. You want to minimize complexity and this can mean implementing your own Sharding.

There's also no transactions in Cassandra, although that's questionable useful given that the unidirectional map may be on a different machine.

Cassandra has very lightweight transactions.

What he lists as his transaction use case (update if unmodified) I do right now with Cassandra and an IF clause and a timestamp...

i.e. update foo set x = y AND last_modified_timestamp = 456789 IF last_modified_timestamp = 12345

see http://www.datastax.com/dev/blog/lightweight-transactions-in...

Even today I'm not sure I'd recommend Cassandra for his use case. A stated requirement he needed was:

>Support asking for N number of Pins in a board in a deterministic order (such as reverse creation time or user specified ordering). Same for Pinner to likes, Pinner to Pins, etc.

This can be a pain to model in Cassandra. It would require denormalizing for every key you wanted to order on, and it generally makes updating data a pain.

I don't think that the schema here allows you to do this kind of query even in MySQL. They are throwing everything in that blob, which makes it pretty difficult to sort on anything that's in the blob. So, I think they would have to do what you are describing, denormalize everything... or get all the data and do the sort in the app. Which is precisely what would be required with Cassandra or another NoSQL system. Am I missing anything that would invalidate what I've just said?

If you only need 1 ordering, not too hard in cassandra. If you need a few different orderings, you would need a side table - which seems to be what you need here with mysql.

While that would have made scaling easier they would then run into a different set of massive problems because Cassandra isn't a relational database.

But neither is a sharded database. You can't do fancy joins and groups across 400 shards. (Well you can, but you are writing your own code to split up requests and stick it back together.. which is the same thing you would need to do in Cassandra).

> 'We had several NoSQL technologies, all of which eventually broke catastrophically'

They also use HBase for a lot of their non relational data.

Reply on news.ycombinator.com