Should you use incremental repair in Cassandra?

thelastpickle.com

34 points·JensRantil··4 comments

Learning Cassandra was really fun, it’s very complicated behind the scenes but also incredibly powerful and the ideas behind it helped me learn a lot about what databases are really doing under the hood. Thinking about table shapes (which is effectively the output your application needs to consume in the right order) up front is very difficult but in return you can scale to almost as many writes as you need. Recommended.

Just curious... At what scale have you run Cassandra? We're running a cluster containing around 50 TB of data and we see massive issues around compaction and repair. While I agree Cassandra is impressive, it definitely comes with a lot of management overhead.

Yes, I've just played around with it, even if what I built was fairly complex and well tested. I assume you are running with a lot of extra servers - you need extra capacity for the operations you suggest (they move data around right) that might amount to a constant 20% over provisioning from normal use at the times these operations run, obviously also adding and removing servers adds overhead too. It's not like a normal database with provisioning at all.

I'd actually like a mode where Cassandra made itself artificially slower forcing you to over provision and then removed said slowness during operations that need the extra performance.

I help run clusters about 100 times this size. Things work remarkably wonderful.

Huge database systems require investment in setup and operation.

What is causing problems that need repair?

How much spare space do you keep per node for compactions?

What is your spec on a per Node basis?

Reply on news.ycombinator.com