Hacker News Reader

Comment by candiddevmike | Hacker News Reader

I thought Cassandra was bad at storing big files?

Correct, that’s what object stores are for. However, metadata on said files, is probably very handy to have in a database.

I’m quite sure not all this Cassandra capacity is just file/photo metadata storage either.

you can easily chunk big files.

It’s a horrible use of the tech. It’s like storing jpgs in postgres. You can do it, it’s just a really bad idea.

it depends, storing files in cassandra and jpgs in postgress allows you to not add another software with bugs and maintenance burden.

If it works - everything is fine.

It may work, until you start to perform maintenance things, like repair, or new node bootstrap, etc. Then it may fail with high probability

Depends heavily on the use case. I know multiple video related companies that chunked data for replays into 1s and 500ms segments.

Having many keys, means that you can preform thousands of asynchronous requests for small pieces of data and then piece it together on the client side.

Super low latency is just something else to optimize for.

But then you need to push these segments into partitions, and big partitions are really bad, especially for old versions of Cassandra… Although I met customers with partitions of size of 100Gbs…

chunks will be evenly distributed between many partitions, no need to store chunks in one partition.

As another commenter said, 500ms clip is super tiny. Each key is a a separate partition.

So each segment is a separate partition.

Reply on news.ycombinator.com