Hacker News Reader

Comment by mneil | Hacker News Reader

You could look at tuning hdfs or Cassandra to be strongly consistent. By default though they both use eventual consistency for replication. The difference is you can set the required number of nodes to accept a write as a success. What I can't say for sure is if this is a good idea.

View on HN

Hi mneil,

Thanks for the suggestion,

a.Cassandra I don't think Cassandra is good option in this as well since it's by natural is AP database (designed for high availability)

b. HDFS I wonder for keeping the strong consistency I should use other database on top of hadoop maybe like bigtable/Hbase since they are designed for CP database. Maybe you know the data consistency model in HBase or Bigtable?

c. Maybe GoogleCloud Spanner or CockcroachDB? I just read from this about possiblity of NoSQL to be ACID compliant, but I'm not sure about this option is suitable for big data or not for long term since our data growth is super fast. Did you ever know about this is being used in big data use case?

You can make Cassandra cp by making writes persist to 100% of nodes. Obviously you lose the availability.

Mongo, for example, can be acid but you lose write throughput again. Postgres, I believe, has better throughput for most workloads over mongo and is acid compliant. I do not have any experience with spanner our cockroach so I can't really comment there. I don't believe hbase offers strong consistency either.

Maybe postrges is a good choice? I didn't mention it because the "data lake" term made me immediately think hdfs and I wouldn't call postgres a lake.

Reply on news.ycombinator.com