Hacker News Reader

Comment by manigandham | Hacker News Reader

Great options already exist:

MemSQL for blazing fast distributed full SQL database with cross-datacenter replication and in-memory rowstore + disk-based columnstore.

ScyllaDB for Cassandra rewritten in C++ for blazing fast dynamo-style multi-master multi-datacenter disk-based wide-column database.

I'll also throw in AMPS (by 60East) as the best messaging platform that supports innovative SQL and real-time state-of-the-world queries on it's message streams (instead of using that rabbitmq or kafka crap).

View on HN

https://github.com/Netflix/dynomite

Dynomite for production proven, limitless scale for Redis.

Provides ability to Redis for both in-memory and on-disk workloads.

Dynamo-inspired linearly scalable, shared nothing multi-master architecture. Pairs well with Cassandra.

Supports pluggable backends (ex. RocksDB).

Been in production 2+ years. One of the larger clusters handles over 3.6 million sustained ops/sec in production, every day.

Redis is only the api/driver, not the features, right ? Meaning it's just crud and not the many features/data-types that redis provides, correct ?

It's the full Redis API and protocol. See https://github.com/Netflix/dynomite/blob/dev/notes/redis.md for a list of all supported Redis features.

The benefit of Dynomite's support for the Redis API + protocol is a.) you can use any Redis client and b.) the same code can be used for standalone Redis on your laptop and on a distributed Dynomite cluster.

This is neat, but why use this over just using Cassandra-based DBs?

Also you were a speaker at Data Layer right?

Yes, I was a speaker at DataLayer.

There are two answers to your first question.

1. Dynomite pairs well with Cassandra. At scale, Cassandra is not a speed deamon when it comes to reads. Dynomite helps to improve Cassandra's read performance.

2. The use case for Dynomite as the primary database is for workloads that require high throughput and low latency. In other words, Dynomite delivers consistently lower latency at any scale.

Better to just use scylladb which natively handles it rather than run another layer of software.

Memsql + AMPS have "contact us" pricing unfortunately. Do you know any sharded queue (like amazon sqs,iron-mq) open-source ?

Some companies are closed source for a really good reason: it allows for Vertical Integration and Focused Execution. Bay Area companies spend millions of dollars worth of engineers salaries in projects that layer complexity upon complexity (and you need dozens of engineers to maintain them), just because they are in love with idea of open source. This is dangerous proposition, and it stifles innovation. There is an opportunity to dramatically simplify a lot of data pipelines. Check this out: http://docs.memsql.com/docs/pipelines-overview

I understand that but not everyone lives in SF or creates a microservice for each function they have hosted in an overpriced ec2-vm inside a subpar docker-container using json-serialization.

You can have (open-source or free) and develop without community (i think like chrome does that).

The question is: is it worth it ? Without vc-money-cash it's a little hard AND time-consuming.

Yes, those are both closed source but great companies that will work with you. Happy to put you in touch if you want. MemSQL also has a completely free community edition.

For open-source fast and clean messaging, I'd recommend NATS, they recently built NATS streaming that builds on top of the pub/sub to include kafka like persistence - http://nats.io/

Yes but memsql community doesn't have high availability (voltdb does the same thing).

I don't think nats can be used like I said by looking at the docs.

It has clustering: https://nats.io/documentation/server/gnatsd-cluster/

What are you looking to do? Sharding for what? Data size? Throughput?

Reply on news.ycombinator.com