Comment by eatonphil | Hacker News Reader

Comment by eatonphil | Hacker News Reader

Parent Full thread

Link to the paper [0]. Interesting stuff.

> This paper presents Amazon MemoryDB, a fast and durable inmemory storage cloud-based service. A core design behind MemoryDB is to decouple durability from the in-memory execution engine by leveraging an internal AWS transaction log service. In doing so, MemoryDB is able to separate consistency and durability concerns away from the engine allowing to independently scale performance and availability. To achieve that, a key challenge was ensuring strong consistency across all failure modes while maintaining the performance and full compatibility with Redis. MemoryDB solves this by intercepting the Redis replication stream, redirecting it to the transaction log, and converting it into synchronous replication. MemoryDB built a leadership mechanism atop the transaction log which enforces strong consistency. MemoryDB unlocks new capabilities for customers that do not want to trade consistency or performance while using Redis API, one of the most popular data stores of the past decade.

[0] https://assets.amazon.science/e0/1b/ba6c28034babbc1b18f54aa8...

I don’t get this part:

> MemoryDB solves this by intercepting the Redis replication stream, redirecting it to the transaction log, and converting it into synchronous replication

Replication is eventually consistent in Redis - is it saying that it’s intercepting the stream at the source and blocking the write from completing until replication completes? Cause intercepting it at the point it’s going out (which is what the word interception implies to me) wouldn’t get you strong consistency I would think.

from sec 3.2 in the paper

"Due to our choice of using passive replication, mutations are executed on a primary node before being committed into the trans- action log. If a commit fails, for example due to network isolation, the change must not be acknowledged and must not become visible. Other database engines use isolation mechanisms like Multi-Version Concurrency Control (MVCC) to achieve this, but Redis data struc- tures do not support this functionality, and it cannot be readily decoupled from the database engine itself. Instead, MemoryDB adds a layer of client blocking. After a client sends a mutation, the reply from the mutation operation is stored in a tracker until the transaction log acknowledges persistence and only then sent to the client. Meanwhile, the Redis workloop can process other operations. Non-mutating operations can be executed immediately but must consult the tracker to determine if their results must also be delayed until a particular log write completes. Hazards are detected at the key level. If the value or data-structure in a key has been modified by an operation which is not yet persisted, the responses to read operations on that key are delayed until all data in that response is persisted. Replica nodes do not require blocking as mutations are only visible once committed to three AZs"

> the reply from the mutation operation is stored in a tracker until the transaction log acknowledges persistence

What are the consistency and durability properties of the tracker datastore?

Are replies from tracker mutations stored in a tracker-tracker until the tracker-transaction-log acknowledges persistence?

Is it trackers all the way down?

Strong consistency at the expense of availability.

Maybe it could be done round-robin?

this sounds mostly like how MS implemented synchronous availability groups in sql server

this is kind of a strange design. in order to support this we build an external model of the database for all potentially conflicting state. wouldn't it be easier to make a redis lookalike that supported high isolation levels?

The external model is much simpler to do for Redis than a database with complex cross-key queries like an SQL database. Redis has scans but no queries or “real” transactions with rollback. To me it sounds like more work to implement MemoryDB and then additionally re-implement the Redis interior than to implement MemoryDB and use the off-the-shelf Redis.

I also think that the decoupled design is kind of elegant, it allows the logical implementation to be developed independently of the durability bits. It’s open-core but someone else is building the core.

Yes, it does seem like all that effort to wrap redis is wasted, and that it’d be easier to just create their own. It isn’t clear to me why this is better than creating their own. Does anyone know why they would go the route of trying to wrap something around redis rather than just replace it with something that has a redis compatible API?

They also likely have technology / libraries / expertise that make building the tracker straightforward.

It's smart because you have deep control of bugwards compatibility, and can swap parts of the stack later.

Reply on news.ycombinator.com