Hacker News Reader

> Summary: This is a system which couples your application logic with your persistence layer (in the same OS process).

I don't think the paper is talking about the persistence layer. Rather it's focusing on the cache layer.

From the paper:

"Modern internet-scale services often rely on remote, in-memory, key-value (RInK) stores such as Redis and Memcached. These stores serve at least two purposes. First, they may provide a cache over a storage system to enable faster retrieval of persistent state. Second, they may store short-lived data, such as per-session state, that does not warrant persistence."

If you look at Fig.2 in the paper, the database (persistence) layer is always there. What the paper is advocating is to move from an out-of-process cache server like Redis to an in-process cache library like Java Caffeine

So for the canonical use case of session state in a load balanced server farm, is the session state being replicated across every single one of these in-process caches? Or, if a request is routed to a server without a copy of the session state, does the in-process cache have to go find a peer server with a copy of the session state?

This seems like an easy “solution” to show how it speeds up the happy path while not fully addressing the real-world tradeoffs.

Stateless application servers come and go without a care in the world. You can reason very simply about the cost and effects of bringing them up and down.

Add in a KV store with sharding, replication, a discovery protocol, a heartbeat protocol, a sync and recovery protocol, etc... and put it all in-process on every application server?

Have fun monitoring and debugging this.

I much prefer the 3-tier system of a local RAM cache, a Redis cache, and a persistence layer. As much state as possible is idempotent so you can cache it locally as well as in Redis. The load balancer can best-effort route back to the same app server, and your KV lookups automatically check local RAM first. But the central KV store handles replication and is easy to monitor and scale out if needed.

The paper assumes that you have a good amount of related context. I'd recommend at least reading the Slicer paper mentioned several times (https://www.usenix.org/system/files/conference/osdi16/osdi16...).

How's in-process issuing the same library much different than using a separate process? What's complicated there in monitoring/debugging that's different that's not otherwise?

I think that debugging with one less process is a good thing. And monitoring with one less process boundary to traverse is easier as well.

For me the most important part was in the abstract: autoscaling. It’s not just about going to zero but going to hundreds of thousands. I can remember my two most painful times with memcache:

1. A client scaled rapidly & we had repeated hits on the same key. We were saturating the NIC in the memcache server. We had to addd indirection for some data and introduced an local in-memory layer in front of memcache.

2. We had to extend the ring size to give memcache better CPU utilization. We had to teach all our code to handle a migration process (reads were fall through to the new ring and writes purged both). We couldn’t just turn off memcache because a total dump would overwhelm the database.

> I'm actually surprised there not more "database as a library"-projects out there.

For distributed systems, you tend to need quite a bit of configuration (how to find the peers, authentication, things like namespaces).

And then if the communication is TCP-based, you tend to keep connections open to avoid the handshake latency when you need to retrieve a value. Which sounds like something you typically don't want running inside a library, but rather as a separate process.

And then you've basically arrived at the memcached design.

So the crux of this paper is "if you have perfect auto-sharding as a service, it's better to cut out a middleman and have it shard stateful app servers, rather than just sharding a KV store layer." In many ways, this paper is an extension of the 2016 paper it references, which introduces their internal sharding-as-a-service, Slicer: https://www.usenix.org/system/files/conference/osdi16/osdi16...

Once you separate out Slicer, this result is pretty obvious in theory - in fact, in that Slicer paper they mention machine learning as a driving example, where it absolutely makes sense to colocate application logic with user-specific models cached in memory.

Honestly, this paper, with its one-paragraph implementation section, reads almost like clickbait - "let's say something controversial, release an un-reproducable paper that adds very little to the literature besides 'this is something cool you can do if you have a tool only Google has,' btw do you want to work for Google?" The Slicer paper itself has much more detail and IMO is a much better read.

SQLite (:memory:), lmdb, berkeleydb, memcached-over-unix-socket, APCu (php), EHCache (java)..

I'm sure there are others; I don't get what this paper presents that is new

Oh, sorry, I was imprecise: I meant "distributed database as a library". As in, you compile/bundle your application logic together with the persistence/replication/transaction logic and that is what you're running in a cluster.

Datomic is a distributed database written in Clojure, and is best described as "cqrs-as-a-service". It allows you to run a "peer" inside your local Java VM, which essentially does exactly this: it caches the parts your local vm needs, and if not it looks elsewhere.

https://docs.datomic.com/on-prem/peer-server.html

This is do-able with FoundationDB. You could distribute your application colocated with the database nodes and write a router to redirect requests to the machine which owns the data you're looking for. Then your application logic is computing against data on the same machine.

FoundationDB exposes the key-range to node mapping to you to make this router possible.

The BeamVM languages (Erlang, Elixir) seem to have this in the form of Mnesia, and libraries that are built on top of it, such as riak-core.

Ha! I was just about to ask about that.

‘I bet Elixir already has something like this’.

Riak uses Mnesia?

Not that I know, but it uses ETS to handle state, and the global process registry to store peer information, as far as I know.

Also, be aware that OP mentioned riak-core, which is a different thing from Riak. riak-core is a library to build dynamo-style distributed applications, which gives you virtual nodes, consistent hashing and data migration as building blocks.

Riak KV uses leveldb, but as @ergl mentioned, riak-core is essentially a hash-ring and replication (aka dynamo) layer.

EHCache has tons of capabilities and is bundled directly into your app('s war file)

Apache Geode, Hazelcast, how some people are using Kafka. I am just listing a few others, please don't consider this endorsement.

Apparently I have designed a few production "LInK" systems over the years (now that someone else has named it), with relatively sophisticated algorithms for load balancing and replication. These architectures are great for fast-twitch mixed workload use cases, rapid mutation rates concurrent with relatively light, low-latency analytic queries. Think operational real-time data models.

Conceptually, these are similar to the simple database kernels used as caches inside BI systems like Tableau but extended to support write workloads and scale-out. These may have a large amount of storage in them but only to increase the quantity of data each server can handle (i.e. swap space). Storage is not for durability, other systems handle that. The primary advantages of this architecture is it dramatically reduces data motion and wasteful computation, and you can often push complex data model logic to this layer since it implicitly contains more application context.

All "databases as a library" (e.g. SQLite) are designed around a much simpler set of constraints. The combination of high write rates and continuous background resharding/rebalancing across servers requires low-level architectural support that you can't graft onto a database kernel that was not explicitly designed for it. To start with, any query execution has to have sophisticated topology awareness in a system where the topology is constantly changing due to resharding/rebalancing and you can't rely on global consensus for everything. The library you are linking into your application contains all the clever algorithms for navigating a data cache that behaves like this, you don't want the app developer to have to reason about it.

The level of dynamic data model "fluidity" in the underlying organization and algorithms means that a mature design has little in common with traditional in-memory databases even though there are many superficial similarities. This isn't something you grind out over the weekend, there is a lot complex and non-obvious theory behind robust designs.

(I agree that the paper is not very clear about what they are talking about. This is my interpretation of the paper based on my recognition of the claims and architecture.)

Do you have any reference for your load balancing and replication algorithms?

On the topic of pruning logic more complex than put & ge in the application, to me that is kind of like want to add iterable functions to a language that only has for loops. That was enough for me to appreciate the paper. I was recently working on adding some query logic to an app that talks to Redis, and it definitely felt like more work than I cared for. At a minimum I would have liked to see that logic go into a more feature rich Redis client

https://www.usenix.org/system/files/conference/osdi16/osdi16...

> I'm actually surprised there not more "database as a library"-projects out there

Sqlite?