Comment by chipdart | Hacker News Reader

Comment by chipdart | Hacker News Reader

Parent Full thread

> Because everyone insists on insanely distributed architectures, most people will never really see the point of Redis, which is that if it is running on the same machine as the application, it can respond in much less than a millisecond.

I don't think this is a realistic scenario at all.

If you need a KV store, you want it to be the fastest by keeping it in-memory, you want it to run on each node, and you don't care how much it cost to scale vertically, then you do not run a separate process on the same node. You just keep a plain old dictionary data structure. You do not waste cycles deserializing and serializing queries.

You only adopt something like Redis when you need far more than that, namely you need to have more than one process access shared data. That's at the core of all Redis usecases.

https://redis.io/docs/latest/develop/interact/search-and-que...

I develop and maintain multiple applications that use a worker pool, and are small enough to run on a single host. We used pg for the user sessions, which get read and written on every single page request. Some of our apps are Internet-facing, and web crawlers can create sessions that get read and written (recording recent pages) as they browse the site. We switched to a redis service on the same host as the app and saw 3 main benefits: faster session loading and saving, less disk activity on the Pg server (so all other queries run faster) and less writes to the Pg WAL, so our backups require drastically less GB per day of retention.

After the significant success of the first conversion, we've been working to convert all the rest of our apps.

And no, host language data structures aren't useful because they aren't in shared memory between all the worker processes, and even if we found a module that implemented them in shared memory, we like to be able to preserve the sessions across a host restart, and then we'd need a process to save the data structures to disk and load them back, and by the time we did that we'd have just reinvented redis.

This is the best response so far. Session churn creates lots of db activity but lots of it is of low business value. Better to offload to a separate process.

Also session data is often Blobs which db's don't process as efficiently as columnar data.

> then you do not run a separate process on the same node

You might if you want the KV to persist between app restarts (for warm starts.)

Reply on news.ycombinator.com