Noria: Dynamic, partially-stateful data-flow for high-perf web applications | Hacker News Reader

Noria: Dynamic, partially-stateful data-flow for high-perf web applications

micahlerner.com

61 pointsmlerner10 comments

The author of Noria, Jon Gjengset, gave a great talk about the system: https://www.youtube.com/watch?v=s19G6n0UjsM

The speeds achieved are impressive but you do not get consistency.

+1 for anything that schematizes caching semantics

I've gotten this wrong so many times, and because it only bites you in read-after-write scenarios, it can be difficult to catch with types + tests

feels like there are a bunch of patterns in client-server protocols that should be 'as declarative as the schema', but aren't currently. caching, sync, field transformation, batching rules, multi-RPC flows like oauth

I could see this being very useful. The standard cache + db architecture has some sharp edges that end up biting people all of the time e.g. the thundering herd problem.

Not only that but your database and app are working under the assumption of Read Committed or better and now by introducing a cache that is out of band from your database transactions you’re introducing a whole slew of consistency bugs (that often go undetected).

I may be missing the point here, but isn't it common to address this through a combination of timestamped keys[1] and interleaving database and redis transactions (start redis transaction, start db transaction, update db and add cache operations to current trx, and finally commit db trx and if it succeeds commit the redis trx, otherwise rollback the redis trx.)

[1] https://blogcabin.37signals.com/posts/3113-how-key-based-cac...

Is the interleaving of redis and db transactions atomic?

Not technically, but you are guaranteed that if the db transaction fails, we don't end up with invalid data in redis.

The other side is not true though, but that will just cause a cache miss if the keys are timestamped.

Noria is great research. Though, for anyone wanting to play with it for new projects, be aware that Jon (the author) has said that it's basically research-quality software, and the point was more to be a proof of value rather than to build a new production database.

Where is the link to the code and installation steps?

Some googling turned up https://github.com/mit-pdos/noria.

Reply on news.ycombinator.com