EventStore: Open-Source, Functional Database with Complex Event Processing in JS
github.com
github.com
What I've realised recently is, this is basically event sourcing. Replace actions with events, selectors with projections, and memoization with snapshots. So this submission is certainly timely and of interest to me. Thanks!
You can go even further and use the exact same "events" or "actions" for event sourcing on the server side too. You just need to save the list of actions to the server. This lets you do realtime sync and multiplayer really easily!
It makes Implementing CQRS in a single service really easy. GETs load the event log, replay and display the results while POSTs send a command that gets translated into an event. If it is valid it gets appended to the log. This approach is storage agnostic: It doesn’t matter how the log ultimately gets stored
That's another nice way of doing things that way, better diagnostics eh?
A list of all past states avoids the determinism problem, but the consequence of each action is only implicit in the difference between two states.
For event sourcing, "the state" is the list of events, and what you're calling "the state" is just another selector/projection.
Got any examples of this?
Also, how do you feel it's better than a typical Redux app setup?
As it moves all the focus from reducers to selectors, and has quite different requirements for how to do caching there, it’s led me to work on a custom selector library with quite different API from reselect/re-reselect. I’d like to release it when my experiments start to feel... less experimental.
But y'all must be storing this `(point-in-time-state, action-that-produced-state)` somewhere, because the Redux Chrome plugin displays this data. Was I wrong in thinking the Chrome plugin was just visualizing data already stored by Redux?
function createStore(reducer) {
var state
var listeners = []
function getState() { return state }
function dispatch(action) {
state = reducer(state, action)
listeners.forEach(listener => listener())
}
}
That logic has been split out into its own package, which you can see in the https://github.com/zalmoxisus/redux-devtools-instrument repo.
Very vaguely related side note: earlier this year, I spent a couple days to add an "action stack trace" tab to the Redux DevTools Extension. When you click on an action, in addition to seeing the action contents, state tree, and diff, you can now also see an actual formatted stack trace that shows you exactly where the action was dispatched from (including the actual code if you've got sourcemaps enabled). Sadly, the extension maintainer has been MIA recently, so we're considering forking it into the Redux org. Until then, you can download my custom build of the extension here: https://github.com/zalmoxisus/redux-devtools-extension/issue...
edit: Here's the link where it's implemented in redux-devtools if anyone else is curious https://github.com/reduxjs/redux-devtools/blob/master/src/cr...
https://medium.com/@jamestthompson3/beyond-counters-using-re...
https://medium.freecodecamp.org/how-to-build-a-github-search...
https://twitter.com/gabrielvergnaud/status/10573757954155233...
[1] maybe this talk? https://www.youtube.com/watch?v=JHGkaShoyNs
Otherwise, most event sourcing uses different "streams" of events for different application functions, so you can shard by stream in whatever way works for you.
The easiest way would be to cast uuid as a 64bit unsigned int, then mod by the number of shards. If the number of shards is dynamic, then use consistent hashing.
You make an architecture that optimizes for future features at the expense of a coherent flow for the current features that you know you have.
it's like if you made pub-sub the core architecture for your system. very actor-system like, but also probably harder to understand the runtime behavior - control flow.
I’ll admit it’s probably harder to debug than a monolith, but that’s true of any distributed system.
So another huge amount of resources wasted.
Is there a particular reason this is suddenly exciting again now?
I think the idea is that it's something simpler than a database. It's more like an append-only file that has on-insert triggers/durable running queries. (Sort of like https://www.pipelinedb.com/ does, or like blockchain nodes do.)
Or, you could think of it as message queue like Kafka, with permanent durability of all "messages", and a single fixed subscriber bolted onto the queue server, where that subscriber is exposed through the queue server's API, allowing users to reprogram it arbitrarily.
Also this still isn't functional. Is adding records to a set not mutation?
Edit: Is a backup mechanism implemented? It's for persistent data, right?
Event-sourcing can be a really useful tool in many domains, but especially where having a state-of-the-art audit log is helpful.
The key design requirement here is to deeply understand the reads your application will need to perform.
The downsides are you have to push the data somewhere else to digest and report on it. Relations are tough to model, too, as events that happen to more than one aggregate essentially have to repeat the data in each stream in the form appropriate to that domain entity. (You can often model this as one event causing another. Projections work less well for this circumstance.)
This can make it hard to draw data from the underlying store. Instead you must "hydrate" it into objects in memory to ask questions of it, though projections can help. EventStore is definitely not the right choice if you need ad hoc query capability. (I used EventStore in production for 2 years with F#.)
I'd classify this as an exotic data store. Use it if you have a really strong need for event-sourcing. Me... I'd likely choose Datomic instead.
A new realization of an existing concept can be notable.
There is however benefit to modelling a system in that way and not just an internal transaction log. I have done many talks on the subject and you should be able to pull one up on youtube etc fairly quickly (its a bit much to go through in a comment).