A couple of months back, a QA review reported a bug - the log system didn’t work. Turns out it’s never worked. Nobody has ever done a rollback either.
After investigation I found nobody understood what it was doing or why. I thereby found the safest approach was to delete the ES implementation.
After that everything worked and everyone was happy.
We were modeling well dynamics. And to calculate interesting properties of the well we needed sensor data, the previous state of the well, and a model of the well built by a petroleum engineer.
Unfortunately the model of the well wasn't always up to date (pump was replaced, etc..), so we needed to be able to replay all of the sensor data into the well when a model was retroactively changed. Event sourcing was such an easy pick because our events were very simple (15 fields of data) and our replay requirement was a showstopper.
where inserted >= the_date
order by insertedif you use a timeseries databases like kdb+/q, TimescaleDB, InfluxDB, QuestDB, etc., you don't even need to order by the insertion timestamp.
Unlike RDBMS/SQL where rows/tuples are unordered, in timeseries databases they're usually ordered by the insertion time, more similar to dataframes than to SQL tables.
--
[1] Martin Kleppmann — Event Sourcing and Stream Processing at Scale
The entities/aggregates have clearly defined state machines and the commands/events that change their state are clearly documented and simple.
The system is easy to understand, the business people understand it because we used their words to describe it.
We were able to model it and workshop it with them by literally having them each take on the role of one of the aggregates and pass paper "commands" and "events" between each other.
CQRS/ES is actually the way the world works. If you can't imagine a bunch of bureaucrats from Brazil, all sitting in a large room, passing files and paper forms between each other, then you're not understanding the actual work.
- relational database systems (their journals and async replication in particular)
- git
Perhaps not so shit. Perhaps a bit practical. It's just a bad fit for some problems and architectologists like to say it's good for everything for obvious reasons.
And it's should be pretty obvious that architecture doesn't matter at all if it's implemented poorly.
eg. Git ALWAYS applies a commit in a consistent way (and where it doesn't it's a disaster)
eg. If you modify the transaction replay code from one version to the next of a DB, it creates a mess. (and why many databases won't replay transactions from previous versions, or even boot at all, see Postgres)
The problem with event sourcing is that the same bad practices that came from the CRUD system are recreated using events, which makes the system inherently worse because now the state of your system is unstable, instead of just the manner in which you got to the current state.
Imagine if everytime your rebooted your Postgresql cluster and it replayed Postgres 7 transaction logs, even though your running on Postgres 13, and generated an objectively different state, except, now your using CI and every commit from every developer will boot your system into a different state. Also, your database now takes 2 months to start as every transaction gets replayed.
Take rails db migrations for example, the best practice is, every 6 months or so, to just dump the schema from production, create one large initial "migration" and add migrations from there, because eventually they become desynced, and you can't cleanly replay your migrations into a schema that mirrors prod.
If you are extremely careful, and follow a bunch of best practices you can, but in the general case it costs less to just dump schemas.
Git doesn't really store file changes as events, rather, it's a long chain of state snapshots, in something like a persistent data structure. Sure, the history is all there, but so is the current state in its most efficient form.
DB transaction logs might be considered event sourcing by some definition, but their use is very different. It's purely a technical trick. As a consequence, logs are truncated as often as possible/reasonable, and you never rerun the transaction log from time zero. Very different from the event sourcing idea to keep events as long as possible.
Financial systems can really take advantage of it. Your post is hyperbole.
Event Sourcing with its current ecosystem and frameworks mainly add technical complexity with very little added clarity of business.
Generally I follow the Kafka approach of, essentially, shards that are each processed serially. So unrelated messages may be processed out of order, but related messages will be processed serially.
If you don't have a good enough key, or you need to join two streams, then you have to use CRDTs or equivalent. This can be hard, and you will have bugs, but since you retain the original events, you can always fix things up and recover the correct data, whereas when you get an SQL datastore into an inconsistent state you're generally SOL.
But if you have the luxury to do consistent CRUD with a relational/graph, ACID database, then a temporal data model might give you much more leverage.
Essentially you think of your records as bookkeeping entries and go from there. You retain the ability to do ad-hoc relational queries, while not having to build core database functionality yourself.
classic event sourcing require playing your events in order to calculate state. I think that's a complexity. Just do normal state computation but write it into a new timestamped row