Hacker News Reader

Comment by junto | Hacker News Reader

You need to improve your telemetry to feel the benefits. I can trace all the way through multiple services easily on a simple detailed flame graph in our systems.

https://www.datadoghq.com/knowledge-center/distributed-traci...

Unless you have a single monolith, you’re going to face issues with versioning whether it’s event based or API based. In each case you can usually add new properties to a message, but you can’t remove properties or change their types. If you need that, create a new version.

The author does a lot of videos on the event sourcing topic. Event driven I get. It works well in several applications I’ve helped to build over the last 15 years. But event sourcing? I truly don’t get it. Yeah I get it’s nice in terms of auditing to see every change to an entity and who made it, or replay up to to change x on y date, but that really is a niche requirement.

View on HN

> I can trace all the way through multiple services easily on a simple detailed flame graph in our systems

I'm not sure what point is being made here. It's good that you can do that - but are you implying that that's not possible in an API-driven system?

There are many examples of event sourcing but perhaps the most classical one is the bank account.

It's not just about auditing, it's also about transactionality and atomicity.

If you want to withdraw $5 from your account, the traditional approach of locking, updating everything, unlocking (or in other words wrapping everything in a transaction) doesn't scale as well as the notion that you just record the transaction (event). Implementation-wise this withdrawal can involve, updating two accounts and updating the audit/account transaction logs. We also want this to scale since our bank has millions of customers all operating more or less concurrently. A distributed log (like Kafka) is easy to scale and easy to reason about. You just insert the transaction record and you have a distributed system that will scale and is easy to reason about.

Another driver/flavour for something like event sourcing is what some might call state-based or state-oriented programming. That is instead of modifying state directly you are synchronizing state via events. This lets you e.g. code state machines around those that can lead, again, to easier to reason about (and test) code.

Do you know for a fact that banks use event sourcing for transactions? I thought they were extremely eventually consistent

I'm pretty sure different banks use vastly different architectures. Some run nightly batch jobs on mainframes that are written in COBOL. But alas, I've not worked for any bank, this is just a commonly used example. I am willing to bet the transaction log, or ledger, is indeed a very common approach since that's also common in accounting.

EDIT: Also event sourcing would typically be eventually consistent. I imagine for some banking applications a stronger consistency guarantee might be required, e.g. to prevent you from withdrawing the $100 in your account multiple times.

I’m sceptical, because I can’t find any examples of banks using it in production, just lots of blog posts by consultants and companies selling event sourcing solutions.

I did work somewhere that used a Kafka stack in production. It wasn’t a compelling use case and they spent almost an entire year on infra and productionizing it. It left me extremely sceptical about anything “big data streams” related :)

Fair enough. But I've used event sourcing in production in two companies. One project was a large scale distributed object store and the other was network equipment (like switches) and network management. The banking example is a classical one but I can't tell you if it's actually used in production. What I do know is that accounting software follows the ledger approach which has a similar spirit, recording transactions, and my guess would be regardless of technology banks also are transaction/event based as their source of truth (even in a COBOL mainframe batch processing scenario).

As far as I know banks in my country use transactions and IBM mainframes, not distributed systems.

I once spoke with some engineers at Monzo and at the time (2018) financial transactions were handled via a combination of Kafka and Cassandra.

> Implementation-wise this withdrawal can involve, updating two accounts

Can't wait to read the incident report when one of the consumers successfully receives and reads the message but the other doesn't.

Depends what domain you work in. Auditability is a key/mandatory requirement in a lot of regulated industries.

There are of course other ways to do auditability.

Event Sourcing + Projections provide a nice way to build multiple models/views from the same dataset. This can provide a lot of simplification for client code.

A niche requirement? There are big accounting firms who organize payrolls, where the changes that you mentioned are an important part of their business.

There are also other companies, which do the typical snapshot and roll up to the current time, when they start the services, that need the data without having access to the database.

You don't need event sourcing to organize payrolls event "at scale".

> I can trace all the way through multiple services easily on a simple detailed flame graph in our systems.

That's not exactly an obscure feature exclusive to datadog. From the top of my head, both AWS and Azure support distributed tracing with dedicated support for visualization in their x-ray and application insights services.

I doubt GP was suggesting it is unique to DD.

Reply on news.ycombinator.com