Hacker News Reader

Comment by sks | Hacker News Reader

Providing API for building applications that have transactions and help with idempotent producing is really exciting. This will help lower a lot of pains associated with building stream processing systems. Doing it with very little performance overhead is amazing.

I do feel that calling it "Exactly-once delivery with kafka" is slightly misleading as this requires the applications to be written in a certain way. The title makes is sound too general and borders on claiming something that is close to impossible. I dont want to be too critical here as the author was very honest with what this means in the blog post. Regardless of the title this is an amazing feature.

View on HN

> calling it "Exactly-once delivery with kafka" is slightly misleading

Indeed. Idempotent operations is the mainstay of managing "at least once" message delivery. If you had a true "exactly once" system, idempotency would be unnecessary.

> Regardless of the title this is an amazing feature.

I feel it's a good iterative step forward, but nothing near the "revolutionary" that's being boasted.

Not "close" to impossible -- actually impossible.

The reasons this is "currently" impossible are: - We cannot build communication infrastructures that are 100% reliable - We build apps that actually crash.

I would not venture as far as to say this is impossible. If one day we are able to leverage quantum entanglement phenomenon in telecommunications network this would certainly be very much possible.

The machine that reads the quantum entagled particles will still occasionally break or have coding errors.

I don't really know a thing about that sort of network, but if you can assume a reliable network, the machine reading the packets going down shouldn't matter. The sender won't get an "ACK", will know the other machine is at fault and not the network, and can resend.

Given a "perfect network" I think exactly-once-delivery is possible, right?

What if the sender fails before it can fully process the ACK? When it comes online again it won't know that the last message sent was delivered.

Transactions.

You've been downvoted, but the trick would be to convert asynchronous messaging to synchronous processing that proceeds at a predictable tick. That would make it possible to bypass the FLP result. I'm willing to imagine quantum entanglement could pull this off if we were able to entangle actual transistors. Dunno how plausible that is, but points for creativity if nothing else.

> We build apps that actually crash.

And disks that fail and memory that bitflips and communication methods that are susceptible to random noise and machines dependent on failable power sources.

Also known as the Two Generals Problem

Reply on news.ycombinator.com