What makes Apache Kafka fast (2020)

> "For those unfamiliar with the term, ‘real-time’ does not mean ‘fast’, it means ‘predictable’."

So, if a system is predictable up to an upper bound of one year, is that still real-time? My understanding of real-time was pretty much as quickly as something happens with hard constraints on latency. I think "fast" is implied when we say real-time.

TLDR: They use many good industry practices:

- Append-only log with sequential IO

- Record batching on client and broker side (works well with sequential IO)

- Batch compression (Json and other text formats could be compressed very well)

- Unflushed buffered writes

- Zero-copy approach for working with data

In addition to that the architecture of the Kafka also plays a major role so that the clients don't require much overhead. The code is written to not to put much pressure on GC and a number of other more specific optimizations was implemented on both client and server sides.

The most interesting part for me was how they achieve zero-copy when working with sockets. That whole section is worth reading and they could have made another blog post about only that.

Edit: formatting

Can't read wihout signing up.