Hacker News Reader

It would be very interesting to see a comparison of Pulsar or LogDevice on Twitter scale workloads.

Both Pulsar and LogDevice solve the problem of far-away-consumers with kind of segmenting the log and storing each segment in separate machine to reduce disk seeks and page cache pollution (when c1 reads head of the log and c2 reads tail).

So technically they are better than Kafka. But as the article mentions, they just used SSDs to get around this issue in Kafka. By looking at article the only valid reason to switch to Kafka seems to be KStream.

Pulsar has a much better storage system that scales independently so latency stays low regardless of consumer offset. It can also now tier to S3/cloud storage natively and it has many other features like supporting millions of topics and per-message acknowledgements.

Reply on news.ycombinator.com