Hacker News Reader

HNHacker News

Top New Best Ask Show Jobs

Comment by skybrian | Hacker News Reader

Full thread

skybrian·

How does it compare to Apache Kafka? (I haven't used either.)

View on HN

Kafka is a high performance, persistent message queue.

Storm is distributed stream processing.

The two work together very well. Often, a server will publish messages to Kafka (or another queue), which are then read and processed by Storm.

Interesting question and response. I haven't used either but read about both systems independently. A bit more googling and I found:

http://www.slideshare.net/gschmutz/kafka-andstromeventproces...

Slides 42 and 43 describe an architecture with all three ... hadoop, storm and kafka. Seems like a beast and I don't have applications but pretty cool combo of technologies.

http://manning.com/marz/ - this is often the trifecta used in the "lambda architecture", coined by nathan marz. This is also the system they built @ backtype to do their real-time analytics system.

Also worth checking out http://radar.oreilly.com/2014/07/questioning-the-lambda-arch... from Jay Kreps, who's part of the Kafka braintrust.

Complementary technologies, often used together. In fact, in Storm 0.9.x, Storm actually bundles a JVM implementation of a Storm Spout that plugs directly into Kafka:

https://github.com/apache/storm/tree/master/external/storm-k...

In no way -- apples vs. oranges. Kafka is queue, Storm is processing engine. A better question how it's compared to other processing engines like Spark Streaming, Samza, etc. But just Google it :)

Reply on news.ycombinator.com