Hacker News Reader

"The Apache Flink project is a relative newcomer to the stream-processing space. Competing Open-Source platforms include Apache Spark, Apache Storm, and Twitter Heron."

Can someone explain why Apache are creating projects that compete with each other? Why not focus on one?

View on HN

Apache often houses existing projects; sometimes becoming a home for formerly-proprietary software that gets thrown over the wall. Remember Google Wave? That's Apache Wave now. Apache Storm started out as just Storm, open sourced after a Twitter acquisition. Twitter Heron was open sourced by Twitter. Flink is a fork of Stratosphere. Et cetera :) So: ASF is doing no such thing, they're just providing a framework for open source projects to function in.

When it comes to high volume stream processing there are various tradeoffs that result in incompatible design decisions. Thus it makes sense there are multiple stream processing projects.

Reply on news.ycombinator.com