Hacker News Reader

Any consideration given to Scala or Clojure, or did raw performance (via Java 8) win over greater abstraction capabilities from the start?

IIRC, Twitter's Storm replacement, Heron, is written in C++ -- i.e. they didn't go with Spark/Scala, which, given that Twitter is probably the largest Scala shop in the world, speaks volumes about the volume of data these sytems need handle (read: Spark is far from slow).

View on HN

Hi, I'm the author of the blog post and wrote SuperChief. We chose Java 8 because it's the language we're most comfortable with on the data team and find it easiest to reason about. We also reused some of the existing bolt code we wrote for Storm to run the time series aggregations. It was already written in Java so we just had to make it thread safe. I suspect you could get similar performance with other jvm based languages.

I was interested in looking at Spark for this for a bit but coming from Storm we decided we wanted something stripped down that was more purpose built.

If I remember right, the team that wrote Heron was most comfortable with C++, and that was a major reason in choosing it.

Heron is half Java and half c++ and could have been written on top of finagle had a different team inside Twitter had implemented it

Reply on news.ycombinator.com