Hacker News Reader

Started using Spark 1.6 a few months ago. Excited for the Kafka Connector feature.

What exactly was the mention of Kafka referring to? Spark has had decent kafka integration for a while now.

I think it's support for Kafka Connect, new in Kafka 0.9: http://kafka.apache.org/090/documentation.html#connect

It doesn't have anything to do with Kafka Connect.

Matei was actually talking about the existing Spark Kafka direct stream implementation, which has been available since Spark 1.3

The video of the talk is available here: http://livestream.com/fourstream/sparksummiteast2016-tracka/...

Ah, thanks! I have unfortunately been too busy to watch the streams this week, and assumed it was Connect because I think Confluent is/was doing a talk with Kafka Connect and Spark at the summit.

Me too. Matei said that Spark 2.0 should be released (stable) late-April or early May.

But you can always download and build the source!

Reply on news.ycombinator.com