Hacker News Reader

How does this compare to Apache beam?

An Apache Beam Runner is already implemented in Jet: https://beam.apache.org/documentation/runners/jet/

Beam is just an API layer with different backing implementations. But you don't typically use Beam to work with Jet, instead you use its own Pipeline API which is mostly like Java Streams. Jet will also soon get an SQL API.

Very cool! Is it possible to mix apis in a single project with Jet Beam Runner? This would make it easier to port Beam projects to Jet, as the migration could be progressive.

Do you mean for a single job/pipeline? This wouldn't be possible at the moment. Our current focus has shifted from Beam a little bit - as we found out the beam threading model didn't play nicely at all with Jet's green threads (there is no way to distinguish between blocking and non-blocking calls).

Reply on news.ycombinator.com