Hacker News Reader

Comment by sorenbs | Hacker News Reader

This is a really exciting project! I recently learned about https://github.com/vmware/database-stream-processor which builds on a new theoretical foundation and claims to be 9x faster than Flink. It is also written in Rust, and there is a compiler from SQL to Rust executables. Can you comment on the differences?

View on HN

I'm a developer of DBSP. Our repo now lives here: https://github.com/feldera/dbsp/. And here is some more benchmarking data vs Flink and Beam: https://github.com/feldera/dbsp/tree/main/benchmark.

See feldera.com for more info.

Thank you!

Really happy to see all the activity at the new repo. For a while there I thought the project was dying...

As an aside - we are evaluating different streaming engines to power data projections for Prisma.io. Excited to see support for Debezium coming in v0.4.0.

Would be interesting to somehow make Arroyo run the Nexmark benchmark so we can clearly compare to Flink and DBSP: https://liveandletlearn.net/post/vmware-take-3-experience-wi...

Hi there! We actually already have a built-in Nexmark source. It's pretty useful for developing new capabilities, and available as a source out of the box.

Just read through the DBSP docs and it looks like it is working in a similar space. The biggest differences in my mind are around distribution and reliability. Arroyo works across a cluster of machines and has built in fault tolerance, while for DBSP that's still just planned for the future.

(I'm the co-creator of Arroyo, for context)

Thank you!

I'll try to get something set up to compare performance of the two on the same machine.

That'd be great! We have versions of most of the nexmark queries and some internal benchmark vs Flink, we'd love to help if you're interested in benchmarking against more systems. Reach out at micah@arroyo.systems.

Reply on news.ycombinator.com