Hacker News Reader

I'd recommend looking into Substrait[0] for anyone interested in this mid-/low-level database/query engine space. It tries to establish a vendor-agnostic set of relational operators (and all the things surrounding it) to make it easier for query engine implementations to share individual components.

[0]: https://substrait.io

View on HN

That's cool, thanks for sharing! Do you know how close they are to their example use cases [1]? So far I've only been able to find a tool for calcite SQL parsing [2] but not the portion connecting to Arrow C++ compute kernel yet.

[1]: https://substrait.io/#example-use-cases [2]: https://substrait.io/tools/producer_tools/

I'd check out the Slack, which is where I've seen a few projects integrating it coordinating.

I think DuckDB is one of the projects that has the best support for executing Substrait query plans. I think for most other projects there are some forks with substrait support (e.g. datafusion) but nothing merged to upstream yet.

I'm not sure if there are any systems where it is integrated and yields tangible benefits yet (though there is decent progress on common tooling, so that shouldn't be too far in the future).

Reply on news.ycombinator.com