Can anyone comment on how this compares to KSQL[1]?
Personally I'm not a big fan of KSQL, given that:
(1) KSQL is built on top of Kafka Stream. There are use cases that we don't think Kafka Stream is a good fit. Please see the explanations above
(2) Inventing yet another SQL dialect is a bad idea in practice. Not only it incurs additional learning curves, but more importantly you have few hopes winning in development velocity. Calcite has been used by Flink, Storm, Beam, Dremio, etc. The community is simply way bigger even compared to the total number of engineers in Confluent.
Again this is just my personal take it does not reflect the stands of Uber.
I haven't followed Spark in recent months, but what I recall was that the SQL DSL had some caveats at first, because certain things weren't yet implemented.
My anecdote has been that projects that implement SQL after a while, typically don't deliver the whole thing on initial release. I imagine it's often quite a lot of work.
The more projects that use Calcite, the more upstream contributions there would be ...
I think the space for streaming processing is still quickly evolving. Many features like stream-table joins, CTEs, streaming joins with late arrival data are unimplemented or do not even have clear semantics yet. It would be great to see a benchmarks like TPC-DS in the domain.