Very cool! Seems like a Rust version of something like Bento? [1] Have you done any benchmarking against similar stream processing tools?
Highly recommend checking RisingWave out if you have real time streaming transformation use cases. It’s open source too.
The benchmark was some high throughput low latency JSON transformations.
Benthos: https://github.com/redpanda-data/benthos
DataFusion: https://github.com/apache/datafusion
A contrast between Arroyo and systems like Benthos and from what I can tell ArkFlow, is that Arroyo is a "stateful" stream processing engine, which means that we can support things like windows, aggregates, and joins, with exactly-once semantics and fault tolerance, at the cost of significant additional complexity[1].
[0] https://www.synnada.ai/ [1] https://www.arroyo.dev/blog/stateful-stream-processing