Question for OP: I'd bet Flink's Statefuns comes in Restate's story. Could you please comment on this? Maybe Statefuns we're sort of a plugin, and you guys wanted to rebase to the core of a distributed function?
Yes, Flink Stateful Functions were a first experiment to build a system for the use cases we have here. Specifically in Virtual Objects you can see that legacy.
With Stateful Functions, we quickly realized that we needed something built for transactions, while Flink is built for analytics. That manifests in many ways, maybe most obviously in the latency: Transactional durability takes seconds in Flink (checkpoint interval) and milliseconds in Restate.
Also, we could give Restate a very different dev ex, more compatible with modern app development. Flink comes from a data engineering side, very different set of integrations, tools, etc.
Flink doesn't persist intermediate state synchronously at all. It runs asynchronous global snapshots in the background, which avoid capturing in-flight messages, just store state, aligned through epoch markers (a synchronization step). On a failure, typically seconds of work need to be redone. That's fine, because it is for analytics, and that approach results in good throughput.
Restate won't start step 2 of a sequence before step 1's result is durable, so it needs to make sure that this durability is achieved quickly. It does frequent (batched) log appends, and each partition does that by itself without synchronizing with others. The result is faster (latency) because it is more fine grained and has less coordination, but it is also more work that is done.
> Stateful Functions (in Apache Flink): Our thoughts started a while back, and our early experiments created StateFun. These thoughts and ideas then grew to be much much more now, resulting in Restate. Of course, you can still recognize some of the StateFun roots in Restate.
The full post is at: https://restate.dev/blog/why-we-built-restate/