Have you looked into the semantics of their like StreamDB stuff ? Pretty interesting
Also there is this that was built on top of it. Sounds somewhat similar goal as yours but I could be off. Need to explore more https://ursula.tonbo.io/
Have you looked into the semantics of their like StreamDB stuff ? Pretty interesting
Also there is this that was built on top of it. Sounds somewhat similar goal as yours but I could be off. Need to explore more https://ursula.tonbo.io/
I did, in fact, initially write PicoMQ with OpenRaft, similar to Ursala, but I really wanted the operational complexity to be minimal and the nodes to be stateless (at least for most use cases), Pico uses SQL database as a metadata command log, inspired by RisingWave.
But Ursala would, without a doubt, have better durability ACK latency, as it wouldn’t have to wait for an ACK from S3. That said, I do plan on extending disk/EBS-staged WAL to hit similar low-latency durability ACK numbers. But again, most use cases don’t need single-digit-millisecond latency for ACKs.
Unfortunate timing and self hosting isn't complex or anything but still as you said, durable streams as a project is most likely going to be abandoned.
If you'd like to consider using PicoMQ, I have examples to deploy on Fly.io and AWS. ursula.tonbo.io is also a good option to try.