Few questions come to mind. Firstly, is Quickwit compatible with any S3 compatible object storage, such as Cloudflare's R2? Are there particular considerations to keep in mind for this kind of setup? Secondly, do you see Quicwit being used for analytics, such as tracking daily visits or analyzing user retention?
Your insights on these would be greatly appreciated.
> Secondly, do you see Quicwit being used for analytics, such as tracking daily visits or analyzing user retention?
Excellent question. Quickwit is very fast on ElasticSearch aggregations. We do not support cardinality-aggregation but it is scheduled for version 0.8.
Analyzing user retention and generally speaking running complex analytics, will not be possible any time soon. Maybe next year?
There is a great presentation from their recent conference [1]. I am asking this because I find it quite similar to what you are doing - decoupling storage from compute and utilising S3. I really appreciate if you could share your insights.
[1] starts at 30:45 https://m.youtube.com/watch?v=ZX7rA78BYS0
For OLTP, the equation is less obvious. I'd be scared to suffer from the cost associated with PUT/GET requests. (Pushing once every 30s or so equates to a few dollars per month.)
Since Scylla is based on an LSM-Tree (I think), I would have expected the talk to be about saving sstables on S3 but keeping the WAL on disk... But the slides seem to point to pushing the WAL to S3.