10 karma · joined December 12, 2021
I have seen it used to mean WAL before, so I am taking this with a dose of skepticism.
I am wondering how portable is parquet format and how interchangeable it is now?
There is likes of comet and blaze that replace execution backend of spark with datafusion and then you have single process alternatives like sail trying to settle in "not so big data" category.
I am watching evolution of projects powered by datafusion and compatible with spark with keen eye. Early days but quite exciting.
I am quite curious about the plans for python dataframe like API for duckdb, and python ecosystem in general.
When we did a PoC, the operational aspect of clickhouse and performance was severely lacking as compared to druid. Clickhouse had bigger resources at its disposal than druid during this PoC.
If they could improve the operational aspect and introduce sensible defaults so that the users don't have to go through 10000 configuration to work with data in clickhouse, I am sure I will give it a go for some other usecase. It is simple on surface but devil is in the details. Druid is much simpler and sane at the scale I need to operate.