When I tried to use DuckDB on the same dataset as ClickHouse, it simply did not work due to OOM: https://github.com/duckdb/duckdb/issues/3969
I also told them about our experience of using various memory allocators, and why you should never use the GLibC's malloc.
This issue was fixed.
I tried to play with "serverless ELT" (the other kind of serverless) where I would define AWS Lambda that turns incoming CSV files into Parquet for archiving and querying.
It seems that in DuckDB, the amount of memory it needs to do that is always at least a few gigs, and the memory goes up proportionally with the file size (which I think is due to data type inference?), which is both expensive and annoying because you are either overpaying or you need to go and increase a size of your Lambda when things start crashing.
I wonder if ClickHouse-local or some other tool can do that with constant memory, no matter the file size. I know Spark can, but Spark is kind of pain to work with.
(Yes, I do realize this is a bizarre use case.)
I did not catch you address correctness?
FWIW I have some experience with Clickhouse as I ran product at Firebolt and played a critical role in being more transparent about their foundations and giving credit where it is due. However, I do have some first hand experience with Clickhouse, which was jarring, considering my previous experience was with BigQuery.