Hacker News Reader

Comment by Kydlaw | Hacker News Reader

Happy to see activity around Polars. This has been my go-to library for data processing due to the enhanced ergonomics compared to Pandas and SQL.

But they were a bit quiet lately, and I started looking more and more into DuckDB recently… until the recent acquisition of DuckLab by AWS

View on HN

I've been using clickhouse-local for quite some time, instead of DuckDB. There is also chDB.

After using pandas for 10 years, I favor SQL now, for some reason.

Maybe because it's like a swiss army knife for data work, regardless of whether you need it for OLTP or OLAP workloads. Having different SQL dialects is a bit annoying, but the base is the same more or less, so switching doesn't come at too big of a cost.

I haven't tried chDB yet, but I heard about it. Thank you for reminding me of that option.

I use SQL in data pipelines and processing that is going to require interoperability.

But for data exploration, I usually prefer Polars (imo it is easier to work with text, semi-structured data, etc.)

As a huge duckdb fan, I'd love to see chDB to get proper windows support - that would make it real competition (having WASM coverage is already a big step) which would be good for the space as a whole.

Interesting, why do you typically prefer clickhouse local to duckdb?

Because we use regular CH in production. I'm pretty familiar with it and I can always run large queries on a production server with 1TB of RAM.

Regular CH also support external data sources, so I can read 500GB of JSON from S3 and group by it on production server very fast and in memory.

Reply on news.ycombinator.com