How does this compare to Postgres + parquet FDW? Is zero copy feasible in Postgres with FDWs?
DuckDB is often compared to SQLite. But a more apt comparison might be KDB+, a proprietary vector embedded database.
Postgres is a row store engine rather than a column store, so I believe there will need to be quite a lot of translation for Postgres to be able to process parquet data (DuckDB and parquet are both columnar). My hypothesis is that DuckDB would be significantly faster! However, feel free to benchmark things!
We saw speedups (20%+?), but wrong orders of magnitude of perf that we advocate DB vendors to aim for when we do visual analytics integrations. Arrow opens up saturating networks & PCI cards for DB<>GPU, so think going for 10-50GB/s.