1egg0myegg0

This fall, DuckDB will support larger than memory indexes! I'm really excited about it. https://duckdb.org/2026/08/17/duckdb-20-highlights#7-storage...

Oh, then you might like these DuckDB community extensions that provide stats functions!

https://duckdb.org/community_extensions/extensions/stats_duc...

https://duckdb.org/community_extensions/extensions/stochasti...

Whoever gets that magical -2,147,483,648 is going to be really surprised that things keep working

We support spatial calculations in the UI, but not spatial visualizations just yet. Thanks for the feedback!

Sorry you hit that! This is actually already working on version 1.2.2. Could you install that version? That should get you going for the moment! We will dig into what you ran into.

It is already available in the local DuckDB UI! Let us know what you think!

-Customer software engineer at MotherDuck

IN list filter predicate pushdown is much improved in DuckDB 1.2, coming in about a week! I am not sure if it applies to Arrow yet or not. Disclaimer: I work at MotherDuck and DuckDB Labs

That is very surprising to hear! If you can reproduce it, could you please file a bug report on GitHub? That would be a huge help!!

Are we in the Dune timeline?

Hello! I would recommend trying out DuckDB's SQLite attach feature! You can read or write data, and even make schema changes, all with DuckDB's engine and syntax. The storage then uses SQLite, which is row oriented!

https://duckdb.org/docs/extensions/sqlite

(I work at MotherDuck and DuckDB Labs)

Extension downloads are validated using a signature check to prevent tampering!

(I work for DuckDB Labs and MotherDuck)

Howdy! I work at MotherDuck and DuckDB Labs (part time as a blogger). At MotherDuck, we have both client side and server side compute! So the initial reduction from PB/TB to GB/MB can happen server side, and the results can be sliced and diced at top speed in your browser!

Howdy! Thanks for your benchmarking!

Your blog does a great job contrasting the two use cases. I don't think too much has changed on your main use case, however here are a few ideas to test out!

DuckDB can read SQLite files now! So if you like DuckDB syntax or query optimization, but want to use the SQLite format / indexes, that may work well.

Since DuckDB is columnar (and compressed), it frequently needs to read a big chunk of rows (~100K) just to get 1 row out and decompressed. Mind trying to store your data uncompressed? Might help in your case! (PRAGMA force_compression='uncompressed')

Links: https://duckdb.org/docs/extensions/sqlite

We heard your feedback! Backward compatibility was just implemented! Version 0.9 is actually fully readable by 0.10. With version 1.0 coming in a few months, this will be readable for several years' worth of version updates.

There are many! https://www.thespruce.com/duck-species-list-385436

No worries! Thanks for reading the docs!

Howdy! The docs reflect the latest dev branch. For a specific release, you can pick it from the version dropdown in the top right!

You could definitely run a pivot with the SQLite attach! All of the DuckDB syntax works!

Great question! For analytical queries, DuckDB is ~100x faster than SQLite. It also has great connectivity with Parquet, Pandas, Arrow, Postgres, and more! Plus, the syntax follows Postgres closely, so you have strong data typing and true timestamps, etc.

All that said - SQLite is still faster for transactions. Now DuckDB can actually read and write from/to SQLite, so you can use SQLite for OLTP and DuckDB for OLAP.

https://duckdb.org/docs/sql/statements/attach

Sorry to hear that!

(I work on docs for the DuckDB Foundation)

Starting in this release, the DuckDB team invested significantly in adding memory safety throughout catalog operations. There is more on the roadmap, but I would expect this release and all following to have improved stability!

That said, at my primary company, we have used it in production for years now with great success!

DuckDB uses all of your cores! It just uses threads, not processes!

Let us know how you would improve our Modern Data Stack in a box!

Thank you for filing a bug, we will have a look at that I'm sure!

You can think of the attach operation as creating views in DuckDB with Postgres tables underneath! DuckDB will then query those Postgres rows (using the typical Postgres wire protocol, except in binary mode).

No data is persisted in DuckDB unless you do an insert statement with the result of the Postgres scan. DuckDB does process that data in a columnar fashion once it has been pulled into DuckDB memory though!

Does that help?

Yes, DuckDB supports writing lists and structs to Parquet! https://github.com/duckdb/duckdb/pull/2832

Does that help or do you have any other questions?

You can use the DuckDB FDW (linked in the blog post) for this! https://github.com/alitrack/duckdb_fdw

We have one of those! :-)

And yes it is fast!

https://github.com/duckdblabs/sqlite_scanner

Howdy! What are the rough edges when persisting to a server? What are the gaps relative to something like PouchDB?

DuckDB 0.5.0 "Pulchellus" released with persistent indexes, out-of-core hash join, new join order optimizer, and many other improvements!

DuckDB can actually read SQLite or Postgres directly! In the SQLite case, something like Litestream plus DuckDB could work really well!

Also, with Pyarrow's help, DuckDB can already do this with Delta tables!

https://github.com/duckdblabs/sqlite_scanner

https://github.com/duckdblabs/postgresscanner