I like DuckDB but I'm not sure what it wants to be. There's always new ways to use it and it's not easy to see what's the right one.
RDBMS have always been multi-user concurrent systems. DuckDB is a very fast local engine that has a multitude of use cases because it is a embeddable in other systems.
It’s like saying what does SQLite wanna be? It’s in your phones, your browser, your desktop apps, iot devices and people have extended it in different directions. The only difference here is that this is first party not third party. But to me it’s a very legible move.
I'm not knocking Quack or DuckDB but I'm starting to get a bit confused.
Is the concern that the duckdb messaging is now diluted by it having all these extra features? That you can’t sell it to friends as “this thing” like you can a one use tool like curl? I get that, but I also feel that duckdb is so much bigger than a “do one thing and do it well” tool.
It’s an engine that drives the modern data tool stack. Duckdb’s team has been prescient in that it has made many tasteful bets on what users want —- the ability to interop with pandas and polars, addition of geospatial, the plug-in infra. They’re all optional but when you neeed these things, they’re so useful. They’ve also clued me into what the broader data world is thinking about (I didn’t know about sketches and hilbert, but those are so useful in probailistic large scale queries and in geospatial queries). And they exist in larger database systems like Redshift too.
So far duckdb’s bets have been tasteful, and mostly ignorable if you don’t happen to use them.
The engine is often not the painful part anymore. The pain is the stuff around it: live DBs, S3 paths, Parquet files, credentials, repeatable runs, exports, validation, and the moment a one-off script quietly becomes infrastructure.
Quack makes the remote/server part cleaner, but the bigger trend seems to be DuckDB becoming the SQL layer inside tools, not necessarily the final user-facing tool.
I can't think of many use cases for this and Arrow Flight, other than moving data around.
This is not commercially a terrible idea. Why keep paying Snowflake for bog-standard SQL query workload when SF makes it easy to migrate to Iceberg & commodity engines like MotherDuck?
https://duckdb.org/quack/faq#what-is-the-relationship-betwee...
Of course, in the future MotherDuck can also support Quack, but this is not the only interesting use case for Quack.
Just making the point that DuckDB is disruptive technology & what it’s most likely to disrupt.
Ducklake handles the metadata and storage, but a local duckdb instance connected to it still has to do the compute itself. This lets you federate access to the compute.
Fun for me, I just finished a big streaming implementation doing essentially the same thing in Go-gRPC with arrow table record batches. It was fun though.