Hacker News Reader

Comment by twic | Hacker News Reader

Anyone looked at DuckDB [1]? Like SQLite, it's an embedded relational database with a C API. In fact, it has a wrapper which presents the same API as SQLite [2]. But it's optimised for analytical rather than transactional work - a few huge queries rather than lots of little reads and writes.

[1] https://duckdb.org/

[2] https://github.com/cwida/duckdb/tree/master/tools/sqlite3_ap...

View on HN

DuckDB offers a SQLite compatible interface; we use it extensively ourselves as it allows us to use the SQLite shell directly.

The internal design of DuckDB, however, is different and not directly related to SQLite. As you mentioned, it is entirely optimized for analytical workloads whereas SQLite is optimized for point-queries and point-updates. The main things we share is that we also have a single-file storage format and offer an amalgamation (i.e. single-source file compilation, duckdb.cpp and duckdb.hpp).

So I guess I can use SQLite connectors or GUI to play with it? I was just looking for this.

Not a dev so I'm a bit dumb with this stuff, but I was definitely looking for pivoting stuff and such, which requires a lot of extra steps with sqlite.

For analytical workloads, DuckDB may be better than SQLite, but it's just incomparably slower than e.g. ClickHouse (which is admittedly not embedded).

The main reason for that is that DuckDB is currently single-threaded, we are currently actively working on parallelism for DuckDB which will resolve this :)

I think there are also tons of other optimizations currently missing in DuckDB, vectorization probably the most important.

Very true, DuckDB is very young and we have so far focused on correctness over performance. As a result we are still missing several optimizations that more mature systems do have, for example, proper cardinality estimation. We are working on adding those as well.

Reply on news.ycombinator.com