Hacker News Reader

On the subject of foreign data wrappers, there's an extension for storing columnar data in Postgres which is implemented via this concept: https://github.com/citusdata/cstore_fdw

I haven't used it myself but it's pretty cool that it's out there.

View on HN

Splitgraph co-founder here. cstore_fdw is great (we even use it in Splitgraph to store data [0])! It's not as fast as purpose-built columnar stores like MonetDB. However, it plugs seamlessly into PostgreSQL and supports all types, even those added via extensions. Read performance for OLAP workloads like aggregations is better than PostgreSQL [1] and it has a much smaller IO load and disk footprint (you can get long runs of similar values in column-oriented storage, which compress better). As a nice bonus, you can simply swap cstore_fdw files in and out of the database without having to "load" them into PostgreSQL. We use this idea too to enable data sharing.

[0] https://www.splitgraph.com/docs/concepts/objects

[1] https://tech.marksblogg.com/billion-nyc-taxi-rides-postgresq...

Alternative storage formats like columnar storage are better served by Postgres’s new custom access methods (introduced in PG12 I believe).

Reply on news.ycombinator.com