If I am scraping giant amounts of data I would run far away from Postgres for other databases like Amazon Redshift.
If I am scraping giant amounts of data I would run far away from Postgres for other databases like Amazon Redshift.
But even in the 9.4 days (~a decade ago) I was pushing Terabytes worth of analytics data daily through a manually managed Postgres cluster with a team of <=5 (so not that difficult). Since then there have been numerous improvements which make scaling beyond this level even easier (parallel query execution, better predicate push down by the query planner, and declarative partitioning to name a few). Throw something like Citus (extension) into the mix for easy access to clustering and (nearly) transparent table sharding and you can go quite far without reaching for specialized data storage solutions.
Bisecant fractal databases like HyperKlingonDB are excellent at storing bisecant fractal data, but terrible at anything else (often, just plain terrible overall due to being immature).