984 karma · joined June 23, 2011
Also, is there any compliance that requires it to be in different Postgresql servers? I assume most companies just use some sort of isolation (tenant_id column or dedicated tenant database/table) so I wonder if this problem could better be solved as a proxy layer.
For the project I'm working on, the tech such as Apache Iceberg and embedded DuckDB enables querying Snowflake + BigQuery tables directly from your BI tool, without any compute cost: https://github.com/buremba/universql
Snowflake is very powerful at scale but for small data (< 100GB) I rather work locally and then deploy my models to Snowflake later on. Universql lets you keep using Snowflake clients with its proxy layer and executes the queries locally on DuckDB.
This tech only became possible with Iceberg as it keeps the data references in manifest files where you can intelligently cache the data locally, similar to Snowflake's warehouse cache.
<a href="bash:rm -rf /">Destroy me</a>