Comment by kiwicopple | Hacker News Reader

Comment by kiwicopple | Hacker News Reader

hey hn, supabase ceo here

this one might be more in of a “Show HN” because it’s a pre-release - something that you might want to try yourself or contribute to. The GitHub repo is here: https://github.com/supabase/wrappers

For context, Postgres has Foreign Data Wrappers (FDW) [0]. These allow you to connect one Postgres database to another, and then query the secondary database directly in the first.

    CREATE FOREIGN TABLE other_database_table ( 
      id integer, 
      title text 
    )
    SERVER other_database;

The data does not need to be “moved” to the primary database - it stays in the secondary and is fetched when you run a select query:

    select * from other_database_table;

Our release today is a framework which extends this functionality to other databases/systems. If you’re familiar with Multicorn[1] or Steampipe[2], then it’s very similar. The framework is written in Rust, using the excellent pgx[3].

We have developed FDWs for Stripe, Firebase, BigQuery, Clickhouse, and Airtable - all in various pre-release states. We'll focus on the systems we’re using internally while we stabalize the framework.

There’s a lot in the blog post into our goals for this release. It’s early, but one of the things I’m most excited about.

[0] Postgres FDW: https://www.postgresql.org/docs/current/sql-createforeigndat...

[1] Multicorn: https://multicorn.org/

[2] Steampipe: https://steampipe.io/

[2] pgx: https://github.com/tcdi/pgx

The blog post, repo and your comment still leave me a little confused about what Supabase's wrappers project is, the intent and its current state.

So this "wrappers" project is just a framework that simplifies the development and manages the curation and distribution of Postgres Foreign Data Wrappers? At the end of the day, are these just FDWs that run on top of any Postgres instance? Do I have to run Supabase to take advantage of the FDWs created through this framework?

Additionally one of your examples is Snowflake, but I don't see a matching SF FDW in your repo? Is there a generic JDBC/SQL/etc FDW for any SQL based database like Snowflake, Oracle, etc? Or is this just to create a spark in someone's mind (guilty!) about what is possible and yet to be implemented?

Despite my confusion, this sounds like a very exciting project to follow.

| At the end of the day, are these just FDWs that run on top of any Postgres instance?

Yes, the FDWs developed by Wrappers can be used on any Postgres db.

| Do I have to run Supabase to take advantage of the FDWs created through this framework?

No, you don't need to. Those are just normal FDWs can be used on any Postgres db.

| Is there a generic JDBC/SQL/etc FDW for any SQL based database like Snowflake, Oracle, etc?

The Snowflake FDW isn't developed yet, the example is just for concept demo. It has ClickHouse and BigQuery FDWs although they are not JDBC based. Generic JDBC FDW will need JVM embedded so it is not in plan at this moment.

Disclaimer: I am Supabase developer.

Thanks! Very awesome and exciting stuff!

I looked into using a postgres FDW years ago when I was trying to come up with an easy way to ELT data out of Mongo into a DB. Effectively using Postgres as the ETL tool. Understanding where to find FDWs, who maintains them, etc... was overwhelming. I can see the merits of this framework that you're building, and def going to keep an eye on it.

Also always excited to hear about Rust being used.

Full documentation is here: https://supabase.github.io/wrappers/

Reply on news.ycombinator.com