Multigres: Vitess for Postgres | Hacker News Reader

Multigres: Vitess for Postgres

supabase.com

49 pointssougou17 comments

Hello HN, co-creator of Vitess here. The goal of Multigres is to build an open source adaptation of Vitess for Postgres.

In terms of project plan, we're still deciding. One option is to fork and retrofit. This will give us immedate results, but we'll be inherting quite a bit of legacy that we'll have to overcome later. The other option is to start with a clean slate and move pieces from the original Vitess code. This will ensure that Multigres has the Postgres DNA. We'll also avoid inheriting legacy features. Let us know your thoughts on this.

I'll try my best to answer any other questions you may have.

This is very interesting, thanks for offering to answer questions. I'm curious of two things:

* Are you also considering going the Postgres extension route, like Citus? It is after all the best attempt at sharding Postgres so far.

* If you are willing to share, why not doing this from inside Planetscale? I assume it was at least considered over the years.

It will be a proxy layer. Since Vitess has proven itself with this approach, I don't see a reason to try anything drastically different. As for Planetscale, I've posted my thoughts on X and LI :).

Interested to know how it compares with PgDog: https://pgdog.dev/

PgDog looks very similar to Vitess in terms of approach.

This is super interesting - will you be open to contributions here?

Of course, but not right away. We'd like to get the project to a decent stable state first. Then, we'll open it up.

Your unique advantage is you know Vitess super well. Your unique disadvantage is you know Vitess super well! Second system syndrome is real. Using as much Vitess as possible could help you guard against it.

Excited to follow your progress :)

It sounds like a great thing to work on! but for HN purposes, the thread needs more than just an announcement to have a substantive discussion.

Probably the best thing would be to wait until there's some kind of v0 and then post https://github.com/multigres/multigres. It's certainly on topic, there just needs to be some meat on the bones, or some bones.

One peice of tooling I've wanted for Vitess is for it to make recommendations about sharding keys based on the statistics it collects + query patterns. Seems like it's in the perfect location to collect the data for that and remove the mental overhead associated w sharding. If much of the implementation has to be rebuilt, is there a path for that in this implementation?

We absolutely need this!!! I vote yes :).

I am wondering at this point if Supabase will just release a complete DB based on Postgres, OrioleDB and Multigres along with many other extensions.

I am a little sad Plantscale didn't get this. But hopefully this will push Postgres ecosystem forward. In many ways both Postgres and MySQL is still so far behind Oracle and MSSQL.

I don’t know if this is a question for you or for the Supabase leadership team. But, what is the primary reason for building something “new” instead of using the tools already existing in the Postgres ecosystem like Citus or pgdog?

What are the gaps in those which you expect to solve with Vitess for Postgres?

The Vitess architecture was traditionally built in a database agnostic fashion. Most of its features should port smoothly over to Postgres.

The cool features that I can think of: A formal sharding scheme based on relational foundations, a fairly advanced query analyzer and routing engine capable of cross-shard functionality, HA and durability, abilities to reshard safely, seamless migrations, etc.

It's a pretty big list of capabilities.

how are you thinking the differences between mysql and postgres will impact multigres vs vitess? The ones the jump to mind are

- Postgres has transactional DDL vs auto-commit. Can that property be retained?

- Postgres leans into extensions much more that Mysql. Will types/indexes/etc introduced by e.g. pg_vector or postgis, have a path for support?

anything similar that will be a challenge?

You won't believe what a nightmare it was to work with transactionless DDLs in MySQL. Transactional DDL will be a dream come true for Vitess: we can throw away all the hacks we had to do for MySQL's sake.

I also see such a clean 2PC API. This was another huge mess on the MySQL side.

I am interested in the extension question too. What kind of challenge that will present.

Reply on news.ycombinator.com