One of my fears. That unless we, as the PostgreSQL Community, do something about it, and embrace the efforts that AWS started doing and I hope will continue, it would effectively end up being a fork. And a fork with a collaboration on GitHub, which many would prefer to contribute to rather than current's Postgres contribution mechanism.
And if that's the case, I wonder if it could become the MariaDB, or the LibreOffice, of Postgres. I hope we will be able to do the best thing and have the contribution that Postgres is receiving integrated and merged into Postgres.
Personally, I think that the word "fork" has a lot of historical meaning in the world of Free and Open Source Software. To me, "fork" means that there is a "vote of no confidence" in the direction of the people who are making decisions about open source software. That clearly is not the case here. The PostgreSQL core team has a very good reputation of being collaborative, and thoughtful, in their work to make PostgreSQL the best database it can be for the community at large, and not build it to benefit any particular commercial interest.
Often software development happens on branches. These are experiments, not forks. Over time, good ideas are propagated, and grafted, between the branches. These are not "forks" to me.
I see this as a very open and collaborative approach. To announce such a big contribution to Postgres, and be it open sourced when published, is very welcome, and I believe one of the boosts, as I mentioned in the post, that Postgres needs.
I'm also convinced that Babelfish and PostgreSQL will be properly integrated, and this will be a great win. I consider it too a development branch more than a fork, which certainly has "no confidence" connotations. But there's still some worry that it would become one, one day, if there's no sufficient effort towards its integration --as the conversation that I wanted to spark here was almost non-existent up until recently.
This actually brings the topic of more explicitly branch development in Postgres, but it's a different story ;)
But the source code hasn't been published. As I said to you today on pgsq-hackers, that's why it hasn't been taken seriously - that's just how it works.
> But there's still some worry that it would become one, one day
Who are you attributing that concern to, exactly?
You're basically claiming that it will be a great opportunity. But you're also claiming that it might be an existential crisis. Do I have that right?
Have you considered the possibility that it might actually be neither of those two things?
And maybe that's just how it shouldn't. Technology moves fast, very fast. Markets do so. Users, you get it. Competing technologies, undeniably.
To delay a discussion for months just because the source is not published is in my opinion only reflecting on part of the opportunity/problem. I'm calling for an strategic discussion, which is higher level than a code-only, limited, partial analysis.
> Have you considered the possibility that it might actually be neither of those two things?
Sure thing, like anything in life, it may be neither. But what I know for sure is that a) we as the PostgreSQL Community need to mitigate risks and plan for them; b) leverage (or transform into!) opportunities as soon as we identify them.
There is practically no information about Babelfish available to analyze. You can currently request a preview of Babelfish if you have an AWS account, though I'm not aware of anyone having even that level of access. There is a single page placeholder on a Github.io that says "Babelfish for PostgreSQL will be available on Github in 2021".
And so I genuinely don't know what a strategic discussion really means here. How much can be decided about something based on a vague press release?
I'd also love to have more, architectural/design documents, the code itself, etc, and I'm sure they will be sooner than later released.
But that doesn't prevent us, as I said, to discuss the items I suggested above. They do not depend on the source code, at all.
And independently of this, there is already one thread on the -hackers mailing list that provides some information and proposes to introduce a technical improvement that not only would ease the Babelfish integration, but will also provide significant benefits for PostgreSQL itself, allowing for other protocols to come (including a potential v4 and an HTTP-based protocol): https://www.postgresql.org/message-id/CAGBW59d5SjLyJLt-jwNv%...
This isn't the sort of thing that is decided with blog posts, but personally I am glad to see there is interest in this project.
To indicate a non-collaborative fork, I usually use the term "hard-fork", which I know from the blockchain world, where it has a similar "vote of no confidence"/"we are starting our own project with new leadership" meaning.
When you look at something that has nothing to do with the Babelfish functionality but would absolutely help PostgreSQL, consider PHOT. PHOT (Partial HOT) is a great opportunity for PostgreSQL that Amazon is trying to give back. However, I wouldn't be surprised if the .Org community were to not accept the patch, the feature would end up in Babelfish. If that happens, we now have a viable fork which has 100% compatibility++.
AWS has enough scale to have its own gravity and will likely run development forward regardless of .org, it would be GREAT if .org could actively work to integrate where possible the open source options here.
I remember some early efforts of google to contribute wakelocks and other features to the linux kernel - so annoying when that was shot down and created a pretty significant split, and the commercial entity wasn't going to hold off on wakelocks because a subsystem maintainer didn't like it - so android starting going off into its own land.
Wakelocks also ended up proving a reasonable power management approach (despite complaints they didn't use existing linux mechanisms etc).
There was a big push ultimately to get wakelocks into mainline and drive some convergence.
This is true only to a degree: Adding new procedural languages is supported, but support for new query dialects (i.e. the main contact surface of most applications with an SQL RDBMS) is not currently a supported feature.
As mentioned in the psql-hackers thread from the article, currently the postgresql-specific parsed syntax tree is passed through many components, and to replace the syntax you'd have to alter and recompile the core application.
https://babelfish-for-postgresql.github.io/babelfish-for-pos...
Can the title of this post be changed to remove "AWS" please?
If successful, Babelfish will grow a community, and work harmoniously as an extension of PostgreSQL. And including "AWS" in the brand of the project may hamper the efforts to build a community around it (in addition to complications of using a brand / trademark like "AWS" in an open-source project).
The TL;DR is: "AWS" is not part of the branding of Babelfish for PostgreSQL, and that is intentional. The reasons for this aren't about obscuring the origins or initial sponsorship of the work, but are rather about establishing a separate identity.
[1] https://www.jwz.org/blog/2016/10/they-live-and-the-secret-hi...