Hacker News Reader

Comment by ahachete | Hacker News Reader

OP.

One of my fears. That unless we, as the PostgreSQL Community, do something about it, and embrace the efforts that AWS started doing and I hope will continue, it would effectively end up being a fork. And a fork with a collaboration on GitHub, which many would prefer to contribute to rather than current's Postgres contribution mechanism.

And if that's the case, I wonder if it could become the MariaDB, or the LibreOffice, of Postgres. I hope we will be able to do the best thing and have the contribution that Postgres is receiving integrated and merged into Postgres.

View on HN

Disclosure: I work for Amazon where I build AWS infrastructure, but these opinions are mine alone.

Personally, I think that the word "fork" has a lot of historical meaning in the world of Free and Open Source Software. To me, "fork" means that there is a "vote of no confidence" in the direction of the people who are making decisions about open source software. That clearly is not the case here. The PostgreSQL core team has a very good reputation of being collaborative, and thoughtful, in their work to make PostgreSQL the best database it can be for the community at large, and not build it to benefit any particular commercial interest.

Often software development happens on branches. These are experiments, not forks. Over time, good ideas are propagated, and grafted, between the branches. These are not "forks" to me.

Indeed.

I see this as a very open and collaborative approach. To announce such a big contribution to Postgres, and be it open sourced when published, is very welcome, and I believe one of the boosts, as I mentioned in the post, that Postgres needs.

I'm also convinced that Babelfish and PostgreSQL will be properly integrated, and this will be a great win. I consider it too a development branch more than a fork, which certainly has "no confidence" connotations. But there's still some worry that it would become one, one day, if there's no sufficient effort towards its integration --as the conversation that I wanted to spark here was almost non-existent up until recently.

This actually brings the topic of more explicitly branch development in Postgres, but it's a different story ;)

> To announce such a big contribution to Postgres, and be it open sourced when published, is very welcome

But the source code hasn't been published. As I said to you today on pgsq-hackers, that's why it hasn't been taken seriously - that's just how it works.

> But there's still some worry that it would become one, one day

Who are you attributing that concern to, exactly?

You're basically claiming that it will be a great opportunity. But you're also claiming that it might be an existential crisis. Do I have that right?

Have you considered the possibility that it might actually be neither of those two things?

> that's just how it works.

And maybe that's just how it shouldn't. Technology moves fast, very fast. Markets do so. Users, you get it. Competing technologies, undeniably.

To delay a discussion for months just because the source is not published is in my opinion only reflecting on part of the opportunity/problem. I'm calling for an strategic discussion, which is higher level than a code-only, limited, partial analysis.

> Have you considered the possibility that it might actually be neither of those two things?

Sure thing, like anything in life, it may be neither. But what I know for sure is that a) we as the PostgreSQL Community need to mitigate risks and plan for them; b) leverage (or transform into!) opportunities as soon as we identify them.

> I'm calling for an strategic discussion

There is practically no information about Babelfish available to analyze. You can currently request a preview of Babelfish if you have an AWS account, though I'm not aware of anyone having even that level of access. There is a single page placeholder on a Github.io that says "Babelfish for PostgreSQL will be available on Github in 2021".

And so I genuinely don't know what a strategic discussion really means here. How much can be decided about something based on a vague press release?

There is information about Babelfish, including reInvent keynote, website and other sources. Here you have a 30min session about Babelfish for PostgreSQL: https://www.youtube.com/watch?v=DcGMoxPCNcQ

I'd also love to have more, architectural/design documents, the code itself, etc, and I'm sure they will be sooner than later released.

But that doesn't prevent us, as I said, to discuss the items I suggested above. They do not depend on the source code, at all.

And independently of this, there is already one thread on the -hackers mailing list that provides some information and proposes to introduce a technical improvement that not only would ease the Babelfish integration, but will also provide significant benefits for PostgreSQL itself, allowing for other protocols to come (including a potential v4 and an HTTP-based protocol): https://www.postgresql.org/message-id/CAGBW59d5SjLyJLt-jwNv%...

It is interesting to think about this in the abstract, but I think that Peter is right. The real collaboration only comes with getting the code published, design proposals on extension points in the core engine, and so on.

This isn't the sort of thing that is decided with blog posts, but personally I am glad to see there is interest in this project.

I think Github has softened the term "fork" a lot, with putting all personal branches into forks.

To indicate a non-collaborative fork, I usually use the term "hard-fork", which I know from the blockchain world, where it has a similar "vote of no confidence"/"we are starting our own project with new leadership" meaning.

I would add that depending on context, the hard part can be implied, particularly of the name is changed. E.g. "LibreOffice is a fork of OpenOffice." It is very context dependent, whether I interpret "fork" to mean hard or soft.

Agreed - a concern for me as well.

AWS has enough scale to have its own gravity and will likely run development forward regardless of .org, it would be GREAT if .org could actively work to integrate where possible the open source options here.

I remember some early efforts of google to contribute wakelocks and other features to the linux kernel - so annoying when that was shot down and created a pretty significant split, and the commercial entity wasn't going to hold off on wakelocks because a subsystem maintainer didn't like it - so android starting going off into its own land.

Wakelocks also ended up proving a reasonable power management approach (despite complaints they didn't use existing linux mechanisms etc).

There was a big push ultimately to get wakelocks into mainline and drive some convergence.

This indeed is the concern. I don't think most people are going to care if it is PostgreSQL.Org PostgreSQL, or Babelfish for PostgreSQL in the long run. What they will care about is modernity of the development process and who has the "cool" features.

When you look at something that has nothing to do with the Babelfish functionality but would absolutely help PostgreSQL, consider PHOT. PHOT (Partial HOT) is a great opportunity for PostgreSQL that Amazon is trying to give back. However, I wouldn't be surprised if the .Org community were to not accept the patch, the feature would end up in Babelfish. If that happens, we now have a viable fork which has 100% compatibility++.

This is the only reference I found for the exact query "partial heap only tuple". It's hard to find anything using "partial HOT".

https://www.talkend.net/post/268380.html

here is a link to the thread on the mailing list: https://www.postgresql-archive.org/partial-heap-only-tuples-...

Reply on news.ycombinator.com