Hacker News Reader

Comment by west0n | Hacker News Reader

PostgreSQL is a very unique community compared to other database community(MySQL, Redis, MongoDB, ClickHouse et.c). It is INDEED decentralized, which means NO single company control it. This is related to the style of PostgreSQL's primary maintainers and leaders, who have ensured that the project's decisions and direction are determined collectively by the community members, rather than being controlled by a single company. Hopefully, their departure will not change this aspect.

View on HN

The foundation is set up as a 501(c)(3) and to the benefit of the public. While it is not the only open-source project foundation working like that, many others are 501(c)(6)'s and primarily for the benefit of their (most often corporate) sponsors.

Redis ownership is decentralized as well. Redis the company owns the trademark and they were responsible for about one fifth of commits in recent years. The majority of contributions is external to them. The code base is BSD licensed and anyone is free to create a fork and continue development. Which looks like it is exactly what will happen now that Redis has decided that they don't need the input of the other 80% of commits from external contributors. Those contributors will inevitably shift their attention to one or several of the other forks. The Linux Foundation's Valkey fork looks like one of the main likely candidates for committers to rally behind. The biggest change will be the name change. The notion of the likes of Google, Amazon, Microsoft, etc. abandoning their Redis user base is unthinkable. They'll continue to offer that and they'll continue to lead the development of Redis. Redis the company will loose what little influence they had. I've never heard of anyone using hosted Redis directly from them.

In general, projects like Postgresql and other successful open source projects have in common that generally contributors are not coming from a single company. Community diversity is what gives open source projects resilience against corporate shenanigans. Mysql in some form or another will persevere as well. It has survived a lot of this stuff already and it's still there as an open source option.

I use it as a guide to select which things I use. I look for three things in open source tools and libraries that I use: 1) proper licensing (No agpl or shared source nonsense) 2) community diversity (no single companies that can change their mind), 3) active & recent development demonstrating the project is healthy.

Of course the tragedy of individuals like Simon Riggs passing away is that they are so important for the health of these projects. With postgresql, I'm confident that there are others that can step up. But still, he's been very important and it's important to recognize their amazing contributions. The OSS world is full of these type of hero developers and it's what makes using OSS so wonderful. With Redis, that person would be Antirez. And he stepped back from Redis the company some time ago. It will be interesting to see what he does post fork.

> MySQL

What‘s the point of it, by the way? Why would one start a new project based on MySQL instead of postgres today?

MySQL has an actively maintained LSM-tree based storage engine like MyRocks, while Postgres doesn't have production-ready options in that regard.

Why is LSM useful for a RDB workload?

Log Structured Merge Trees are superior in write volume and scaling B+ trees higher. LSMs are part of the sauce of Cassandra and (I believe) DynamoDB for horizontal scaling.

RocksDB is a single node very efficient (better than Cassandra's in the 2.x/3.x releases, not sure about now) single node LSM implementation.

Because you know it better? Same reason as for most technical decisions tbh.

Besides that, there are still some points where the MySQL and/or the tooling around it simply performs better than Postgres. Anything to do with replication and big table migrations comes to mind.

Not an expert, but briefly looking into this, MySQL is

1. easier to configure and manage

2. faster for read-heavy workloads

3. has pluggable storage engine (though, if you care about this, then 1. likely doesn't matter anymore)

Just as a note, Postgres also has pluggable storage engine, using their tableam API.

It's much less emphasized compared to MySQL, I didn't even know about it until your comment, and search results are sparse.

MySQL has had multiple viable storage engines for most of its existence. MyISAM, InnoDB and more recently MyRocks.

You might be able to argue that Postgres' foreign data is a different form of pluggable storage engine. E.g. you can even use a CSV file as a backing store for a table with file_fdw.

https://www.postgresql.org/docs/current/ddl-foreign-data.htm...

https://dbaclass.com/article/how-to-access-csv-files-on-file...

Someone should do an experience report on adding the same storage system to both databases.

What you say looks powerful but Facebook added their MyRocks to MySQL.

Multi-master replication out of the box. Very useful, very occasionally.

Because I make money with WordPress.

I personally also prefer postgres but I have used mysql in the past as well on and off over the last 20+ years or so. Managed postgres was at some point slightly more expensive in Amazon for some reason. So that's a good reason. If all you need is some simple database, either is fine.

Because it may be good enough. In my case, because I already maintain an instance of MariaDB in production, and I may prefer to share this that maintain another thing

I would choose MySQL because I'm more familiar with it and it's good enough for what I need. PostgreSQL might be able to give me better benchmarks but it won't have any meaningful benefit for me the developer or for the user.

Benchmarks are not the reason people choose PostgreSQL over MySQL. I have never even bothered to benchmark the two for any project that could have used either.

Features, better ACID (at least historically) and maybe better standards compliance.

The big one for me is being able to run schema changes in transactions, which makes it easy to roll back a failed migration.

Also, the license. PostgreSQL is controlled by the open source community via a nonprofit organization.

True, but I would say governance rather than license.

Not being Oracle is a huge advantage ;)

I wouldn’t, only because the big O is behind MySQL.

Otherwise, why not? It’s basically a Chevy vs Ford decision.

Had Oracle stopped investing into MySQL, may be people would have moved on. But they didn't. Just like when people were worried about JAVA, instead we have 15 years of continuous improvement. And that is the same with MySQL. There are lots of features MySQL has as defaults, while postgres simply accept it is not something they want to deal with but leave it to extensions.

What’s wrong with AGPL

AGPL is fine for open source projects. It isn’t really useful for a commercial closed-source or even an open-core codebase in some cases.

On the other hand, usually that’s the intention when a project selects AGPL. There’s usually a commercial license you can buy instead (see iText for example).

AGPL is absolutely fine for commercial closed-source projects. Don’t fall for anti-AGPL propaganda.

https://drewdevault.com/2020/07/27/Anti-AGPL-propaganda.html

It isn't fine because it will come up in due diligence as a risk, every single time. It's easy to say "it's fine", and in theory I agree with you. But it's a cost that doesn't make sense if you know what the auditors are going to say and expect. Auditors flag even more benign things than the AGPL, but to pretend it's not a thing is just whimsy.

I very much want drew's post to be correct, but I can't let confirmation bias cause me to consider this due diligence.

Are there any lawyers or legal cases that you know of that have proven this? or at least lawyers who have reviewed it and given a legal opinion?

It has a few clauses and language in there that generally scare corporate lawyers. There are two main groups of people advocating the use of this license.

1) companies that sell non oss commercial licenses for their AGPLed software that they own the copyright to and want you to buy those. A lot of those companies are now starting to prefer shared source type licenses.

2) open source advocates that don't like any commercial usage of their software and will actively want to prevent any form of intermingling of closed source and open source components like is common in many commercial projects. This is nominally to protect their freedom. But of course it has consequences in the context of commercial projects that don't want to opensource their proprietary stuff. Whether that is actually true or not for any particular use requires a bit of careful legal scrutiny.

Some places that do license audits (e.g. most banks, insurers, and other large companies that need to be alert to potential legal pitfals) would probably flag anything under this license. Three reasons for this: these licenses are only fine under very specific circumstances and certain combinations of licenses are not compatible. And finally of course these audits and lawyers are expensive. So, the easiest way to stay safe would be a blanket ban on anything with this license. Which is my general attitude towards this license.

Anyway, don't take your advice from random commenters (including me) on hacker news and consult a lawyer when in doubt. Yes that costs money. Alternatively, save some money and just steer clear of this mess.

I’d guess Microsoft ends up replacing Redis with Garnet[0] in their stack at some point.

[0] https://www.microsoft.com/en-us/research/blog/introducing-ga...

Interesting comment, why do you think PostgreSQL succeeded where others with the some structure don't? Is it most probably the kind of people involved more than the structure?

I don't have an answer but I'd love to know this too. Why has Postgres got this unique staying power?

Coming at this from a small sample size, every time I've seen it used has been because some of the developers on a team love it and think it's cooler than the other options. And, over time, the operational experience has gotten better (AWS' Postgres support for RDS/Aurora is all recent, for example); and, in fairness, I'd take psql over SQL Server any day of the week.

Regarding why it has popularity beyond mySQL/mariaDB is still a confounding mystery as far as I'm concerned. The additional behaviors Postgres tends to encourage (I'm looking at you, publisher/subscriber and trigger functions) seems to lead to devs advocating it as 'easy' while those in my position are left to keep the damn thing running.

I developed my preference for PostgreSQL years ago, before MySQL supported foreign key constraints or defaulted to durable commits. MySQL also had this annoying tendency to silently store invalid timestamps as zero. All of these things have been fixed since (I hope?), but I still can’t shake my impression that PostgreSQL takes correctness more seriously.

I would say it's similar to Linux:

It's a free, solid foundational technology, guided by steady hands.

In a software economy full of profiteers, charlatans, and marketing babble, the project is providing real value to users.

> It's a free, solid foundational technology, guided by steady hands.

Beautifully said.

This is how ALL open source used to be! Like literally ALL, this is the norm not this bullshit VC funded fremium restricted/tiered fuck the customer trap nonsense.

People built things because they loved it and wanted to help others , not to get rich. Now everyone just wants to get rich, and fast.

While I agree with your sentiment, maintaining software like postgreSQL is a full time job. But your last sentence seems to apply to everything on the internet lately. People used to do podcasts, create guitar tabs or publish cooking recipes because it was their hobby and they wanted others to participate. Now everything seems about making money.

People would do these things for free because they had a stable job which guaranteed their material needs. Now every type of job can be automated and done better/cheaper by a machine, people will be forced to "monetize" everything that exists unless we get a literal revolution in how we tax and distribute the produced wealth.

The jobs we’re talking about here, podcasting, development, etc aren’t jobs where everyone is forced out. Everyone is just more into making money these days and decide they want to make money doing those things rather than just fun. Let’s not try making excuses.

You are getting at it backwards. People are doing podcasts about investing, cooking, music production, <anything> because even those careers are being automated away and the money that they could be getting working is going away.

Even Software Engineers: take all the swaths of engineers who were productive but didn't want / didn't make to a FAANG company and now are having to compete in a world where most companies can replace a lot of the people they don't need a team of 8 engineers because their team of 4 now can have Co-Pilot and most of their "middle management" roles could be effectively replaced by some cheap, off-the-shelf SaaS.

I'm literally in this scenario. I'm too old to be interested in competing with someone who is 20 years younger than me but can call themselves a "programmer", and whatever knowledge/experience I have can be had at a fraction of my "cost" by using a commodified service that automates a process. So, what is left for me? Either I need to go downmarket and work for "programmer" jobs (further increasing the supply and lowering salaries) or I need to find someone who is willing to invest in my "idea for a startup" (thus getting into the Silicon-Valley way of life), or I need to find a way to take my unique experience and repackage as something of value - and then get to be called "greedy" by people like you.

By the way, may I interest you in becoming a customer of my not-yet VC funded company (https://communick.com) and/or join the people sponsoring me for $4/month for my Fediverse work (https://github.com/sponsors/mushroomlabs)?

I do not believe you can replace a competent developer with an AI, or say you have 2 and replace them with 1 dev and 1 AI.

You can't just type in ChatGPT something like "write me GTA5" and you get running code, just seen today an example of someone complaining that he asked soemthing like "Create a website in PHP for a company that does X" and they were expecting that by magic a website will just appear.

Aside from clueless people on Elance and upwork, no one goes to a developer and says "write me GTA5" or "make a website in PHP that does X", either.

What AI will do is leverage productivity of the individuals. Any new story will have its complexity reduced because the developer will be to use the existing codebase and say "hey, our current code is connecting with Foobar via the Zoberg SDK, now we are adding a customer that uses the BazBah platform and they need to change the order flow for 'deliver on payment' to 'deliver on invoice sent'. Show me what changes are needed to make this happen, and please write the integration tests to make sure that we are not breaking things from existing customers"

This goes from a one week task that will require three hour-long to something that can be done in an afternoon, reviewed by the developer and (most importantly) cheap to throw away if the original requirements change.

Does this work today? I guess it might be able to write tests but does the rest just work? In my experience the AI

- uses bad code practices because there is more bad code on the internet then good

- hallucinates APIs , so it tells you to use X but X does not exist in the library/framework you asked for

- suggests wrong solution

- if your language is not precise it gives you the answer to the wrong thing, like you see the answer and you realize it did not understand you

In my experience if your developers are 20% more productive you do not fire 20% of them because there always is a big backlog of features or bugs to be handled.

One of the reasons that I didn't drop out of college (almost 25 years ago) was because I was working part-time proofreading (and occasional translating tech manuals) for a translator who used to get about $25 per 1000 "touches". It could be good money for an experienced translator, but nowadays it's a dead profession outside of legal documents who need a certified notary.

Google's automatic translation was not good enough at the beginning to replace the translator's job, but by the time I was already graduated it was good enough for her to not need my proofreading and it was good enough for her to effectively get 60% of the job done. She has then effectively become the proofreader for a bad translator.

And nowadays, the bad translator is good enough to the point where her customers can just throw the original document on Google and do themselves the proofreading.

This is what will happen with programming tools. Code generation tools are still just at the "smart autocomplete" stage and the experienced programmer is still needed to act as reviewers, but as AI gets better, it will be cheaper to drop the "professional expert" altogether and let someone with tangential knowledge (maybe a product manager) in charge.

People still complain that machine translated Japanese is garbage so I bet will be the same with programming, some easy tass will be automated, complex stuff will be still done by humans with experience and understanding of the domain.

- There is not that much "complex" stuff going around for all the people that will be looking for a job in the field.

- what you call "garbage" might be someone else's "good enough for my needs". If I can go to Japan and a " garbage translator" still is enough for me to help navigate the city or poorly talk to a shopkeeper, then it's mission accomplished and I don't need to worry about a local guide.

- lots of "complex stuff" are dependent on context, and can be made less complex if we relax one single design constraint. E.g, centralized social media networks have a strong requirement for not losing user data. Distributed systems solve this by (a) duplicating data between every node and (b) letting it be deleted by users and node operators who do not want to have the data stored for long term.

It seems to me that you believe that what most software engineers is some dark magic that only a select few can master. It really isn't. The whole "software is eating the world" essay never mentioned what was going to happen after it ran of out of things to eat, now it is kind of obvious that it will gladly get into cannibalism.

My point was that your example was flawed, your translator friend can still have a a lot of work to do since the translators are average or garbage still.

A true intelligent AI sure could be a problem, but this stuff will just be an copilot, good enough to do basic stuff and maybe double check the programmer.

When you predict it would be possible I give the AI a JIRA ticked and it could open the application, reproduce the issue, update the ticket with details about the bug , then find the issue in a giant code base, fix it correctly etc .

Because today an AI can't do anything from the above. It can't replace a human.

My translator friend speaks no Japanese. She used to work with English, German and Portuguese. The fact that translators are still not Professional-level (yet?) is no consolation for the thousands of other professionals like her. She retired already.

> It can't replace a human.

If it provides enough leverage to today to make one person 20x more productive, then it is effectively replacing 19 humans. When it is effective to make one employee 200x more efficient, it will replace 199 humans.

And if you have enough hubris to think you are always going to be the lucky one out of the chopping block, it's not for lack of warning.

>If it provides enough leverage to today to make one person 20x more productive,

But this is not reality. Is this happening for your work? Or you read this somewhere ?

Sorry, my mistake. Replace "today" with "someday".

For "today", I've seen good engineers solving specific tasks in a third of the time already, but I won't make specific claims about absolute productivity multipliers.

>For "today", I've seen good engineers solving specific tasks in a third of the time

Specific is the important word here. Some boring tasks that can be automated in all jobs will be automated though you still need to check the AI. I assume no competent developer was fired because of that productivity boost in that specific task

You don't need to "fire" anyone for AI to cause a significant impact. All AI needs to do is to allow companies postpone hiring more people.

I really don't understand why you are being so obtuse about this. Do you honestly think that you can make the argument that software development (as an industry) is somehow immune to automation?

It’s less automation and more about cheap labor. Content farms sprung up and flooded the landscape with worthless content to get a micro-slice of the pie.

Very discouraging to many content creators when their work is just going to be buried in SEO chaff.

Also, the automation wave is just beginning. Soon the human run content farms will be overwhelmed by AI created crap.

This is likely to happen in software as well. Every product will need to compete with some AI generated piece of garbage that’s barely passable functionally, but being sold at a fraction of the cost.

Fun times!

100% agree on this. Ansible sell out and Hashicorp are fine examples of this.

What's wrong with trying to get rich? Please explain.

Just tell everyone you're dealing with that your primary purpose is getting rich with the software.

Don't tell them you have always been and always will be open source, just until you're big and give the middle finger to OSS in order to get richer.

A given piece of open source code when licensed is always open source. Changing the license doesn’t retroactively do so for the previous code.

There is no lie.

No. No no no. Don't weasel-word out of this with bullshit technicalities.

The phrase isn't "FooBar v3.11 is free and always will be".

There is no version number in the phrase, so the common understanding is that the product and every version of that product will always be free.

lol. If it helps you cope, imagine the company died, another company forked it with a new license.

Same thing, same result. We are literally discussing this in a thread where a prominent maintainer died. Nothing is forever.

There is no way to guarantee something will be the same forever.

Again, fork and move on if things change to your dissatisfaction.

You apparently are not familiar with the concept of software maintenance, upgrade, security patches. Or completely ignored it when wrote this comment.

Yeah man, screw the VCs! That’s why we’re communicating through an open source platform… oh.

Well at least this site isn’t created by a VC… oh.

Things are nuanced. VCs can fund valuable useful things sometimes.

Nobody's saying screw VCs.

We say screw to fooling your users that you're an OSS adopter and supporter, just until your project is big and you can say screw OSS.

You seem to be confused. Even if the project is big and they change the license, so what?

The old code is there with the existing license still. Fork it and move on.

People, man.

you can't fork and maintain everything yourself, and that de-facto lock in is exactly what companies bank on when they pull this kind of bait and switch. The idea is precisely to gain popularity with open source, "the first dose is free" style, and then capitalize on the dependency and popularity. Literally just the developer analog to the misleading "everything is free and always will be" advertisements of consumer facing software.

Ok find other people to help, that’s how open source works no?

There’s no issue here. Just whining. There is no lock in at all.

Even if it were OSI open source the maintainers like the very thread we are in could die. Then what? Oh you fork and maintain yourself, or the project rots.

License changes are irrelevant.

Do you have any idea of what constitutes an OSS project besides characters written on an versioned repository?

People should be more aware of what the license open source software is developed under allows.

Amazon can wrap an open source project in an AWS front end and create a paid for cloud service off the back of community effort. Or, key contributors can decide they want to take the existing code and change the license their contributions are released under going forward.

If the original license allows both these things to happen, then both are a risk and no one is being fooled.

Who cares if they do that? Do you see Torvalds and co running around crying because the entire world runs on Linux Kernels?

I would love nothing more than for a project I built or contributed to wound up as an AWS service.

Writing the code is just part of the value, running it is also very difficult. Especially as the use increases and expose new code paths and bugs and what not.

I think if a big company or two decided to lead development and charge for their Linux Linux kernels he’d have an issue as his influence etc would change. Also he is lucky in that he doesn’t have to care about the making money part. Companies have that issue.

Companies can make plenty of money.

What they can't - without giving the middle finger to OSS - is satisfy greed.

If you want to satisfy greed, fine, but be like Oracle. Sell a commercial license upfront. Don't pretend to be OSS.

Well I’m not sure if it’s just greed at the level at Amazon, Microsoft etc packaging your work and take all the support money from their vast influence.

OSS users don't complain about AWS wrapping around it. It's very much welcome.

The greedy people behind businesses managing OSS are concerned, because they are not satisfied with making money. They want to be THE ONLY ONES making LUDICROUS profits on top of community contributions.

I also have this feeling, but i do feel myself doubting from the lack of examples in this conversation. What are some recent examples of this type of scandal that we can use to solidify this conversation?

I don’t believe HN was created or is being maintained out of the goodness of anyone’s heart.

HN has monetary value to someone somewhere. Plus it’s cheap to run.

It’s also a good advertising and recruiting platform for YC.

There by the grace of VCs goes HN.

Monetization-era

The main alternative to open source monetization is XKCD 2347 (one guy in Nebraska). PostgreSQL appears to have hit that sweet middle ground that is so rare in open source.

https://xkcd.com/2347/

Wow

Design by committee has downsides. Let’s not put Postgres’ development practices on a pedestal.

Design by committee is not really common in the PostgreSQL community. Instead people just work on whatever they or their employer wants to. Makes having a roadmap or cohesive plan impossible but the issues of design by committee rarely show up.

PostgreSQL's development never looked like design by committee.

Postgres is one of the top open source projects of all time. That doesn’t mean everything is perfect with no room for improvement but almost anyone could learn from what’s worked there.

Reply on news.ycombinator.com