Hacker News Reader

> If open source makes little effort to optimize the vast majority of other operational costs at scale, and this is largely the case in my experience

I feel the opposite is true. There are many open source projects that are at least on par with their proprietary counterparts. Postgres is competitive with every SQL DB I've heard of except maybe in some niche use-cases. Redis is open source and incredibly performant. Same with Kafka (if you use it well). Linux sure runs better than Windows in my experience.

There is also a whole class of open-source libraries like React that have taken over. I can't even think of a proprietary frontend framework.

What software are you thinking of when you make these claims?

There are two major dimensions to the operational cost weaknesses of open source. Neither is guaranteed to be meaningfully exploited by proprietary software in any particular case. Since we are talking about operational costs, this is almost entirely about data infrastructure software, not things like React.

First, there is a vast amount of supported operational tooling that is simply missing from open source ecosystems. PostgreSQL, which I still use extensively, is a perfect example of this. The code bases around this tooling are several times the size of the product they support, and it isn't the kind of code that most developers aspire to write despite its high value to the customer. Oracle and SQL Server have architectures that are as obsolete as PostgreSQL, they don't compete on the basis of being modern, efficient designs but on the basis of having dramatically better operational tooling. Cloud RDBMS attack a different aspect of this.

Second, is the operational infrastructure costs i.e. how much hardware is required to run a given workload. This is the larger threat to open source, particularly as the data intensity of business increases. PostgreSQL, Kafka, and Redis (I've used all three operationally) have several times the hardware requirements to deliver a workload in practice than is required with a state-of-the-art architecture. This isn't hypothetical, I've designed engines that replaced them when warranted. An 80% reduction in infrastructure cost is attractive when you are already spending tens or hundreds of millions of dollars per year on it. Some popular proprietary software is just as inefficient but that isn't guaranteed to remain the case (and I've replaced those systems too). Designing efficient software architecture isn't magic (see: ScyllaDB), you simply see very few examples of it in open source because it isn't a priority (see: use of managed languages for data infrastructure despite this being a known limitation).

The reality is that I can guarantee data intensive businesses who make competent use of open source that I can reduce their infrastructure costs by 4x with ease. You can bury a massive amount licensing and engineering costs in those savings.

Also, some companies are explicitly looking at this as a major "green" initiative. The wasteful infrastructure footprint to deliver a workload with open source is viewed as environmentally unfriendly, and this is pushing companies to consider proprietary solutions even if they are not sensitive to operational costs. That isn't a conversation open source is currently prepared to have but it is coming.

> I can guarantee data intensive businesses who make competent use of open source that I can reduce their infrastructure costs by 4x with ease

I am interested in your perspective and experiences.

> Oracle and SQL Server have architectures that are as obsolete as PostgreSQL, they don't compete on the basis of being modern, efficient designs but on the basis of having dramatically better operational tooling.

What dramatically better tooling does SQL Server has?

> PostgreSQL, Kafka, and Redis (I've used all three operationally) have several times the hardware requirements to deliver a workload in practice than is required with a state-of-the-art architecture.

What should I think of when you talk about state-of-the-art architecture? And what would replace aforementioned tools?

What should the open source offerings do to win you over?

> What dramatically better tooling does SQL Server has?

I don't know about Oracle but SQL Server Data Tools (SSDT) and SQL Server Management Studio (SSMS) are vastly better than anything I've seen for open source databases.

SSMS is just a really good visual client with an excellent table/view designer, diagram builder and query builder with GUI management features that let you manage virtually every facet of your server and databases.

SSDT lets you treat your SQL schema, database settings and deployments like code, with versioning. All of your table schemas, views, functions, procedures, settings, seed data, etc. are stored as text in your git repo. When you're creating, it has excellent intellisense/autocomplete. You can use it to diff your schema and data against a given SQL Server database to generate migration scripts. You can also do data comparisons between your seed data and your server and you can diff schemas and data between 2 different servers without even having created an SSDT schema. Another very useful feature is that you can reverse engineer a SQL Server database that was already created in order to start your SSDT project. It also has visual designers for all of the aforementioned things. It used to be a separate tool, but now it's part of the free Visual Studio community edition. Both tools are free, as are various editions of SQL Server.

Also, SQL Server itself has had features for a long time that PG is just starting to get such as real Stored Procedures, ones that can return multiple heterogeneous result-sets, run transactions and do control of flow with SQL-like statements (IF/THEN, DO, WHILE, etc.) It also has table variables, which are like in-memory tables, that you can use as a faster type of temp table for certain sets of data.

Since I moved off of Windows for my workstations, I've been running SQL Server in Docker on Linux for the past few months - it's rock solid.

Afaik, those things are available in DBeaver [0] and MySQL Workbench [1] as well.

@jandrewrogers also mentioned a 4-fold performance increase by switching to proprietary solutions from pgsql. I am still wondering what those solutions might be...

---

0. https://dbeaver.io/

1. https://www.mysql.com/products/workbench/

I’ve used both and I can assure you that both the products you mention fail to approach the utility and more importantly, the quality of either SSMS or SSDT.

Have you tried the SQL Server based products at all?

Not that much, so I might have to take a better look then. :)

Not the parent but I thought of AWS services like RDS and S3.

I believe the point being made was the money you save on licensing fees are insignificant compared to the amount you spend on paying people to maintain your infrastructure.

I have used Postres during 4 years, Oracle and SQL server since 2000.

Postgres is a nice database, but there are enterprise deployment use cases that it doesn't cover, which might be consired a niche, fair enough.

What I don't consider a niche use case, is that its development tooling, meaning graphical DB management, graphical debugging of stored procedures, data modeling tooling, or integration of language runtimes into the database are all a bit behind of what those big DBs are capable of.

I disagree about the opex cost, as anyone who has had to pay for oracle consultants to come in and fix a borked dB would.. I think that a more accurate assessment is 2x-10x the claimed sticker opex cost that the vendors talk about. The problem bring that "boss my system has shat its pants and I need half the budget to g fix it" is a story that NO ONE in ANY corporate can tell twice.

How is it a different story if it was Oracle that shat its pants instead of OSS?

You are not limited to Oracle engineers and access to proprietary source code and documentation to clean your systems pants. That makes individual fixes cheaper and the fixes can be shared publicly, leading to additional cost savings.

> The poor optimization of operational costs (except for license fees) is a critical weakness in open source, and it is increasingly being attacked successfully.

This is the developers perspective. In the time it takes to develop something the licence fees are indeed at tiny part of the cost.

However, over the lifetime of the software itself it's usually a different outcome. Software typically lasts decades. Banks are still running cobol software written decades ago. Perhaps it's possible even over those long lifetimes the licensing costs still win. When it breaks down completely is when you deploy that software multiple times, paying those licence fees each time you do it. If you start paying for 1000 CAL's you have a problem.

The other comment here that is a red flag to me is "all the tools that give you visibility into the database save you time". If it is development time then well and good, but if you are needing DBA's once the thing is deployed then just no. No matter how sophisticated those tools are, they still cost far more than needing no attention whatsoever. This is where open source usually wins, because it's lack of sophistication is usually an case of being pared back to the bare minimum to do the job.

Reducing OpEx often requires cooperation and pooling of resources that a proprietary vendor can force on their users with little choice, while by default software like Mastodon, Pleroma, etc treat other instances as potentially hostile, replicating data (often) needlessly.

Efforts like https://jortage.com can reduce OpEx quite significantly.