Show HN: Pigsty – Free RDS Alternative | Hacker News Reader

Show HN: Pigsty – Free RDS Alternative

github.com

103 pointsVonng50 comments

Interesting, but unless you understand every piece of software running and how they interact and are deployed, running this sounds like a devops and security nightmare.

That’s my thing. I closed the docs as soon as I opened them because I already couldn’t imagine convincing anyone this was a good idea enough to convince me it was a good idea. If that made any sense

You need a lot of clout to get something like this out. Kubernetes is an example of something in this category, for example. That baby needed to come out of Google to be both fully formed and in the "didn't get fired for choosing" bucket :-).

Speaking of which, why isn't this project delivered as a Helm chart?

Yup me too, it's great that it's extensible, but it's also it's immediate problem.

It might require a bit of elbow grease to grasp every component. I'm trying my best to smooth out that learning curve:

https://doc.pigsty.cc/#/PGSQL-ARCH?id=component-overview https://doc.pigsty.cc/#/PGSQL-ADMIN https://doc.pigsty.cc/#/SECURITY https://doc.pigsty.cc/#/ARCH

However, when it comes to getting it up & running, the process is pretty straightforward, and the outcome is leagues ahead compared to running the kernel in the raw way.

This looks very very fragile. If all of this works, great job though.

For the extension part, maybe. we only use very limited extensions in production: PostGIS, TimescaleDB, and PGVector will suffice for most cases, plus pg_repack, pg_cron, and wal2json for maintenance.

The core/RDS part (HA/PTIR/IaC/Monitor) is quite robust, which has served our 25000 core deployment for 3 years+ and survived dozens of hardware failures.

How does it look fragile?

It runs on or depends on so many disparate software packages that each comes with their own versioning strategy, security model, etc.

Usually when you install a package in a linux distro either the distro or the package maintainer of the package you install is responsible to ensure it keeps working (with varying levels of effort depending on distro, support strategy, etc...).

With these sort of amalgamation packages that becomes weird. Nobody is responsible and its built with many parts that never tested new versions in that configuration.

You need to either understand what you run or understand who you delegated that understanding to.

This is true for individual packages and even more for amalgamations like this.

In most of these amalgamations you delegated it to nobody and you don't realize until it breaks and you cannot fix it.

The project lists dozens of extensions that it supports. The likelihood of one of them not working properly is very high.

For frequently used extensions such as PostGIS, TimescaleDB, Vector, Repack, etc., it can be assured that they function together as expected. Regarding other extensions, they are only guaranteed to be installed and `CREATE` without error.

I think all the additional layers are the issue for me. There's a lot of moving pieces here that should each be integration tested independently rather than included all together.

This looks very good, but the marketing copy (the README is written as such) is horrendous.

This HN thread is already full of people saying "this looks brittle", "maintenance nightmare", &c. because the README spends so long trying to convince you that this product is a huge, multifaceted kitchen sink. People want simplicity, noone is looking for a plethora of things they need to run, why would you ever try and sell anything as that?

Most of the "features" they enumerate are either:

- optional pg extensions that are available in normal pg by default

- orchestration software that one might use to deploy HA pg clusters in a distributed / k8s type env anyway (i.e. it's not extra, it's just your underlying infra templates - e.g. Terraform, Patroni, &c.)

The only thing I really see there that is "not-really-needed-kitchen-sink-extras" is the observability stack (grafana, loki, &c.)

> Free RDS Alternative

RDS is a managed service. This is a code repo. It is useless until you deploy it at which point... it's no longer free. Then you have to manage it yourself and deal with the overhead yourself.

This is like saying I can fire my gardener because you have a free gardener alternative, and then handing me a pair of scissors. I now have all the problems that I paid the Gardener to solve.

Likewise, we pay AWS to manage all the headaches that deploying this would introduce. And trust me, if you are running databases at scale in production, then RDS feels very affordable considering the problems it solves.

The scale of operation is the crux here. For a modest number of cores, RDS is a congenial choice. However, when you're going to hundreds or, as in our scenario, tens of thousands of cores with PostgreSQL, clinging to RDS is sheer lunacy.

I was the designated gardener to tackle this: architecting and managing a PostgreSQL deployment with 25K cores and 3M TPS. We've been shelling out a cool $1M annually, covering the whole thing - hardware, software, and DBAs. Meanwhile, the toll for RDS is an astronomical tenfold of our current expenditure, yet it comes with a lesser degree of availability and a starkly crippled observability and other stuff.

This is cool and I applaud the effort but basically no one should use this.

Most folks should use RDS etc. (Yes, it's expensive and has limitations.)

If you need or want to self-host, you need to understand every moving part and how they fit together. The effort and knowledge required to assemble your own setup are essential, and outsourcing them to a magic bundle would be a mistake.

Title should be "Open-source Amazon RDS alternative".

Yeah, the original title was "Show HN: OSS PostgreSQL RDS with Supabase,PostgresML,Vector,HA,PITR,Monitor,&100+ Extensions" ... now auto-renamed without the "RDS" part...

I have never needed to use 95% of these extensions. Why would someone want to install all of them by default? Update nightmare.

Where were you a year ago. I had to do ha with patroni etcd and pitr with pgbackrest myself. What I did not see is what will happen if one node dies. What is the procedure to jump start it back. Etcd is a complicated beast...

I'm so confused, what's the role of Redis here? , looks like a "special case" of running redis instead of pg?

Redis is more like a cherry on top. Many applications utilizing PostgreSQL also employ Redis, Gitlab being a notable example, and that includes us as well. Supporting Redis merely involves adding a couple of RPMs and Playbooks, so why not go for it ;)?

Why would you ever want that many extensions?

They make it pretty clear that the extensions are optional. Providing support for installing a diverse set of extensions seems fine to me - everyone has different use cases

Imagine a realm where a myriad of extensions can freely interlace, conjuring a synergy where 1+1 transcends 2 within the PostgreSQL ecosystem.

Given that all extensions are optional, bundling them all together poses no harm to the stability of the core part.

Why would they want that many available, or installed? Those are two different questions.

to increase attack surface

Wouldnt you be locking this entire infra in a local network behind a firewall?

So what’s the difference between this and me just running postgres in a docker container and enabling these extensions?

You don't need to rebuild the docker image and restart the container to get the extension installed & created.

And, I haven't found any Postgres docker image with all the extensions I need: you can reuse the RPM in dockerfile to build your own image, which always has to be done somewhere.

Isn't PostgreSQL Opensource?

It’s an RDS alternative, aka managed operations service for postgresql and a curated and extensive catalog of optional pg extensions. The title should be fixed as its misleading.

Sounds like a security nightmare.

If all 150 extensions are installed simultaneously, It's definitely a nightmare. Luckily they are all optional components.

AGPL3, no thanks.

Curious if there are runtime downsides to having so many extensions installed

These extensions are pre-packed and included in the local yum repo, which can be enabled by configuration. Only core extentensions are installed by default, which are: pg_repack, wal2json, postgis33, timescaledb, citus, pgvector. And the only extension activated (`CREATE EXTENSION`) by default is pg_repack.

The AGPLv3 is infected by the Grafana, MinIO and citus.

https://github.com/Vonng/pigsty/blob/master/docs/PGSQL-EXTEN...

Personally, I like the AGPL. It means that when a place uses / extends the code, it's pretty likely they're intending on contributing back.

It's the places that go "Ewwww... AGPLv3!" that you have to wonder about.

Until AGPL3 is tested on court I'll keep avoiding it for my day job. While I appreciate the idea, I'd rather not risk my livelihood.

My employers do contribute (however modestly) to FOSS, though typically MIT licensed.

What test do you want to see in court? Is it due to concern of the virality of the license infecting software that uses it? Or due to concern that the license doesn't actually protect what it claims to and would be found unenforceable by the court?

Whether or not AGPL3 software will virally infect an entire service of which it constitutes one part.

Can't say I'm following the Why? entirely.

Is this intended as a prototyping tool?

Supabase and PostgresML have been introduced in the latest release. At this juncture, they are in a prototype stage, aiming to expand the functionalities' spectrum.

The RDS part and most extensions have served as production tools for us for a long time.

This gives me a lot of mixed feelings.

HN title notwithstanding, this is a distribution of PostgreSQL (i.e. many optional extensions built in), not an “alternative”. Also, both PostgreSQL and this alternate bundle are indeed Open Source.

The title was auto-updated without the crucial "RDS/Distribution" part before "alternative"...

> many optional extensions built in

> Battery-Included PostgreSQL Distribution, with 150+ powerful extensions!

Wow, you weren't kidding.

This is cool, but this sort of proves my point to friends that Postgres isn't "production ready" out of the box, you need all kinds of tooling around it to make it so.

Curious which DBs are "production ready" according to you, and how you define that exactly.

“Production ready” is context-dependent. If you have no need of particular features, it may make sense to not have them installed for a number of reasons.

Can you speak more on what you think is missing? Looking at the repo I semi-regularly deploy grafana (which I'd argue should never be part of core postgres) and pgbackrest (I wouldn't hate having more sophisticated backup built-in). I once deployed citus for an application that needed it's special talents. In general I'd say my production use of postgres has been pretty vanilla.

Reply on news.ycombinator.com