Hacker News Reader

Comment by jdreaver | Hacker News Reader

> If you see idle is above 20 it's recommended to explore using PgBouncer.

Exploring pgbouncer when you have lots of idle connections is a great tip, but 20 idle connections feels _extremely_ low to me. I've seen postgres databases on AWS Aurora serving over 13,000 transactions per second with hundreds of idle connections (because of client side pooling with a few dozen backend clients) just fine. In fact, around that scale is when we switched _from_ pgbouncer to client side pooling to simplify our architecture, and we noticed no degradation in any major metrics.

View on HN

Isn't AWS Aurora PostgreSQL a completely different (but compatible) product to PostgreSQL? If so it would make sense that its ability to handle many connections is just an implementation detail

Aurora forked PostgreSQL very early on (before 9.6) and has been improved independent and a lot has been changed. They are effectively different databases now.

That's not true at all.

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide...

AWS has "only" replaced the storage layer.

They changed the storage layer and the query planner in some parts to work correctly with their new storage layer. So it‘s a fork and not behaving exactly identical. I have seen some different execution plans for the same model and data because of the implications pf the changed storage layer and query optimizer.

Do you know of any credible source for Amazon forking postgresql? I did some googling and can certainly find lots of people saying it, but nothing authoritative from Amazon themselves.

AWS is basically never sharing any details, only when it‘s benefical in terms of marketing. The changes they made had not been possible with an extension in the past, and they are most probably still not fully possible without forking. But thats not problem, they are free to fork and merge upstream changes.

For MySQL any of the big users like facebook & co. Are running (heavy) forked mysql versions with changes they need.

Doesnt Aurora effectively have a PG bouncer in front?

Nope. There is an AWS blog post [0] from September 2021 about setting up pgbouncer in front of Aurora Postgres, and that blog post references the AWS RDS Proxy service [1], but Aurora doesn't have pgbouncer in front by default. For what it's worth, we also handled hundreds of idle connections just fine on vanilla RDS postgres.

[0] https://aws.amazon.com/blogs/database/set-up-highly-availabl...

[1] https://aws.amazon.com/rds/proxy/

Reply on news.ycombinator.com