ahachete

> Aren't there two arguments for why this is bad?

> - db will generate new plans as necessary when row counts and values change. Putting in hints makes the plan rigid likely leading to headaches down the line.

> - as new Postgres comes out, it's planner will do a better job. Again forcing specific plan might force the planner into optimization that no longer is optimal.

It depends what is your priority. Most production environments want to favor predictability over raw performance. I'd rather trade 10% performance degradation in average for a consistent query performance.

Even if statistics or new versions could come with 10% better plans, I prefer that my query's performance is predictable and does not experience high p90s or even worse that you risk experiencing plan flips that turn your 0.2s 10K/qps query into a 40s query.

I'm the founder of OnGres [1] the company behind StackGres [2]. I'd love to hear your feedback if you'd be interested in also trying StackGres. It's one of the most feature-full operators available, has a complete Web Console and REST API and supports close to 200 extensions.

Hope it would be interesting for you.

[1]: https://ongres.com [2]: https://stackgres.io

Author of Dyna53 here. Thanks for the mention!

Fun fact: Dyna53 was made public exactly on the same day, 1 year before (Sunday before re:Invent).

Have a look at the Postgres filter for the Envoy proxy: [1] (blog announcement post).

While it's not capturing as of today query performance, it collects notable telemetry for Postgres in exactly the way you mention: just because the traffic flows through it, making it a way to collect data "for free" and definitely without taking any resources from the upstream database.

It can also offload SSL from Postgres. The filter could be extended for other use cases.

Disclaimer: my company developed this plugin for Envoy with the help of Envoy's awesome community.

[1]: https://www.cncf.io/blog/2020/08/13/envoy-1-15-introduces-a-...

One example is "CONF" [1], a website that provides:

a) Detailed documentation about every postgresql.conf configuration parameter (in several languages), for Postgres version 9.1-current. Includes recommendations, comments and links to relevant threads in SO and PostgreSQL Hacker's mailing list relevant to every parameter.

b) A tool to manage postgresql.conf configuration, with tuning guides, the option to store your configs and download in multiple formats, and an open API behind it.

All website is 100% free for the Postgres Community. Disclosure: my company is behind this project.

Edit: minor edits.

[1]: https://postgresqlco.nf/

Actually I turned DNS into a database ;)

https://dyna53.io/

This is not from Supabase, but as a community contribution. See upthread [1]: "at StackGres we have built a Runbook [2] and companion blog post [3] to help you run Supabase on Kubernetes."

[1]: https://news.ycombinator.com/item?id=36006308

[2]: https://stackgres.io/doc/latest/runbooks/supabase-stackgres/

[3]: https://stackgres.io/blog/running-supabase-on-top-of-stackgr...

A mid way could be self-hosting Supabase, whether you use more or less Supabase features.

I know self-hosting might be challenging, specially getting a production-ready Postgres backend for it.

That's why at StackGres we have built a Runbook [1] and companion blog post [2] to help you run Supabase on Kubernetes. All required components are fully open source, so you are more than welcome to try it and give feedback if you are looking into this alternative.

[1]: https://stackgres.io/doc/latest/runbooks/supabase-stackgres/

[2]: https://stackgres.io/blog/running-supabase-on-top-of-stackgr...

update: edit

Oh, DNS is definitely a database engine [1] ;)

[1]: https://dyna53.io

At our company, we provide Postgres 24x7 support. We have partners that provide support for other databases, and for some projects we work together with companies that have multiple databases.

We have over the years compared the rate of production incidents Postgres vs MySQL. It's roughly 1:10 (MySQL has around 10 times more production incidents than Postgres).

You may consider this anecdotal evidence, but numbers managed here are quite significant.

The gist is that Postgres is not perfect nor free from required maintenance and occasional production incidents. But for the most part, it does the job. MySQL too, but with (at least from an operational perspective) many more nuances.

> Well i think that is the only thing that matters

It's not the only thing that matters. It may matter more, but not all. Yet since I'm not your CPO/CMO I won't get into the effort into analyzing their weights ^_^

> SAML client and OAuth client are both free. You can add auth with any OAuth 2.0 provider to SuperTokens.

I'm not denying what you say, but in your pricing page I read:

* "SAML Auth" --only proprietary version

* "2FA" --only proprietary version

so if they are open source, this feature naming is confusing.

We can go back-and-forth debating the merits of the non-open sourced features. But that doesn't change the gist of my comment: you are advertising something as Open Source, where only a fraction (big or small) is, and I consider this misleading. At least for me. I find it more honest to remove that prominent Open Source calls and instead replace for less prominent comments about part of your software being open source (which is fine and great!). But this is just my 2 cents, take them or leave them ;)

[Edit: formatting]

> Many of the core features are open source.

Out of the 15 features I see in https://supertokens.com/pricing, 7 are only proprietary. That's roughly half of them. Without qualifying the weight of every feature, it numerically raises a significant challenge to your statement.

SAML, OAuth and 2FA strike me as key components for me that are not open source.

---

So I stand by my words. I feel put off by a wording that makes me believe a project is open source, when it is open core. Even if you don't like open core or argue the definition is not clear (which I'd disagree), at least marketing it as open source so prominently is IMO misleading, and puts me off (and apparently I'm not alone here).

It's fair to have a business model on open source (obviously!) and I wish you all the luck. But being honest about your business model choices should be the #1 tenet.

This is an Open Core product. The open source part of it seems to be quite limited (see https://supertokens.com/pricing) and therefore I have a hard time believing this version can be "an alternative to [...]".

Actually, the main motto of the frontpage is "Open Source User Authentication", which I also think is a bit of a mischaracterization of the software, since key features I'd look for on an authentication software are not open source.

I love that this is a Java-based project and the goals and ideas behind it; but I think the so prominent use of the terms "open source" is misleading and I recommend demoting them or using alternative terms to reflect a more precise reality.

The recommendation for work_mem doesn't account for all the possible cases. It is already noted elsewhere on this thread [1] that the use of memory per connection could be higher than work_mem, and this is true even if you don't use stored procedures, as the memory incurred can be on a per-query node. So it can be a multiple of work_mem per connection.

But there's a factor that even worsens this: parallel query, which is typically enabled by default, and will add another multiple to work_mem.

Tuning work_mem is a hard art, and requires a delicate balance between trying to optimize some query's performance (that could avoid touching disk or using some indexes) vs the risk of causing db-wide errors like OOMs (very dangerous) or running out of SHM (errors only on queries being run, but still not desirable). So I normally lean on being quite conservative (db stability first!) so I divide the available memory (server memory - OS memory - shared_buffers and other PG buffers memory) by the number of connections, also divided by the parallelism and by another multiple factor --and then leave some additional room.

In any case I'd recommend reviewing the detailed information, suggestions and links on the topic on postgresqlCONF [2] (disclaimer: a free project built by my company)

[1]: https://news.ycombinator.com/item?id=35697986

[2]: https://postgresqlco.nf/doc/en/param/work_mem/

(edit: formatting)

It's not the same, but reminded me of this post [1] I wrote some time ago, about a 63-nodes EKS cluster running on VMs with Firecracker on a single instance.

[1]: https://www.ongres.com/blog/63-node-eks-cluster-running-on-a...

I applaud this effort.

Many years ago I founded a project called ToroDB [1]. ToroDB had a vision very similar to that of FerretDB's: help MongoDB users feel at home on Postgres. This has far reaching implications, like allowing MongoDB applications to run without MongoDB (this is what FerretDB is essentially and what "ToroDB Server" was meant to be) or to replicate data from MongoDB to Postgres to improve the performance of analytical queries by several orders of magnitude (that was "ToroDB Stampede").

ToroDB ended up being discontinued. Timing was not right. At the time, NoSQL was exploding, and users "didn't want to look back to SQL" --until they learned the notable advantages, but it was a time consuming and hard job. Today, there's a much higher acceptance of SQL and most recognize that data querying in many cases goes through, or is significantly helped, by SQL.

I wish FerretDB a successful road and reach to where ToroDB didn't reach at the time. Good luck and congratulations on the 1.0 launch!

[1]: https://torodb.com

> "Unless you’re offering MongoDB as a service, it’s just as “open source” as ever."

It's not. It's source available, and that's still proprietary.

The reason why the SSPL is not Open Source is pretty simple: it doesn't convey the four essential freedoms of free software [1]. That's it, there is essentially no more to it: it imposes usage restrictions, and that's orthogonality against the spirit.

Whether it is "except this or that" or "AGPLv3 but with this additional clause" is irrelevant: even the tiniest change can cause significant differences, and this is the case: it removes the freedom to run the program as you want, as it imposes restrictions to some use cases. And these restrictions go beyond the realm of the software itself.

In contrast, Open Source copyleft software, like AGPLv3, never go beyond the software itself. It provides guarantees that modified versions of it also remain available for users of modified software (forward carrying guarantees) but do not add a requirement to also provide under the same license other unrelated software (which is essentially a nice way of saying "simply don't do this", turning de facto into a usage restriction).

[1]: https://www.gnu.org/philosophy/free-sw.en.html#four-freedoms

I don't want to go too offtopic on this one --feel free to join StackGres Slack Community [1] to discuss further.

As a one-liner, though, for completeness: StackGres is fully open source (unlike Crunchy that needs a license for production); comes with a Web Console; 150+ Postgres extensions (including Timescale, Citus and many others); and many Day 2 operations fully automated.

[1]: https://slack.stackgres.io

Yes, absolutely.

This is good and interesting recipe to get Keycloak and Postgres on Kubernetes.

There is an important improvement, though: the Postgres deployed here is not production ready (high availability, backups, monitoring, etc).

We run Keycloak on StackGres [1] which gives us production-ready Postgres setup (disclaimer: it's dogfooding). Happy to share the YAML manifests used to deploy Keycloak with StackGres. Maybe we will write a blog post as a follow-up to this one, for completeness.

[1]: https://stackgres.io

The key for me is the level of automation that you can reach at a reasonable "development cost". Let me elaborate.

K8s, if anything, is an API. An API that allows you to interact with compute, storage and networks in a way that is abstracted from the actual underlying infrastructure. This is incredibly powerful. You can, essentially, code and automate all your infrastructure.

But this goes beyond deployment, something you could achieve (more or less) with tools like Terraform or Pulumi. Enter "Day 2 operations".

Day 2 operations are essential for any database. And cloud services have done a good job at automating them. Speaking of Postgres, my daily job, things like HA, backups but also minor and major version upgrades are table stakes day 2 operations.

If you want to build these day 2 operations in the cloud (say on VMs), even though you have APIs do to so, a) they don't implement a pattern like Kubernete's reconciliation cycle; and b) you have a distinct API per cloud. K8s solves both problems, making it way "cheaper" to build such an automation. On K8s, a given operator can code these day 2 operations against K8s APIs. Therefore, if you want to build such automation, either you are a cloud provider (and potentially do this only for your own cloud) or you do it on Kubernetes.

This is so much true, that existing operators have already gone beyond what DBaaS do. Speaking of StackGres [0] (disclaimer: founder), we have implemented day 2 operations (other than the "table stakes" ones that I mentioned before) that no other DBaaS offers as of today, such as vacuums, repacks and even benchmarks (and more day 2 operations will be developed). See [1] for the CRD specs of SGDbOps, our "Day 2 operations" if you are interested.

[0] https://stackgres.io [1] https://stackgres.io/doc/latest/reference/crd/sgdbops/

> I suppose this is mainly a thought for projects like neondb/cockroachdb/stackgres

StackGres is "just" a platform for running Postgres on Kubernetes. It helps you deploy and manage HA, connection pooling, monitoring, automated backups, upgrades and many other things. That you have a tiny Postgres instance; or hundreds of beefy clusters with many instances is up to you. It's not a distributed database (like the other ones mentioned), it is still "vanilla" Postgres.

Disclosure: Founder of OnGres (company behind StackGres)

Relationship is at best one-way only. C++ can mix and use C code. A C++ compiler will do fine with C. A C++ programmer will do more or less reasonable C code.

C code will not accept C++. A C compiler will not support C++ (unless it's explicitly designed as a C++ compiler too). A C-only programmer will be as lost on a C++ codebase as on a Rust, Go or Java codebase.

Interesting journey.

> C/C++ is undoubtedly one of the most popular programming languages for building database systems. Most well-known database systems, including MySQL, PostgreSQL, Oracle, and IBM Db2, are created in C/C++.

Considering C and C++ to be the same language is quite a stretch. They are fundamentally different.

Check DOMAINs. They are derived data types that take a base type and add, optionally, one or more CHECK constraints to them.

They are not perfect, but probably something close to what you are looking for.

Sharding databases is not such a dark, magic art. Citus relies on Postgres for many key features; and Citus does the rest. It's already quite "feature complete".

If Citus would become proprietary overnight, my main concerns of maintaining a fork would be around the codebase and the language expertise more than the sharding concepts.

Note that sharding is different from a purely distributed database. The latter is an entirely different class (and more complex system).

Maybe... but there are many queries that don't use an index (whether fast or slow) and that's the right thing to do. As a DBA, I'd see too many "false positives" due to this reason.

Right, those are compulsory steps for every upgrade.

Yet in the particular case of Citus, history (so far) has shown a) that they update the extension regularly and fast, so by the time you want to upgrade to a newer major version you already have Citus updated too; b) they are going exactly in the opposite direction of "freemium", they actually open sourced even the previous proprietary bits; c) as OSS, it can always be forked and if one day closed source, being such an important project, it would be definitely forked.

(I don't have any stakes on Citus)

Many of those queries may run sub-second (typical lower-end value for log_min_duration_statement), so they won't get logged. Yet, if called at high frequency, may represent a notable % of your CPU and I/O. The slow query log is not enough in many cases.

> 2) Horizontal scalability without having to resort to an extension like Citus.

Just curious, what would save you having the solution in-core? Installation, sure, but that's a one-off possibly in your deployment code. "CREATE EXTENSION citus" and add that to postgresql.conf? Sure, but not too much work for me. The rest (commands to actually create the nodes, do the sharding itself) are something I cannot imagine being different or simpler if with an in-core solution.

What am I missing?