Hacker News Reader

Comment by eljulio | Hacker News Reader

One missing point Is the scalability. Postgres is awesome until you need to setup a db cluster.

Very few organizations in the world truly need to scale out their database. Their problem is usually that of a bad design. For those that need to scale out, NoSQL is not a magic solution either. The scaling out solution pretty much these days for SQL database is replication, sharding and partitioning tables. Postgres does these very well.

You might want to learn a bit about "big data".

Almost every major organisation will have a big data analytics program doing feature extraction, modelling, machine learning etc. Usually this starts with Hadoop/Spark with data in HDFS but then a point comes when you want this in a database.

PostgreSQL is awful at this which is why almost no uses it in this space. Apart from the lack of native drivers it is simply too difficult to do basic clustering. And the fact that it isn't built into the product doesn't give you a lot of confidence that it is (a) going to work and (b) is going to be supported.

And yes for those that scale out NoSQL is a magic solution. That's why they are so popular. I can deploy a 40 node Cassandra cluster in 30 mins and guarantee it works. Likewise MongoDB replica sets are ridiculously easy.

Maybe you could just design your data a bit better with smaller data types, and keep it all one one machine.

> The scaling out solution pretty much these days for SQL database is replication, sharding and partitioning tables. Postgres does these very well.

I tend to disagree that Postgres does partitioning well; partioning table resolution is rather dumb, there is no hinting (e.g. there is no way to tell postgres that there is a unique mapping for each query to a table) and no caching of CHECK results.

> Very few organizations in the world truly need to scale out their database.

Everyone organization that has a database wants to scale out to at least a 2 node cluster just for not having a single point of failure. Scaling is not always about performance, at first, it's more about availability.

Well, we ran a billion queries per day on a 1.4TB system I was in charge of a couple years ago. Now we're building a 500+TB reporting system and probably putting it on Postgres-XL. Seems to scale fine to me. ;)

You're talking about performance, that's a different thing than scaling.

are you saying they're not related?

Only loosely in so far as bad performance can make scaling more important; however, scaling is not about performance, a super fast db could scale poorly while one that scales well could perform worse than the fast one but scale better. Scaling is about growing across machine boundaries well, so that adding more machines increases the ability to handle more load. That doesn't require good performance, it's requires good architecture.

The first was performance, while the second is a 500TB database, and that very much requires scaling. I'm just saying it can do both well.

Ah, Postgres-XL is multi-machine, ok sure, that's scaling.

Is it actually any better in Mongo? I know their marketing says it is, but from what I have read the reality isn't so.

Agreed. Scaling out databases is never something you want to do, particularly for Postgres.

But it's something you're less likely to need if you structure your data, in an RDBMS. If you know your SQL, then optimizations should be preferred before scaling out. Not just because you need to, but because it can improve user experience (server response latency).

would you like to describe the problems you had with the existing solutions?

Reply on news.ycombinator.com