Hacker News Reader

Comment by ogrisel | Hacker News Reader

...on a 32 physical cores machine.

Actually it's not really that expensive, when we buy dell machine's which costs like ~4000 Euro we get something like 16 core's. The most expensive thing in a server are prolly SAS disks.

Yeah you can get 18 cores Xeon E7, plop that in a quad socket mobo and you've got 72 cores, 144 HT. They're expensive as hell but if that's what you need…

... but it seems to scale linearly for 2-10 cores, isn't that good enough? Looks like a big win on very modest systems to me.

And? 32 cores are very affordable.

They're not always sitting idle waiting for one big OLAP query (well suited to parallel aggregation) to come in to exhibit a 25x speedup.

Let's say you've got a web application with some cookie-cutter OLTP queries (which are well suited to serial plans ... no parallel aggregation benefit here). Your CPUs are running at 50% then some big OLAP query is going to come in and get maybe ... 13x faster.

Which is a nice performance improvement, but not 25x.

These numbers are a good validation of the work done in PostgreSQL, but you can't look at it and look at your X core machine, and say "my workload is going to be 25x faster!"

That's probably true for any benchmark - YMMV.

> Let's say you've got a web application with some cookie-cutter OLTP queries (which are well suited to serial plans ... no parallel aggregation benefit here).

in my case, the OLTP queries run against a dedicated slave that's mostly busy doing nothing, waiting for the next OLTP query to happen. So for me this will be very, very nice :-)

Since I can't edit my post, the "25x speedup" I quote came from the original title of this post: "PostgreSQL Parallel Aggregate shows 25x speedup"

If your server is already running at 100% then there will be no speed increase!

I was not trying to prove a point. It's just that giving a parallel speedup for a mostly CPU-bound task without mentioning the number of cores is not very interesting. 25x on 64 cores is not as interesting as 25x on 32 cores.

If you know the number of cores you get that the efficiency is 80% (w.r.t. linear scaling) which is very good.

But it is explained exactly and in detail how many CPUs they used. They even show a graph that shows performance for different number of CPUs.

the blog post is very good, my comment was just about adding the missing info to the HN post title.

Edit the title of the HN post has changed since I made this comment... probably by being merged with an earlier submission of the same URL. Now I understand why HN readers are misinterpreting my comment. I did not intend to be negative.

I know ec2 people find it extraordinary having 32 cores, but on ovh you can get machine with 20c and 40 hyper threads and 256 GB of RAM for less than 350$ a month.

I know ec2 people find it extraordinary having 32 cores

How do you figure? As an 'EC2 person' I found that EC2 has made arbitrary specs utterly mundane and that is kind of the whole awesome point of EC2.

Need 200GB of RAM for the afternoon to process some data? No problem, just press a button and a minute later you have 200GB of RAM for the afternoon to process some data. Then just turn it off when you are done and forget about it.

I agree, but who spins up 32 cores for postgresql instance which is up 24/7?

If you need it, it's a bit over $1000/month. Which on the one hand is perhaps 2-3x more than renting a dedicated server. On the other hand it's only 2-3x times more than renting a dedicated server and if you have other stuff on AWS and make use of the rest of their features it might be worth while.

Yup. Small multiples of price inefficiency mostly don't bear thinking about until they group together in large batches. The time it takes your engineers to work out the difference is rarely worth it.

Reply on news.ycombinator.com