Hacker News Reader

Comment by menaerus | Hacker News Reader

"get statistics" is not what TPC-B does. Nor the invalid queries nor ...

From https://www.tpc.org/tpcb/, a TPC-B workload that pgbench runs by default:

    In August 1990, the TPC approved its second benchmark, TPC-B. In contrast to TPC-A, TPC-B is not an OLTP benchmark. Rather, TPC-B can be looked at as a database stress test, characterized by:

      Significant disk input/output
      Moderate system and application execution time
      Transaction integrity

    TPC-B measures throughput in terms of how many transactions per second a system can perform. Because there are substantial differences between the two benchmarks (OLTP vs. database stress test), TPC-B results cannot be compared to TPC-A.

    ...

    Transactions are submitted by programs all executing concurrently.

View on HN

I think you missed the context of what I was responding to, which was about whether databases could even have micro-benchmarks.

You also missed the word "Obsolete" splattered all over the website you sent me, and the text that TPC-B was "Obsolete as of 6/6/95".

I don't think I have. I was only responding to the factually incorrect statement of yours that pgbench is a microbenchmark.

> which was about whether databases could even have micro-benchmarks.

No, this was an argument of yours that you pulled out out of nowhere. The topic very specifically was about the pgbench and not whether or not databases can have micro-benchmarks. Obvious answer is, yes, they can as any other software out there.

I think that you kinda tried to imply that pgbench was one of such micro-benchmarks in disguise and which is why I c/p the description which proves that it is not.

> You also missed the word "Obsolete"

I did not since that was not the topic being discussed at all. And in a technical sense, it doesn't matter at all. pgbench still runs so it is very much "not obsolete".

I didn't pull this argument out of nowhere, please read the direct comment I was replying to. Your position is also completely untenable: this benchmark was obsoleted by its creators 29 years ago, who very clearly say it is obsolete, and you're arguing that it isn't because it "still runs."

I'm guessing that this discussion would be more productive if you would please say who you are and the company you work for. I'm Brendan Gregg, I work for Intel, and I'm well known in the performance space. Who are you?

> I'm guessing that this discussion would be more productive if you would please say who you are and the company you work for. I'm Brendan Gregg, I work for Intel, and I'm well known in the performance space. Who are you?

Wow, just wow. How ridiculous this is?

> Your position is also completely untenable: this benchmark was obsoleted by its creators 29 years ago, who very clearly say it is obsolete, and you're arguing that it isn't because it "still runs."

My only position in the whole thread was "1% of overhead cannot be universally claimed" and I still stand by it 100%. pgbench experiment from Linux kernel folks was just one of the counter-examples that can be found in the wild that goes against your claim. And which you first disputed by saying that it is a micro-benchmark (meaning that you have no idea what it is) and now you're disputing it by saying it's obsolete (yes, but still very relevant in database development, used in the Linux kernel and not the slightest technical reasoning after all).

Personally, I couldn't care less about this but if names is what you're after, you're not disagreeing with me but with the methodology and results presented by Mel Gorman, Linux kernel developer.

It's not ridiculous at all. Who are you?

You are backing away from your other positions, for example:

> I fail to understand the reasoning of it "being simple" or "microbenchmarkey". It's far from the truth I think.

Do you now agree that TPC-B is too simple and microbenchmarky? And if not, please tell me (as I'm working on the problem of industry benchmarking in general) what would it take to convince someone like you to stop elevating obsoleted benchmarks like TPC-B? Is there anything?

A major postgres contributor (Andres Freund) disagreed with you about pgbench but, yes, feel free to dismiss them just because you found some words on a web page.

I am just a minor PostgreSQL contributor and consultant of no import, but do you seriously think you are superior to PostgreSQL core devs and know more than them about PostgreSQL just because you are a Linux kernel dev? I really do not like your attitude here and your appeal to your own authority when you are not even an authority here.

Pgbench is used heavily both in the development of PostgreSQL and by people who tune PostgreSQL. So it is not obsolete. Maybe it is a bad benchmark and that the PostgreSQL community should stop using it but then you need a stronger argument than just some claim about obsoleteness from 1995. If a PostgreSQL core developer says in 2024 that it is still relevant I think that weighs a bit higher than a claim from 1995.

Yes, indeed, it is very ridiculous to pull out arguments such as "who are you". I mean, wtf?

Your replies demonstrate lack of technical depth in certain areas and which makes me personally doubt in your experiment results. And you know what, that is totally fine.

But your attitude? A total disbelief.

> Do you now agree that TPC-B is too simple and microbenchmarky?

No, why would I, you did not present any evidence that would support that claim of yours?

And contrary to you, and to your own embarrassment, I do not use personal attacks when I go out of technical depth.

> what would it take to convince someone like you to stop elevating obsoleted benchmarks like TPC-B? Is there anything?

You don't have to convince me anything. Remember that this is just a random internet page where people are exchanging their opinions.

Wish you a pleasant day.

Reply on news.ycombinator.com