Hacker News Reader

there are the TPC benchmarks which try to cover a wide variety of use cases and scenarios and are designed independently from any one engine: https://www.tpc.org/information/benchmarks5.asp

you post the results for your own product, others do the same, customers can compare: https://www.singlestore.com/blog/tpc-benchmarking-results/

View on HN

It is partially true, but this benchmarks force schema. You can't reorganise data for example in wide table or add indices. So it actually does not show you how to use the system to solve this type of problems in a best way possible, but checks unoptimised results as if you never learn and never utilise best practices of the DBMS you choose for production.

as long as vendors are presenting the results of their own products, I'd expect them to de-normalize and tune the queries to leverage every advantage their engine can possibly bring to bear on the benchmark's problem set - so long as they're transparent about what they did. Clickhouse apparently does quite well on TPC-DS according to: https://aavin.dev/tpcds-benchmark-on-clickhouse-part2/ but I'd love to see a more official result that the CH engineers would stand behind.

Reply on news.ycombinator.com