- All these products have a lot of knobs to set. Performance depends on these knobs. It depends on whether you do batch sends(which some can do), whether you do transactions, whether you chose at-least-once, at-most-once etc. delivery, queue prefetch sizes, I/O worker size, and numerous other settings.
It'd also be interresting to see qpid-cpp in there as well, not just the Java version.
Apart from declaring the testing queues in some brokers’ configuration and the persistence settings, all brokers were running with their default configuration out of the box (no tuning made).
He also casually mentions that ZeroMQ is using an in-memory configuration.
A home-made ZeroMQ 2.2.0 broker, working in memory only (no persistence).
The default configuration of AMQ persists messages to journal on disk. So apples to oranges?
Why do people keep writing "benchmark" blog posts like this?
For example, look at the "Yahoo! Cloud System Benchmark" AKA YCSB paper. Those researchers spent a lot of time designing the benchmarks and properly configuring each database that they were testing. They even dedicated a researcher per database to sit down with the developers to review the configurations and the test runs. I was part of this process for the Apache HBase as a dev. In the end, everyone was still critical of the results once they saw the graphs comparing the DBs.
But, I'm still glad they did it as I use YCSB as one of my tools to benchmark HBase.