As a non-storage person: what should I be using instead of dd and bonnie++?
As it turns out, I've given a few presentations on this stuff. Here's one of the more recent ones:
https://docs.google.com/presentation/d/1CQgn_fXivdXZ-fcqJ39v...
Beyond bonnie++, storage benchmarks are fraught with peril; years ago, I dismembered SPEC SFS as being similarly unsafe at any speed when benchmarking storage systems, albeit for much more subtle reasons than the glaring mechanical flaws in bonnie++.[2] And as for dd, I actually think it's okay as long as you explain clearly what it is (and isn't); Jeff's complaint is that they seem to be treating this single dd invocation as "write performance", when it fact the truth is subtler -- and things like blocksize and synchronicity matter a great deal.
More generally, anyone interested in storage benchmarks would be wise to read essentially everything that Brendan Gregg has written on the subject, starting with his five-part (!!) series on file system latency.[3][4][5][6][7]
[1] http://www.brendangregg.com/ActiveBenchmarking/bonnie++.html
[2] http://dtrace.org/blogs/bmc/2009/02/02/eulogy-for-a-benchmar...
[3] http://dtrace.org/blogs/brendan/2011/05/11/file-system-laten...
[4] http://dtrace.org/blogs/brendan/2011/05/13/file-system-laten...
[5] http://dtrace.org/blogs/brendan/2011/05/18/file-system-laten...
[6] http://dtrace.org/blogs/brendan/2011/05/24/file-system-laten...
[7] http://dtrace.org/blogs/brendan/2011/06/03/file-system-laten...
Longer:
bonnie++ is not a load generator. Really. Start a typical run with a command line test case you find online and you'll see no actual IO. Lots of cache hits. We used to use it, 10 years ago, to play around with load generation, but found that it didn't generate enough IO, or in a way that actually matched what people do.
IOzone is marginal ... I wouldn't use it for a serious test, and when people suggest dd or IOzone, I ask them how well the code actually matches their use case. Chances are that it is also largely irrelevant to this. Worse still is that the IOzone throughput measurements are basically bogus, using a naive sum of bandwidths, rather than showing the interesting data (the actual histogram or distribution of performance, including the start/end times, and the rates per thread/process as a function of time).
fio is good, in that you can implement many types of tests that have a reasonable chance at being meaningful. I caught Sandforce controllers compressing non-random data on benchmarks they were using for the SSDs that used them with fio. Actual SF performance was lower than spinning rust once you fed it real random data.
We wrote something called io-bm (https://gitlab.scalableinformatics.com/joe/io-bm) for pounding on parallel file systems (specifically to stress them and see how they scaled and dealt with contention for network, metadata, etc.). I am not sure if the repo has our histogramming and time series bits in it so we can see individual thread performance as well as overall performance, might be in a private repo.
Basically it boils down to the TL;DR above. If its not your code, then you are likely testing with an application that is somewhere between partially to wholly irrelevant to your use case.
1) You (probably) aren't testing the things most done by your application/platform/etc. And worse yet, the hardware/software maybe simply going for big numbers in the common benchmarks.
2) The test may or may not be garbage depending on your use cases.