That's true of all I/O benchmarks, even those which try to do things like call fsync since the lower layers often lie about whether something was committed to durable storage. The only way to do benchmarks reliably is still to make sure that you're reading/writing data which is much larger than the available cache.

In the case of dd, the real reason why it's often a misleading benchmark is that it's the best-case scenario: full-size streaming I/O where each block is only accessed once. Some applications do have that kind of operations pattern but most don't and can see very different performance characteristics because they depend on things like file creation/deletion overhead, random I/O within an open file, read and write contention if the same file is being accessed by multiple processes, etc.

The only thing worse than dd is bonnie++. Please, folks, use fio, or at least iozone, with multiple threads and/or a queue depth greater than one.

Bonnie++ is heavily CPU limited and should not be used for disk benchmarking