Well Moore's law is falling by the wayside, if they want to start doing more with less the software guys are going to have to stop using interpreted languages, GC, passing data as json rather than as binary, all that overhead that's deriguer but that doesn't directly go to getting the job done
"What Andy giveth, Bill taketh away"
But the main problem seems to be a lack of a clear architecture in many systems. These systems have often accumulated so much technical debt that nobody understands why they are slow. Profiling and optimization might remove the worst offenders but usually don't improve the architecture.
Basically, in the software industry, we use the hardware gains to cover up our organizational deficits.
You can write seven layers of lagging crap in c if you like.
- extra branching in the parsing code (the parser cannot predict anymore what the next field will be, they could be in any order)
- extra memory allocations, decreased memory locality (due to variable-length/optional fields, and also the tree-like structure).
So if your data consists in a single object composed of a timed-fixed set of little-endian integer fields, you're comparing the above costs to the cost of a single fread call* with no memory allocations.
* many other data formats provide similar flexibility, text-based ones (XML) and also binary-based ones (IFF, protobuf, ISOBMFF, etc.)
don't write it as such though, you must write the endianess-decoding code (which the optimizer should trash anyway on little-endian architectures - e.g LLVM does).