"You can't get better by scaling out" disagrees with you. That's my point - if you scale out, it's more expensive.
They are optimized for throughput, not latency. If you want to do things in microseconds instead of anything from milliseconds to seconds you look at all the lovely code everyone has written in the last 40 years then throw it on top of the burning pile of cash.
I work in an area where microseconds matter and have to write everything from scratch in kernel mode C because nothing else cuts it.
Neat. Is that all for custom processors too?
However, when Google buys a WAN link from NYC to CHI, they pay $XX dollars because they don't care about the latency too much. When a HFT buys that same link, they'll spend three times that much just to get a link that is a few milliseconds faster.
Also, the colocation costs at the major exchanges (NASDAQ, NYSE, CBOE) are insane. A cabinet in one of those data centers can be 3x as much as a cabinet somewhere else.
I guess what I'm saying is there are a lot things in the HFT world where you have to spend a lot more money than you would if you were doing similar amounts of work at any other tech firm.
Getting latency in the micros with low jitter is extremely difficult and requires a bunch of custom software. And getting into the nanos requires a bunch of custom hardware.