Yes, the cost is very high. But the cost of an good software engineer at Google, Facebook or another top tier firm is also very high.
However, when Google buys a WAN link from NYC to CHI, they pay $XX dollars because they don't care about the latency too much. When a HFT buys that same link, they'll spend three times that much just to get a link that is a few milliseconds faster.
Also, the colocation costs at the major exchanges (NASDAQ, NYSE, CBOE) are insane. A cabinet in one of those data centers can be 3x as much as a cabinet somewhere else.
I guess what I'm saying is there are a lot things in the HFT world where you have to spend a lot more money than you would if you were doing similar amounts of work at any other tech firm.