My understanding is that O(5 uS) is achievable on optimized FPGAs with reasonably large networks. Because of the parallelization, large networks don’t add that much more latency as long as you have enough gates. But I have little experience on FPGA stacks, so can’t say for sure.

Even in software, I’ve been able to hit O(15 uS) using optimized FANN libraries. But the nets are far smaller than deep, and pretty ruthlessly pruned and compressed. Another trick that helps is pre-differentiating across all the variables you don’t expect to change on a latency critical event. E.g. if you’re running a liquidity take strategy, you can pre-differentiate assuming the opposite touch size and deep book stays constant, because you’re only gonna act following on an aggressor trade at the touch.