Comment by hliyan | Hacker News Reader

Comment by hliyan | Hacker News Reader

Parent Full thread

I was in this area from 2003 to 2013. Can't think of any reading materials, but it all boils down to ultra low latency (think < 100 microseconds within the local network) and high volumes (think > 30,000 transactions per second). The state of the art may have moved since then, but:

1) Network layer: raw TCP sockets with binary packed messages (sometimes structs directly cast to and from raw buffers, keeping network/host byte order in mind); separate message data dictionary to avoid metadata overhead inside messages.

2) Processing: optimised C++ code; pointers; multi-threaded processes that utilise all available cores; minimal locking and contention between threads (queues with spin locks and bulk transfers); threads directly bound to CPU cores (prevent context switching overhead); prevent heap locks by allocating large object pools at startup;

3) Disk I/O: minimise disk I/O in logic-heavy, low latency threads; application level in-memory disk write queues; prefer append mode writes; application level in-memory caches (basically hash maps of reference data loaded at startup)

4) None of the "trendy" programming practices: no OOP design patterns; no dependency injection; functional programming, but without fluent APIs;

5) No popular frameworks: internally developed solutions optimised for our specific use cases, instead of off the shelf libraries or frameworks (e.g. the messaging middleware was in-house)

There is more, but this is what I could think of, off the top of my head.

re (2) for execution code, its actually the opposite. Typically the execution hotpath is done on a single core and some places go as far as disabling hyperthreading to eke out as much single core performance as possible.

yes, no points of contention at all

userspace networking with one thread busy polling, all power state management + ht disabled with receive packet steering

To add to this, I think nowadays even C++ is too slow for some stuff. Many places use FPGAs

Having fast C++ code is always good even if its not directly responsible for execution, but yes typically order entry and trade parsing at really serious HFT firms is done with an FPGA/ASIC. Sometimes a trade packet wont even hit the software and instead be processed entirely on the FPGA.

Some neat tricks people have done is to actually process packet bytes in order as they come in on the FPGA, and if it has some actionable header they just send an action before the entire trade packet is received. With that you can go down into the few ns for your w2w latency.

When I left, they were experimenting with an FPGA implementation of the TCP stack (!). Obviously a very challenging undertaking, not sure how far they got.

Is C++ slow even if you don't use any language features that add runtime overhead, e.g. runtime exceptions?

It’s not so much C++ is slow as FPGAs are so incredibly fast.

The system programming languages (C / C++ / Rust) will always have more overhead than FPGAs.

A microsecond is fast for C++. It’s like an eternity for an FPGA.

Not C++, general CPUs are “slow”.

Interestingly, other than direct cabled network connections and trying to get ground stations physically closer to the satellites, which is obviously impossible, this is almost exactly what we did for first-level ground processing of geointelligence data for the NRO. Major difference would be data from the sensor platform doesn't even bother with TCP. It's just pure packed structs, but also encrypted with decryption handled by dedicated hardware devices. No need for PKI since it isn't public. The collection metadata from the tasking station still uses TCP.

Reply on news.ycombinator.com