Hacker News Reader

Comment by RobotToaster | Hacker News Reader

If you want determinism, wouldn't you use a RTOS?

For HFT you generally just shove the OS out of the way. It exists to provide basic access the machine and for the non-fast-paths, while the hot paths never touch it in the first place. You also aren't hard-real-time, where it's literally failure if you ever exceed a latency target.

In this context, Linux is fairly sensible as it has decent knobs for getting out of the way and is well supported for all of your non-fast-path-code.

For clarity: The HFT space does things like CPU pinning (the cores their tasks run on are cut out of the main cgroup into a dedicated one for their tasks). Likewise, memory regions are preallocated and static. Important resources like network stacks are done in user space - sure, the kernel has an IP address, for the 1g nic that's used for management, but the IB card that powers the trading interconnect (you are less likely to find IB these days, so listen to my story as a little dated) is mapped directly into userspace; there's no driver for it in the kernel.

Usually because the tooling for development on Linux is better and/or more familiar and they doing high frequency trading, not trying to fly a plane.

Some HFT systems are written in Java - they just give it loads of RAM, disable the garbage collector and restart it once a day.

It's like a meme in the world of trading: your system needs to survive for 24 hours, whatever happens after that - doesn't matter. This can really impact the way you design the system. Ignoring memory leaks in C++, or outright disabling the GC in Java is one of the legit trade-offs you can make.

Reminds me of the possibly apocryphal story of the missile whos guidance system had a memory leak.

They profiled the memory leak and gave the chip enough RAM that it wouldn't fill up before the missile exploded. Problem solved!

There are in service commercial planes that need to be routinely rebooted to prevent counters from overflowing, so I really wouldn't be surprised about similar sorts of issues in single use devices.

Imagine the insanity of allocating memory mid-flight.

Early in my career, before "turbo" or "boost" frequencies were a thing in commodity CPU products, I heard a similar tale about boosting the speed of the signal processors in a defensive radar system.

The thought was that you would rather overheat and perform the calculation than protect the circuit and lose the ship. It's kind of a hail mary strategy. It definitely gave me a visceral feeling for how one should be ready to re-frame their systems analyses, even if apocryphal.

That’s common enough there’s a term for it: https://en.m.wikipedia.org/wiki/Battleshort

Another technique used with Java is setting the memory really high, run it for a while against mock data until it's warmed up and the compiler has JITted out all the allocations and then move to real data.

If the mock data is good, you can do the whole thing in testing and fail tests if new code introduces GC events. However the Java code you have to write end up being very C like.

The notion of the JIT warming up is nowhere near as rosy as most people believe. In particular, it is neither true that the JIT will lead you to your program performing at a steady level, nor that such a steady level will be faster than what you started with. C.f. https://tratt.net/laurie/blog/2018/why_arent_more_users_more...

Though perhaps one could write a program in a way that gets reliably jitted?

Don't RTOSes usually sacrifice average latency for max latency guarantees? That seems like the wrong trade-off for HFT.

Reply on news.ycombinator.com