Hacker News Reader

Comment by kccqzy | Hacker News Reader

Good story! I totally didn't know about the trick of changing what the JIT code produces bypassing JNI overhead.

> Never have I ever seen such a highly optimized Java codebase. Not before, not since.

This makes me wonder where it was. I know in the fintech world there's at least one hedge fund (not HFT, not market maker) where all their latency-sensitive trading code is written in Java, and when I asked them about GC pauses they said that their hot path produces no garbage so GC doesn't kick in.

View on HN

I too knew someone in HFT ages ago that spoke of using java even in the hot path. He told me they disabled GC altogether in combination with not generating garbage in the hot path, with large enough machines to not OOM before markets closed for the day.

From what I saw poking at the development trading boxes they over-provisioned both cpu ghz and ram to a ridiculous level. The machines looked nearly idle at their busiest.

It's a vicious cycle between provisioning more server and devs figuring out how to get more speed by using the resources for pre-computation.

> "large enough machines to not OOM before markets closed for the day"

Reminded of that missile defense system that was designed to be software-reset once a day, except the design assumptions changed (a war started) and it was ordered to be left running nonstop, as an emergency measure; after being left on for a week, failed, causing a large number of deaths. That one had some kind of integrator that accumulated floating-point roundoff over time.

> Reminded of that missile defense system that was designed to be software-reset once a day, except the design assumptions changed (a war started) and it was ordered to be left running nonstop, as an emergency measure

I'm sure you've simplified the story, but it seems like a bit of process failure for a missile defense system to assume peacetime. There's writing software that implements the requirements, and then there's making sure the requirements are right both up front and when you really rely on them in a critical way.

Patriot missile range gate. I remember covering that in a numerical analysis class.

https://en.wikipedia.org/wiki/MIM-104_Patriot#Failure_at_Dha...

I interpret that write-up to mean the daily reboot was a temporary user-suggested workaround for a bug discovered in the field, rather than something in the product specs from the beginning. And it makes more sense to me that no one realized the errors would accumulate or thought to test long operation in general than it would for them to have explicitly said it wasn't important.

Kind of like nobody thought to see what happened if Windows 95 managed to stay up for over 49.7 days [0], only deadlier.

[0] https://www.cnet.com/culture/windows-may-crash-after-49-7-da...

Also on the topic of missiles, it reminded me of Raymond Chen's classic Null Garbage Collector

https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...

Disabling GC is wildly inefficient without making sure you don't have garbage because allocation will slow down the application to human scale. The magic is more in the no-allocation code than in the disable GC side. It is very annoying in Java because any non-primitive is boxed type. Object pools etc. Problem is it's easy to start in Java, but then you hit a wall. JNI has huge overhead. Exasperating. But hard to argue, sometimes if you don't start in Java you never get the problem because you never manage to bootstrap.

Yeah it always felt like they were jumping through hoops to basically write C code through Java. I expect some of this might shift to Rust..

Well you do get guaranteed memory safety.

If they were mucking around with asm and custom jit those memory safety guarantees are gone.

I worked with low-latency trading with C# in the hot path; they had started with C++ and eventually migrated to C# and just kept tight control of allocations.

Later they’d go back to using C++ for training deep neural nets.

To avoid going back to C++, is exactly the reason why modern .NET has native support for tensors and various SIMD flavours.

Azul has a JVM that doesn't have GC pauses and has fairly good memory allocation / collection throughput.

    > Azul has a JVM that doesn't have GC pauses

Unless GC is disabled, all GC has some pauses. You can see the chart of Azul GC pauses here: https://medium.com/@jadsarmo/why-we-chose-java-for-our-high-... -> https://miro.medium.com/v2/resize:fit:1400/format:webp/1*yRG...

That's GC duration time but the article indicates:

"This is because Zing uses a unique collector called C4 (Continuously Concurrent Compacting Collector) that allows pauseless garbage collection regardless of the Java heap size (up to 8 Terabytes)."

Azul's GC runs concurrrently with the app so those GC durations occur without a pause. If you think about it, there's no reason why GC requires a pause in order to work, with the right conditions, GC should be able to analyze and remove dead allocations without stopping an app.

There's quite a few. Teza definitely does, but I've spoken to many people who have done this.

Reply on news.ycombinator.com