Hacker News Reader

Comment by robert_foss | Hacker News Reader

Looking at the 99% latency and memory usage Java is pretty far from the winner. 99% latency actually matters a lot.

> 99% latency actually matters a lot.

It does, but the numbers are not too far. It's 3.5ms for Java versus 2.3ms for C++.

Even when running a lot of requests when 99% latency becomes the average one, that's like 1 millisecond difference, way under typical user's network latency.

You assume that this is being used by a user.

These numbers are meaningless unless accompanies by specific usecases - don't just choose java or C++ just because the benchmark says its fast or the best.

High frequency trading needs that extra milli second, but a batch job or backend computation won't.

Also if you want to do high frequency you probably don't won't GC.

...or will you? https://medium.com/@jadsarmo/why-we-chose-java-for-our-high-...

Touché :-)

'allows pauseless garbage collection regardless of the Java heap size'. I thought this is not possible.

Clearly it is, since the C4 collector manages to do it. To be fair though, the "pauseless" part does come at the cost of a slight slowdown in normal code execution since it instates read barriers to work its magic. This means the lack of pauses is paid for by significantly higher overall CPU overhead for garbage collection compared to a collector design that does include pauses. It could be worth it for some workloads where it is very important to have low 99th percentile response times though.

I'm pretty sure that Jane Street begs to differ. They're a OCaml company and they do very well on high frequency trading.

Does the ocaml backend do the high frequency trading?

As far as I understand, it does.

Milliseconds are at least 4 orders of magnitude too high for much of HFT.

And these number were when pushed to the max, when you are running at half the capacity if not lower it really shouldn't matter.

This! Benchmarks that fully sustains the throughput are rarely useful. If you’re running at that capacity your service is melting.

Latency histogram for fixed throughout (e.g. 1k RPS, 10k RPS, 50k RPS) are far more useful. You want to know if going from 1k to 10k increases latency.

With Java you'll hit GC-breaks even under low load conditions. If we were to look at the 99.5-percentile, java would look even worse.

This is a classic issue for Java vs compiled to binary languages such as C. Java does its own memory management thus allocates a big chunk of memory from the OS at startup. It also has garbage collection which runs from time to time dependent on what gc implementation is used.

These issues can be mitigated by carefully tuning the parameters of the Java virtual machine, but in practice, for most projects, it is not an issue.

You are correct. Some of this has changed significantly somewhere between Java 8 and 16. For example the Java applications we're running against Hotspot 16 regularly gives memory back to the host. It's GC pauses are also quite insignificant now for a microservice regularly allocating and GCing.

Seems like rust is actually the winner. Average latency is a meaningsless metric for performance. If not, why not perc20?

Throughput is a very valuable metric.

Ok, so let's compare it to batch-methods of delivery then.

No measurements are valuable all by themselves. None.

Even 95%, 90% or 50% would be better points of comparison than average. Average is nearly meaningless.

Average is the metric to look at if you have millions of requests to do one after the other, and care about total completion time.

Reply on news.ycombinator.com