Hacker News Reader

Comment by qalmakka | Hacker News Reader

This. What is the point of using a higher level, garbage collected language instead of C++ or Rust if you're going to spend the same amount of time fighting against the VM or the Garbage collector to make it actually work the way you want it to? It feels like it's just for the sake of avoiding any kind of exposure to a newer language.

View on HN

With Java vs C++, the experience seems similar. In C++, memory allocation is slow, so you worry about memory allocation up front. In Java, GC is a pain so you worry about memory allocation up front. As Daniel points out, there are a few extra tricks available to native developers (eg vector instructions), so there's a small factor speed-up in best-practice C++ vs best-practice Java, but this is closing (eg the JIT increasingly uses vector instructions). Overwhelmingly the difference between a slow system and a fast system is appropriate choices of algorithms and architecture, not the language.

>In C++, memory allocation is slow, so you worry about memory allocation up front. In Java, GC is a pain so you worry about memory allocation up front...

This.

A lot of people seem to be willfully dismissive of this reality when they are fanboying for their favorite languages.

Ask for memory in C++, you'll take a HUGE hit getting it. Ask for memory in Java, and you'll get it lightning fast, but you'll take a huge hit getting rid of it. Either way, you should carefully plan memory usage up front.

Basically, one road leads to death and despair, the other leads to disease and destruction...

pray you choose wisely.

Any moderately performance sensitive code in C++, assuming it needs allocations at all will use a dedicated allocator.

While it is hard to beat existing general purpose allocators in all the scenarios, it is easy for specific use cases.

True but these approaches in both C++ and Java start to converge: Arena allocation vs Pooling/unsafe. The question is, how annoying is your performance insensitive code?

I can think of a couple of reasons; Java code is portable across OS'es; also while this code reads like C++, it might be part of a larger system with less performance-critical idiomatic Java.

I'm not sure that portability is a good reason to use Java in this case.

Firstly, these trading firms probably have tight control over their server architectures. Secondly, once you start using low-level Java idioms in order to optimize your code, you've started approaching the point where differences in performance between JVM implementations across platforms become noticeable.

The GC isn't the feature you want though, it's the JIT compiler.

JIT compilers optimise based on the actual runtime characteristics of the running code, not the best guesses that a static compiler has to make. The effect can be significant. It's no surprise that the Java leaders in Techempower benchmarks match or surpass the C++ ones.

You can use profile-guided compilation with an AOT compiler, too-- Or you could just add optimization hints to the performance-critical parts of the code. Either way, the compiler doesn't need to "guess"

Also one of the advantages of JIT is that it can take advantage of whatever dedicated instruction is available on the current CPU, while an AOT compiler (discounting runtime dispatching) need to target the lowest common denominator.

In this case though you would know exactly the target hardware.

On the other hand, the JIT has to balance the runtime of its optimization passes with the expected improvement in execution speed of the code. An AOT compiler can afford much more expensive optimizations.

I've talked with HFT firms that spent effort fighting the JIT because they cared more about the latency of rarely reached code paths than the common path.

Java doesn't beat C++ in raw computational benchmarks. The JIT compiler is a marvel, but it doesn't beat peer AOT compilers.

The Techempower benchmarks mostly show how good the entrant's web-serving stack is. Java has a couple of decades of multiple skilled, well-established groups competing to build the fastest web stack. C++ has Boost.Beast, which, although its developers are smart, wise, widely respected, sexy, and moderators of a Slack i frequent, can't compete in terms of resources.

I think you're overstating the importance of JIT vs AOT.

There are a few AOT-compiling JVMs out there, and it's not the case that they reliably outperform the JIT-based JVMs like HotSpot.

The significant differences are in Java mandating array bounds checks, runtime checking for null, no undefined behaviour, etc.

Maybe one already has a heap of Java code and the latency-sensitive part is only a tiny fraction where optimization will be focused.

Completely agree with that 90% of the code I usually work with is not of the 'hot path'.

Too bad the 90/10 rule doesn't apply to size.

Whereas 90% of the time might be spent in 10% of the code image, no 10% chunk of the image can ever be found that contributes to 90% of its bloat.

1% of the image contributes 1% to the bloat, 10% to 10% and so on.

The point is development time. Even in time critical applications the majority of the code is not time critical, you only optimize the small part that really matters.

The advantage over C++ isn't about the language itself. If your datastores all have functioning java connections, and your data modeling is already all done in java, it can be worth the pain to do performance critical code in java. I would rather do it in C++, but sometimes the most rational path forward is ugly.

Rust's future is far from certain. I would feel irresponsible using it on a project that wasn't self contained.

Pareto's law. You optimize the few parts that matter speedwise, and the rest you can do in ordinary "higher level, garbage collected" fashion. In C++ you can't do that. This is the standard argument for Lisp type-optimization (DECLAIM, THE, etc.) as well.

> In C++ you can't do that

What?

Many people are saying that cpp benefits are cancelled by all the pointer/template voodoo required to get them.

Now I'd be happy to see high perf low latency talks about rust, go, ocaml, haskell, sbcl, whatever. I believe limit-cases like these are always very very instructive.

As a very long time Haskell user, I wonder how much latency management would be controlling for the lazy evaluation thunks being evaluated at certain deterministic times.

In my gut I would not choose Haskell for my first choice for a low latency language. I would definitely have looked at Rust before Haskell.

Of course, but still, it's interesting to see how far lazy fp can be reasoned about on this topic.

I always love optim talks whatever the language (granted its not a clusterfk interpreter).

Haskell abstracts away order of operations. Both high performance and low latency benefit from the ability to be very specific about as much as possible and Haskell take the opposite approach. This may be good for data flow and throughput, but as soon as you need to control explicit order, state and resource usage Haskell is going to be working against you.

It seems to me that part of high-performance computing is optimizing memory layout (so you can vectorize, and to avoid cache misses). I have a hard time seeing that even in Java; I have a really hard time seeing that in Haskell. Am I missing something?

As is usual in Haskell for this kind of question, you can theoretically capture the information at the type level, and there are good odds somebody is working in a GHC extension for that.

But on practice, I don't think you are missing anything.

There are people with good results for high throughput Haskell (worse than C, not by much, with code almost as complex as C). But for latency, the GC alone kills it.

It sounds like D is in a good spot for these usages. It's fast, and you can nicely control the usage of GC - GC only gets called on allocations, and you can use @nogc attributes to statically confirm there are no allocations in the hot loop.

At least for Rust, the raw compiled code performance isn't quite there yet.

You can pick languages other than C++ or Java, but for a specialized and bespoke hardware/software combination like HFT, you'll be at a disadvantage vs other teams.

In my experience, or at least something I've seen at 3 different firms now, there's also an issue around the kind of people that pick exotic languages in terms of pragmatism. Often times the Scala/Haskell/whatever else side of the codebase is beautiful and slow moving while the hacky C++ side actually works and is agile.

Compared to the typical functional language applications, Rust is different in that it's mainly being battle tested in the kind of Firefox performance issues that are very interesting to an HFT programmer and Rust easily interfaces with C.

What kinds of performance issues are you running into? We’d love bug reports!

No real world issues yet. Every couple years I look up some Rust performance benchmarks and pass on it. Benchmarks like this: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I'm not sure if it's still an issue since I don't use Rust daily; but one recommendation I'd make concerning array bounds checking is to do something like electric-fence and allow the compiler to be told to allocate arrays to end at the page boundary such that the hardware interrupts when you go off the edge.

If the compiler restricts such an array to only being indexed with an unsigned integer, then it can be safe but forego bounds checking altogether since the lower bound is always 0 and the upper bound check is hardware accelerated.

Thanks for elaborating!

Yeah, a lot of those would be faster now, because SIMD was stabilized. That's the source of the discrepancy on n-body, for example. I don't think anyone has bothered to really submit new implementations.

Most bounds checks can be hoisted out of the compiler can’t figure it out on its own via asserts. In general it’s not a huge problem in real-world code.

> That's the source of the discrepancy on n-body…

You made that claim before, and I showed you the Rust n-body program "vector instructions using SIMD" contributed 5 months ago.

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Edit: the correct URL is https://benchmarksgame-team.pages.debian.net/benchmarksgame/....

Which one of those is the right one? Number 5?

It shows 0.48 seconds, and 2 seconds, but https://benchmarksgame-team.pages.debian.net/benchmarksgame/... shows 13.27. Why isn't that one on the main page?

> Which one of those is the right one? Number 5?

HN have now corrected the URL, as I asked.

> It shows 0.48 seconds, and 2 seconds

It shows 3 different workloads:

	 secs	         N
	 0.48	   500,000
	 2.01	 5,000,000
	20.09	50,000,000

> … but … shows 13.27…

Which we can see is the time for `n-body #2` at the largest workload:

	 secs	         N
	 0.54	   500,000
	 1.33	 5,000,000
	13.27	50,000,000

> Why isn't that one on the main page?

I don't know which you consider to be the "main page".

We can see that measurements for `n-body #2` are shown on both:

    faster/rust.html

and

    measurements/rust.html

Okay, I see now. It's about the three different measurements. Thank you!

Memory safety is a pretty huge one! Better tooling, too.

You heavy optimize only the critical/hot parts, similar how C developers sometimes optimize the code by using assembly in a critical section.

I am waiting for an article on hn how a high frequency trading vompany has rewritten their stack in rust because of the threading features.

Unlikely any time soon - most HFTs will have massive, battle-tested C++ code bases already. The data ownership model in HFT is usually very clear - receive packet, pass through some processing pipeline, emit packet, on the same thread. If you want to send data into another part of the system (eg. because different threads handle different securities), posting a message onto their event queue works well.

Not happening.

Rewriting some hot path parts of the stack in Verilog is a usual thing, and rewriting some other parts in Coq is the future.

Is it possible to synthesize hardware from Coq?

Not entirely on topic, but Jane Street (HFT ETF market maker) wrote an OCaml to FPGA compiler

https://www.janestreet.com/tech-talks/ocaml-all-the-way-down...

Yes.

https://deepspec.org/entry/Project/Kami http://conal.net/blog/posts/haskell-to-hardware-via-cccs

The threading feature that are not stable / experimental? There is not way someone is moving to Rust for this.

C++ can be a higher level, garbage-collected language.

http://www.stroustrup.com/bs_faq.html#garbage-collection

Only theoretically. It is not possible to implement a GC as efficient as Java GCs for C++. Pointer arithmetic, placement new, and other low level features make it hard to determine pointers from non-pointers and to move things in memory.

"It is not possible" claims are very hard to substantiate.

Java is just someone's C++ program.

> "Pointer arithmetic, placement new, and other low level features make it hard to determine pointers from non-pointers and to move things in memory."

Real applications in a garbage collected language are mixtures of the managed code, plus unmanaged components (like foreign libraries).

Garbage collected C++ can work in much the same way: there is a framework for managed code that uses GC, and then there are unsafe parts that are analogous to foreign code. (Except, the managed code can much more easily and efficiently interact with this code than the typical FFI.)

There is more than one way to integrate garbage collection into C++, in any case.

> move things in memory.

Not all garbage-collection schemes move things in memory; only copying and/or compacting garbage collectors do.

Have you watched the talk? The point is that it requires less effort overall, and if you need to go significantly beyond that, C++ can't save you anyway and you'll use FPGA.

There's a good answer to your question in the talk here:

https://youtu.be/BD9cRbxWQx8?t=1025

Reply on news.ycombinator.com