Comment by lpghatguy | Hacker News Reader

Comment by lpghatguy | Hacker News Reader

Parent Full thread

An anecdote from a non-JS JIT, but similar: I once spent a summer working on a game engine with a couple others where the host language was LuaJIT.

It started out great. The iteration cycles were incredibly short, performance was competitive, the code was portable, and we could do wild metaprogramming nonsense that was still fast. If you haven't worked with LuaJIT, its C FFI is also incredible!

As we started scaling though, the wheels fell off the wagon. We'd add a new feature and suddenly the game wouldn't run at interactive framerates. One time, it was the 'unpack' function, which would trigger a JIT trace abort. We would drop from 12ms frames to 100ms frames. I wrote a length-specialized version that didn't abort and moved on.

Another time, it was calling Lua's 'pairs' method (iterator over a map). Okay, so we can't do that, or a few other things that made Lua productive before.

The other problem we hit was GC predictability being impossible. We tried to mitigate it by using native data structures through the C FFI, taking control over the GC cycle to run it once or twice per frame, etc. In the end, like the JIT problem, we weren't writing Lua at the end, we were writing... something else. It wasn't maintainable.

That summer ruined dynamic languages for me. I didn't really want to be writing C or C++ at the time. I ended up picking up Rust, which was predictable and still felt high-level, and the Lua experience ended up getting me my current job.

Thanks for the story details, quite interesting. In the end this was unfortunately a case of picking the wrong tool for the job. Don’t use JITed / GC languages when you‘ve got hard realtime requirements.

Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra). Don’t rely on auto-vectorisation in your inner loop if possible, one tiny change could bring that house of cards crashing down.

I‘d be interested in how your team ended up picking that tech stack. Was it a „rational“ weighing of options with pros and cons? Was it „eh it‘ll be alright“? Was it personal preference and/or prior experience?

> Don’t build a datastore on the JVM if you care about tail latencies, you‘ll be fighting the GC forever (see Cassandra)

OpenJDK's ZGC [1] is well on its way to have worst-case latencies of under 1ms (as soon as this year) on heaps up to 16TB in size.

[1]: https://wiki.openjdk.java.net/display/zgc/Main

This does not speak anything about end result. You might end up with 1000 GC collections per request. Each will be 1 ms, but total will be 1 second. It's still non-determenistic.

That's a question of throughput, not latency, and there's material about ZGC's throughput as well if you're interested. E.g. Go's GC has good latency but horrible throughput; ZGC is much better.

The Java developers have been promising to deliver a performant GC since the 1.0 days in 1990’s, and they’ve failed to deliver one 14 times in a row.

Also, 1ms is an eternity on modern hardware. That’s enough time to handle thousands of disk I/Os or network packets. That means each GC cycle can latency blip 1000’s of other machines.

That seems like kind of an extreme position. They've delivered the most performant GCs around, many of which are performant enough for a huge number of users.

Also, 1ms is an eternity on modern hardware

I hate to break it to you but malloc and free aren't instant either. Also ZGC can do less than 1msec pauses, often way less. At some point it just doesn't matter anymore for virtually all users.

That means each GC cycle can latency blip 1000’s of other machines.

Linux kernel timeslice is longer than 1 msec, so good luck if your server runs >1 process of any kind at all.

> and they’ve failed to deliver one 14 times in a row.

Failed at what exactly? People are happy with what he have now, and generally happier with every release (there have been some issues around G1 vs. the now-defunct CMS collector in some special circumstances). Netflix, Amazon, Apple, Google, and Twitter run much or most of their infrastructure on OpenJDK, without even using the newest collectors (AFAIK).

> Also, 1ms is an eternity on modern hardware.

That depends if you're looking at latency or throughput. Java's GCs have achieved good throughput a long time ago. But ZGC isn't a throughput collector; it is intended for online applications that aren't CPU bound. If you're serving transactions and your application pauses for no more than 1ms every few seconds you're probably OK. There are kernel subsystems that may cause a similar or higher worst-case blip.

In any event, Java is not intended for those who must get 100% of the performance the hardware can support, and are willing to pay a high price for getting there, nor is it intended for specialized, niche usage, where perfect performance is easy to achieve. It is intended to get to 95-98% performance at the lowest effort, for a wide variety of applications, and its main runtime cost now is in memory footprint. It's not intended to replace C/C++; we've got Zig for that.

I can't speak for the others, but I wouldn't use Google as an example there. Java is something like half the LoC but only 1% of the compute cycles (handwaived). It's very rarely used in the systems that do the heavy lifting. Though Go is growing on that metric, and also uses a GC.

Thats not exactly a fair characterisation. The Java GCs have been state of the art for a long time. Maybe they used to optimise too much for throughput, not latency, but that has changed.

That didn't exist yet when C* was started though. I'd be interested in how it compares to ScyllaDB, I suspect still not favourably.

Sub 1ms incremental GCs did exist, but they weren't common in production languages. They typically optimised for throughput rather than latency.

'pairs' was jitted recently by the RaptorJIT crew, fwiw.

I've hit fewer snags with LuaJIT, but they're definitely there. Really wish Mike Pall had written that hyperblock scheduler before retiring...

He did not retire; he is still quite active and improving LuaJIT [1].

    [1] https://github.com/LuaJIT/LuaJIT/tree/v2.1

Well, both are true:

https://www.freelists.org/post/luajit/Looking-for-new-LuaJIT...

It's more of an emeritus situation than anything, he's still active on the list as well.

I'd be ecstatic if hyperblock scheduling, or the quad-color garbage collector, were to drop onto the 2.1 trunk, but I'm not counting on it.

I'm working with Lua right now(gopherlua) as a scripting option for real-time gaming. I've done similar things to your story in the past with trying to make Lua the host for everything and I'm well aware of the downsides, but I have a requirement of maintaining readable, compatible source(as in PICO-8's model) - and Lua is excellent at that, as are other dynamic languages, to the point where it's hard to consider anything else unless I build and maintain the entire implementation. So my mitigation strategy is to do everything possible to make the Lua code remain in the glue code space, which means that I have to add a lot of libaries.

I'm also planning to add support for tl, which should make things easier on the in-the-large engineering side of things - something dynamic languages are also pretty awful at.

You might still run into GC problems, but none of the Go-based Luas (built on Go, rather than binding to another Lua library) I am aware of have a JIT built in.

GP is talking about LuaJIT, you're talking about Lua. Lua has lower performances but should be completely predictable, GC aside (not sure what its GC scheme is), so it's a very different situation.

Reply on news.ycombinator.com