Continuous delivery makes JVM JIT an anti-pattern | Hacker News Reader

Continuous delivery makes JVM JIT an anti-pattern

blog.astradot.com

32 pointsfoxgrover35 comments

Bad deployment system. You should do an A/B deployment and warm up the services/caches before the switch, so that the architecture handles the problem and not the programming language/VM.

At FastComments we run our E2E tests on the new instance to JIT the app. Before the Jit API calls can take 100ms, and after 10-30ms. Still, that is fast enough that most people wouldn't notice...

Also, the problem is not Java. Your application probably has way too many abstractions for a simple login page.

But why all the complexity?

There is a huge DI framework full of reflection and proxy classes, then a complex JIT to make the framework performant and now finally a scheduled warmup/load test phase to make the JIT work.

It seems like the same or better performance could be archieved with far less complexity by using an AOT compiled language.

Well I wouldn't use Spring, so that solves a lot of the problems... :)

I was never really a fan of DI.

Part of my point is your application should be fast enough before the JIT kicks in.

Ideally, you should also do performance testing. Incorporating that with your release go or no go is a great approach, IMO, if you do releases very often.

> with far less complexity by using an AOT compiled language.

Or use reflection free frameworks like Micronaut or Quarkus and enjoy the same productivity. The culprit here is no java, the spring framework.

> But why all the complexity?

Because it's probably a whole lot less work to add this new deployment mechanism than to completely change languages. Even for new projects there are tons of other things to factor in when picking a language; a crutch in one area can sometimes be the right call.

>Molding a 25 year old runtime ecosystem to adapt to AOT compilation feels like putting lipstick on a pig.

Other JVMs like J9 have had AOT support for decades now, its not "lipstick on a pig". There's plenty of material from previous JVMLS meetups about AOT.

Also, AOT support in itself is not hard. Making it performant is more difficult and Graal is truly remarkable/state of the art in many regards.

The article's "container lifespan" chart shows that 21% of containers live less than 10 seconds and 54% live <= 5 minutes. If the base data set has that many very-short-lived containers, it's probably not representative of containerized persistent services. How many services get CD updates every 10 seconds?

Indeed, the original Sysdig report that the chart comes from makes this case:

https://sysdig.com/blog/sysdig-2019-container-usage-report/

"Many containers need to only live long enough to execute a function and then terminate when it’s complete. Seconds may seem short, but for some processes, it’s all that is required. We believe the increased use of Kubernetes Jobs that run finite tasks like batch jobs contributed to this growth. In fact, we expect short lifespans to increase, especially on serverless platforms that are well-suited to running short term tasks."

Good find. So sure, the lifespan is distributed down there, but surely there are some longer-lived services that benefit from JIT. I think most times you're spinning up a container + a JVM for what is effectively a function invocation you are maybe sad about paying the startup time for the JVM. Use Graal, or a different language?

Kotlin methods are final by default unless marked open, and thus nonpolymorphic dispatch unless through an interface.

It sounds to me like the problem isn’t the VM it’s the insane amount of framework code you have to be initialized. This can be optimized with Java via class data sharing and snapshotting but if your code is all written in Kotlin and you toss Spring into the trash can In favor of say, Dagger or hand crafted DI, most of your problems would go away.

Besides, Kotlin/Native can compile and run without a VM.

See https://august.nagro.us/jvm-startup.html for examples of how to optimize startup without Prewarming JIT.

> Kotlin methods are final by default unless marked open, and thus nonpolymorphic dispatch unless through an interface.

This is a language construct and not enforced within the byte code. There have historically been reasons to allow final to be ignored, such as serialization. Instead the JIT will optimize and deoptimize based on runtime behavior, allowing it to cover cases where overriding is designed for but not done so in the actual runtime environment. There was a nice presentation on "truly final fields" at the 2018 language summit [1].

[1] https://www.youtube.com/watch?v=2HfnaXND7-M

It may not currently be enforced, but the compiler could enforce it, depending on assumptions, the same way we enforced it on the GWT compiler (of which I'm one of the engineers who worked on it)

With GWT, we had global information, so we could promote methods to static by class hierarchy analysis.

For a Kotlin class, even with dynamic loading, if the class is not marked 'open', final methods not implementing an interface, or private methods, can be converted into static methods.

e.g.

   class A(val x: Int) {
     fun print() = x
   }

can be transformed into

   class A(val x: Int) {
     fun print() = printStatic(x)
     companion object {
       fun printStatic(self: A) = self.x
     }
   }

And any call site with access to the original signature, e.g.

   val a = A()
   println(a.print())

could be rewritten safely as

   val a = A()
   println(A.printStatic(a))

Now, the current Kotlin compiler frontend or backend may not do any of these, but it could.

There's nothing in the compiler language itself that blocks such optimizations, especially if you're willing to limit reflection (as is the case with Kotlin MPP code), or admit a LTO pass.

This is about runtime binding vs static binding, not static objects or static members. There is still a "this", but the code that will run is known up front -- "ahead of time".

C++ went through this 25+ years ago: runtime binding, what in C++ is virtual functions, is a niche technique. Most C++ programs don't use it at all, or use it in only one or two spots. When it is the right thing, it makes the work convenient, but it really is just a dance with function pointers. In C++, templates do the heavy lifting.

Java never offered any other support for organizing programs, so inheritance and virtual functions have been your go-to for everything, no matter how bad the fit. In a static call there is only one bit of code to run, and it never changes over the life of the program. Just like almost everything, really, except here your runtime knows up front.

It was always a dumb choice to make member functions default to virtual semantics, when they almost always don't need it, and it just costs performance to no purpose. That is what comes out of treating language design as a marketing exercise: Java's designers really (and openly) cared less than nothing about object-oriented programming. They thought people really ought to be coding Lisp. Forcing runtime binding was a way to sneak in something a little bit lispy, and maybe get people used to production code running no faster than Lisp.

Java is based on Objective-C, which is based on Smalltalk. In both of those all method calls are messages, which are even more abstract than virtual functions but a lot more useful.

Java simplified things for performance but ended up with a much weaker and less expressive system, and probably didn't go hard enough for static performance either. Still, it's less of a performance issue than the lack of value types.

> It was always a dumb choice to make member functions default to virtual semantics, when they almost always don't need it, and it just costs performance to no purpose.

If you actually don't need it (i. e., your hot methods are never overridden), then the JIT will trivially compile those "virtual" method calls as non-virtual ones. It has all the information it needs, since it knows what classes are loaded. It can invalidate its code if necessary, if you load more classes that do add overrides. So no, it does not cost performance at the call site. It does cause the compiler to do work, but nothing fancy.

> If you actually don't need it (i. e., your hot methods are never overridden), then the JIT will trivially compile those "virtual" method calls as non-virtual ones.

But isn't that the thrust of this article? Of course the JIT can optimise a monomorphic call-site. The question is, in reality, what percentage of the time will it be optimised for your users?

What the article doesn't acknowledge is that there are many ways to force compilation of your code. You can change the threshold for the number of executions before the JIT is called. You can run scripts as part of your deployment that exercise the paths you are interested in. So yes, it's true that "this code runs very infrequently, but I still care about its latency" is not what JVMs might be optimized towards. But it's possible to understand the issues and solve them.

To need to understand and solve issues is always worse than not to need either.

Java got commercial success despite its many design flaws, not because of them. $1B+ promotion from Sun helped some, providing a route to freedom from Microsoft sharecropping helped more.

And from being way better than either C or C++ to write distributed applications.

Having done that with C across the major UNIX flavours around 2000 and later again with C++ and CORBA a couple of years later, it is kind of obvious why most enterprises moved into it (and its nemesis, .NET).

You can even aot compile a class and have the cached “JIT compiled” code loaded on startup.

https://docs.oracle.com/en/java/javase/13/docs/specs/man/jao...

Others have raised good points, but seriously something is wrong if you’re pushing updates that require restarts every hour.

I get the idea of “continuous deployment”, but this is sounding like restart-per-commit. At that point I question how much qualification and validation is happening. No one say unit tests, they aren’t sufficient, and I’ve worked on multiple projects where the pre-commit test suite runs alone can take more than an hour. Even with those tests there are semi-regular breakages.

> 74% of containers have lifespans ≤ 1 hour

This sounds like a design flaw in the architecture. You don't tear down a house to replace a lightbulb.

Nor do you spent 10 hours with a magnifying glass and tweezers to attempt to change the filament on a burnt out light.

Which analogy you consider spurious likely depends on what kind of theatres of IT war you've been in.

I don't know if it's necessarily BAD architecture, but it certainly isn't typical in my experience. Most of my JVM deployments, even in containerized environments, have lifespans measured in AT LEAST hours, if not days. And the difference in performance between the first request and the requests occurring 30 seconds later is stark.

> Production Java apps also typically run with APM tracing agents that rely on runtime bytecode instrumentation. [...] It is easier to start afresh with modern compiled languages like Go and Rust.

I wonder how they instrument their Go and Rust programs. If they decide not to, maybe it's not that important for the Java version of the same code either.

No idea if this is done, but couldn't you in theory add the instrumentation before or during the compilation?

Something like a tool that runs as part of the build process and modifies the source code before it gets passed to the compiler.

I'll take mature instrumentation that comes in the box and is a major design goal, meant to be transparently run in production (JVM, BEAM...) against anything you can do in Go.

Every serious Java based Web App I've ever worked on had a prewarm step or continuous probing setup for exactly these issues. It isn't unique to Java either - lots of languages have lazy library or dependency loading, or runtime caching of external files or the like that you want to do before you start serving requests.

Or a cache to be loaded. In related news, "Continuous delivery makes caching an anti-pattern".

Not really. You move cache off-heap in Java terms and it persists it between restarts. One Russian social network has about 60 gigabytes of stuff cached that way

> Go and Rust encourage use of static method calls

Not having worked in Go or Rust but having done a lot of work in Java and C/C++, I'm curious how this works out in practice. My experience with developers who default to static method calls rather than objects in Java or C++ is that they also default to static (and therefore global) data as well. Of course, you don't have to do that: you can pass pre-transaction data structures to the static functions and let them operate on them, but that's what object-oriented programming is for in the first place.

Rust is not object-oriented.

You can make "virtual method" calls in Rust (dynamic dispatch through a "trait object", a vtable). This is explicit though, the type will look like `Box<dyn MyInterface>` where Java says `MyInterface` (box = object allocate on heap, dyn = virtual method calls).

I think you might be misunderstanding what "static method call" means: in this context, it means a method which is not "virtual" (in C++ parlance), not a method which is "static" (in C++ parlance, e.g. not called on an object).

You're correct, I was assuming that "static" in Rust & Go meant the same thing it meant in Java (which is different than what it means in C, of course). Thank you for clarifying.

>2 hours till the JVM C2 compiler is able to fully optimize

That's really hard to believe claim with the request throughput you have.

TL;DR; A company that sells Go consulting services bad mouths JVM JIT while completely ignoring JIT caches and AOT compilers that exist since around 2000.

Reply on news.ycombinator.com