Kotlin's hidden costs – Benchmarks | Hacker News Reader

Kotlin's hidden costs – Benchmarks

sites.google.com

171 pointspulse727 comments

Summary: most stuff is within 5-10% of Java. In performance sensitive code be careful of the spread operator and foreach on ranges which have higher costs.

Except the parts where Kotlin actually beats Java.

It was still within 5-10%, just on the other end of the scale!

5-10% is 3-6 months of Moore's Law.

If it still held. "The free lunch is over" [1], and has been for years now.

[1]:http://www.gotw.ca/publications/concurrency-ddj.htm

Are these results relevant though ?

Kotlin is also costly when it needs to create an object in order to pass a function (like in a non inlined higher order function). So I would expect a comparison of the number of allocations too.

From there it gets tricky because it really depends on the JVM you are running on in order to determine if this is going to generate enough pressure on the GC in order to have pauses, but that's a potential cost.

Also .. in the end the results are not that different while the 2 languages have very different capabilities. Delegates, higher-order functions, nullability, data classes, ... Kotlin has a lot to offer.

If I recall correctly kotlin by default compiles to bytecode of Java 1.6. So o be afair comparison you should enable kotlin to compile to 1.8. as it is bytecode anyway I would expect very similar performance. But comparing 1.6 with 1.8 it is obvious 1.6 would be slower.

The bytecode differences between 1.6 and 1.8 bytecodes are mainly invokedynamic and methodhandles.

If Kotlin doesn't take advantage of them when targeting 1.8, the bytecode version won't do any difference.

My point is that kotlin by default targets 1.6. You need to specify to target 1.8 if you want to use it. So for me the experiment is broken. You should either compare java 1.6 with kotlin or set up kotlin to use 1.8 to compare with the same java version.

Over time Kotlin's performance numbers will improve. For most applications though the small performance cost will be outweighed by the increased time to market and improved null handling.

*decreased time to market

(I assume)

Yeah, that's the go-to answer for slow runtimes...

In this case, "5-10% just isn't that much" may be the better excuse. Since Kotlin runs on the same jvm, it is a subset of Java, and therefore impossible for Kotlin to create code (and achieve speeds) that Java couldn't match.

In practice, Kotlin should asymptotically approach the performance of best-practice Java code, with possible performance improvements when it can correct unoptimised code, or implements new Java features without requiring any manual refactoring.

I agree with your conclusion, but it actually could be possible for kotlin to be faster than java for practical purposes, despite running on the JVM. For example, it might be more expressive, allowing the user to give the compiler more information for better optimizations. We see this when C++ is sometimes faster than C.

Kotlin code can be and sometimes will be faster than the equivalent Java code because Kotlin does lambda inlining on the frontend and does not rely on the JVM to do so.

That probably helps a lot, because it takes significant machinery in the profiling and JIT compilers to automatically eliminate the overhead of higher-order functions and the JVM can't always do it due to a problem called "profile pollution". Kotlin pushes a bit of the thinking onto developers, albeit aided by type hints, and then does the inlining itself in many cases. Especially helpful on platforms with weaker JVMs like older Androids (the upcoming Android update appears to have improved ART so significantly that it's now a serious competitor to HotSpot).

Whilst the article doesn't show any difference in lambdas I wouldn't expect it to show up in micro-benchmarks like this one because the profile pollution problem wouldn't occur. In any real app it would show up though.

Also when Kotlin inlines on the frontend it can also use reified generics in a few cases.

Finally, Kotlin provides features that developers can use to write more efficient code, I'm thinking about how easy it makes lazyness in particular. Whilst you always can write equivalent code in Java, it takes more effort to do so, meaning developers probably won't always bother.

"Finally, Kotlin provides features that developers can use to write more efficient code, I'm thinking about how easy it makes lazyness in particular. Whilst you always can write equivalent code in Java, it takes more effort to do so, meaning developers probably won't always bother." -> Agreed!

> Since Kotlin runs on the same jvm, it is a subset of Java, and therefore impossible for Kotlin to create code (and achieve speeds) that Java couldn't match.

This doesn't follow for several reasons. First, there are JVM bytecodes that Java doesn't use, so it would be more accurate to say that JVMLangX and Java both target subsets of the JVM. Second, higher order constructs could allow a JVMLangX-to-bytecode compiler to enforce checks at compile-time that would need to be done at runtime in Java.

Excellent news, Kotlin is getting to be a real option on the JVM.

I've never had to program so efficiently where this kind of thing would matter - only ever optimizing for readability and correctness in terms of bugs.

For people who do, do you enjoy performance optimization or does it get tedious?

Bit of both. I don't have to do it often and when you really need to press all the performance you can get out of some code it can be really maddening when there is no "obvious" reason. Quite often it is just an accumulation of many very small slowdowns which together are significant and then the measure, change, measure, change, measure ... loop can be very tedious. Sometimes, there are interesting things like changing the whole algorithm to something which performs better or reading up on bit-fiddling tricks which make some code remarkably faster (Hacker's Delight is not for the faint of the heart, but an interesting book nonetheless).

As time goes on you keep accumulating these details in your head. Then you just think about the low-level stuff the same way you think about the higher-level stuff, like algorithmic complexity, code structure, etc.

It does get tedious if you need to get something out of the door fast, but have to shave off another 30% of processing time per unit of work. Honestly, I just try to partition the working set and throw more hardware at the problem in these scenarios.

Really wish the author labeled the axes. Took me way longer than I care to admit to figure out what the graphs meant.

Isn't the relevant thing the relative value? What information are you getting from the absolute value which is the only thing you'd need a label for, given the author says taller is better? (We'd assume linear unless otherwise specified.)

I write performance sensitive code in Kotlin. Most of those issues are common to Java8 and be worked around easily.

if it's still faster than javascript / nodeJS, we're in a good place.

Clickbait.

It's the same.

Unless you care about a particular use case, when it isn't.

Measure, profile, refactor....

I felt similarly disenthused to realize the very small number of cases where Kotlin is slower.

The title is a response to another blog series called "Exploring Kotlin’s hidden costs". He is benchmarking the claims in those blog posts. Title Gore, perhaps. But not clickbait in this case.

Reply on news.ycombinator.com