Hacker News Reader

Comment by kirillkh | Hacker News Reader

Kotlin is one fine piece of engineering. It's only too unfortunate that the same Achilles' heel is holding it back as Scala: there is only so much you can do while remaining compatible with JVM and the Java standard library. Actually, it's even worse: Kotlin swore a seamless compatibility with Java. Maintaining compatibility against a moving target has the potential to make things quite messy. An early example of that presented itself just after Kotlin 1.0 release, when Java 8 came out with with new and shiny functional Collections API - incompatible to Kotlin's.

View on HN

The new Collections API wasn't incompatible with kotlin, it is just you can use the new API if you target Java 6 (which makes sense). But you can use the new API if you target Java 8. So I don't see a big deal here.

The problem is not the compatibility with the new Java API, but with the new bytecode definition. But it is going to have the same problems Java have. The big difference with Java here is that you can adopt new bytecode specifications without using the new Java APIs, or doing some modification in the bytecode generation to "simulate" those new features. And this is the key why it is taking over Android community. You can support legacy bytecode with new features. Which you cannot do with Java.

A problem for kotlin (and java) is the new packaging system. But kotlin is already working on it changing how they package their standard library. So it is even less problem than java.

> The new Collections API wasn't incompatible with kotlin

There definitely is a problem, as can be seen e.g. here: https://discuss.kotlinlang.org/t/jdk-8-compatibility-of-kotl...

Quoting: "Sometime ago we introduced kotlin.Stream<T> type, with the simples possible interface – iterator() function – with the idea that we can make it compatible with Java8 Stream<T> type. Currently we seem to come to a conclusion it was a mistake as we cannot fine a way to represent Stream<T> as kotlin type in Java 6 & 7, and then use to Java8's Stream<T> when targeting JDK8. If you have any idea about how to make it work – please share!"

Basically, while Kotlin is more or less a straightforward extension of the Java 6'th Collections APIs, by Java 8's release the two have diverged. When programming in Kotlin, you would normally use its own functional APIs, not Java Streams API. While technically nothing prevents you from using Streams, they don't fully integrate with Kotlin's native Collections, so you're forced to choose one or another.

That more or less what I said, it is not incompatible as you can use the Java stream API if you target Java 8. But yes, it is a pain when you see you have to use the Java Stream API instead of the kotlin Stream API.

Not incompatible but an issue. And they seem to have resolved it. Look at last comment. What you are quoting is from December 14th, before even the initial 1.0 release of kotlin.

Like I said, Kotlin is chasing after a moving target. The issues might be kinda-solvable (even if it requires significantly diverging from Java's APIs), but things inevitably get messier both for the language internals and for users.

> when Java 8 came out with with new and shiny functional Collections API - incompatible to Kotlin's.

This makes no sense.

Kotlin has no collections, it just uses Java's.

He/she means the changes related to streams, I would say.

Kotlin has an extensive Collections library, which started largely as an extension of Java's, but now it offers much more. In any case, I was referering to Kotlin's APIs, not Collections per se, and it certainly has its own APIs.

> It's only too unfortunate that the same Achilles' heel is holding it back as Scala: there is only so much you can do while remaining compatible with JVM and the Java standard library.

Being part of the largest, most successful, best tooled ecosystem in software is an Achilles' heel? Most people would (and should) gladly take that over any particular language feature.

I'm really not proficient with the jvm ( my only experience with it has been slow start time and huge memory consumption).

Could you give some example oF unique features of the toolsuite around the jvm ?

JVM can run "Hello World" in about 50msec these days. The java guys do care about startup time, you can see if you read their mailing lists that they are constantly trying to defend their startup time as new features come in that add initialisation work. It's a priority.

Still, the reason many Java apps may feel like they start slowly is that they tend to do a lot. The app I'm currently working on starts in about 5 seconds, which isn't terrible by the standards of the space but isn't great either. But the reason is that that 5 seconds involves initialising an entire message queue broker, an entire relational database and an entire web server. These aren't tiny 'embedded' versions either: they're Artemis, H2 and Jetty, which are both also used standalone and have cutting edge featuresets. If we weren't using the JVM then we'd presumably be using some container based solution and starting postgres, some other MQ broker and apache or something. Bringing up such a container from scratch could easily take 5 seconds or more I think.

The JVM will by default use all the memory on your machine. This is an unintuitive default, to put it mildly. The reason is that the JVM can make your software run faster by using more memory. Doubling RAM is easy. Doubling your CPU cores whilst keeping them actually used is hard, doubling the speed of an individual CPU core at this point appears to be nearly impossible. So it makes sense in a technical way. But it does make it hard to judge memory bloat.

Again, speaking about the app I currently work on, I got a support request the other week from someone who wanted to run the app on a Raspberry Pi and was complaining it took much memory. Quick testing showed we actually only needed about 50-75mb of heap and simply telling the JVM to limit the Java heap to that size slaughtered memory usage and resolved the problem. Not hard to check that and configure things for small devices, but it's a knob you can't/don't need to twist when using manual memory management, or GCs designed for mobile/laptop devices where minimising memory usage took priority over raw runtime performance.

Always good to hear H2 has cutting edge featuresets :-) (I'm one of the devs)

I'd like to take the opportunity and say thanks! Its a fine piece of software.

H2 is really nice. We did hit a bug the other day though. When write delay is set to zero compaction never occurs.

Honestly 5 seconds is an eternity when it comes to modern processors. I'm not convinced that it's a norm and I'm still wondering, what really happens, where that time goes?

"initialising an entire message queue broker, an entire relational database and an entire web server" is just abstract words. Initialization of an entire web server is creating a socket and binding it to 80 port, that's what should be done, but what else happens? Initializing a relational database is something like opening a file. May be read journal, rollback unfinished transaction, but those things could be done after initialization.

I use Java and I'm always trying to improve startup time. Last time I used undertow for tiny project, with jdbc to access remote database. Project started in mere milliseconds and it was much easier to iterate. But every time I enable something monstrous like servlets, hibernate, spring or, save god, java ee, it's always 30+ seconds start time. And I always wondered, why that's happens? I can serve millions of customers in those 30 seconds, I can read gigabytes of data from my HDD and parse millions of XMLs.

It's not so much the startup time (which is ~60ms), but warmup time. Assuming you're using a JITting JVM, it's compiling. There's now work on AOT compilation in HotSpot (and some JVMs already have it), but it really doesn't matter for big, long-running server apps, which are currently HotSpot's forté.

I haven't done startup profiling in detail but I'd expect most of the time to go on waiting for IO. CPUs are fast but opening a database, checking disk for queued messages etc is slow. Startup involves querying the database in our app, so the DB can't do that work after initialisation. There's also network traffic and RPCs happening in that 5 seconds too.

Without a doubt also some algorithmic waste in various libraries that aren't optimised for startup time. Startup time is rarely a metric that products compete on outside the consumer space, so improving it is not normally a priority.

And then yeah, finally the lack of AOT compilation. When Java 9 rolls around I plan to try and see how much difference it makes.

Sure. The ability to inspect what the system is doing and interact with it while it's running full-speed is unparalleled, and only pulling further ahead than anything else out there. Just take a look at monitoring tools like VisualVM, or Java Flight Recorder, which is part of the Oracle JDK and free for development use. You can inject and take out code at runtime, and the same monitoring and management API, JMX, is standard and used by everything from the JVM, the JDK libraries, various frameworks, and all the way up to the application code. With the proper tools, you can record everything your app is doing with very little overhead.

In addition, the GCs and the compiler are pretty much the best in the business, and the next-gen compiler, Graal, is probably the biggest breakthrough in compiler technology of the past decade. Apart from jaw-dropping optimizations, it lets you control machine code generation.

Last but not least, critical parts of the standard library (though not all of it) -- such as the concurrency packages, which contains schedulers and concurrent data structures -- are of quality that is hard to match.

The ecosystem is great, but it has its own limitations. For example, to my knowledge Scala has not been able to implement tail recursion optimization, citing JVM limitations.

Tail recursion is fully supported in Scala. The problem lies in more general recursion, i.e. two functions that call each other recursively will grow and eventually blow the stack.

BTW, there is a Scala compiler annotation that helps you check whether a function is tail-recursion optimized.

You're talking about two different "tail call optimizations" here.

Scala supports local tail call optimization for a single function. A lot of FP languages support mutually recursive tail call optimizations, where two functions recursively call each other (common in FP). Scala (and no JVM language to my knowledge) cannot optimize this and so programmers have to resort to workarounds like Trampolines.

In contrast, LLVM IR and .NET CLR support it, allowing language compilers to use it. F# supports it and there's plans for C# to support it. It might be supported in the LLVM Scala target (Scala native) at some point in the future.

Sure, and one day the JVM will get TCO just as it's getting value types (which is perhaps an even bigger current shortcoming), but that is a small price to pay for the ecosystem.

> Sure, and one day the JVM will get TCO just as it's getting value types

You're more optimistic than I am. And honestly proper TCO doesn't seem to be all that important to "regular" java programmers[1], it's just that it's crucial for $ALT_LANGUAGE implementers. (Where ALT_LANGUAGE has a functional flavor.)

[1] I'm guessing that this is the reason that it hasn't been prioritized... but who knows what complexity would rear its ugly head if one were to actually try to implement TCO on the JVM... I'd hazard a guess that it's 'highly non-trivial'.

The JVM is full of large projects that are only relevant to non-Java languages, like the vast bulk of java.lang.invoke, the Graal/Truffle projects etc. They've seemed serious about non-Java languages for a long time now. However/unfortunately, almost all the work has gone into support for dynamically typed languages like Ruby or JavaScript. Given the relative popularities of dynamic vs FP languages that's not surprising. If FP languages that relied on TCO were far more popular it'd probably have been prioritised already.

Reply on news.ycombinator.com