Java 9: The State of the Module System | Hacker News Reader

Java 9: The State of the Module System

openjdk.java.net

122 pointspron57 comments

The cool thing is that compared to many languages/platforms, Java already has an excellent module system. Just look at the horrible hacks you have to go through in C++, Ruby or Python to manage multiple projects written in different versions of the language or with different versions of dependencies are just ridiculous, especially if you're developing on multiple platforms. Oh, you want to deploy it too? Have fun.

Let alone the IDE support. I think it's truly amazing that I can make a project with 3 different JVM languages and get instant IDE autocompletion and warnings when calling methods between those languages. The source -> .class -> .jar thing has turned out to be a remarkably visionary design (or just a very lucky hit). For comparison, C# still compiles and "tools" that much slower because .class files are missing in the pipeline.

It's a testament to some of the true goodness of Java, despite its unpopularity in hipper circles, that they don't think a Maven-managed folder full of JARs is good enough.

Of course, this is really a JVM thing and you'll also get it if you use a different JVM language so you can actually get the best of both worlds.

Are there any horrible hacks needed for Ruby to cope with dependencies? Is this something new or you're just ranting?

My experience with this is a bit dated so maybe they fixed it since, but here goes: Gems are installed globally on your system. You need something like RVM to work around this, but oops, RVM doesn't work on Windows and one of your devs uses that. There's a 3-year old tutorial about how you can get it to work with cygwin, but that doesn't work well with your favourite Ruby IDE, and the list goes on. You want to deploy to a server so you try to use bundler, but bundler hard-copies a dependency called somelib-mingw-x86 into the Gemfile.lock and your linux server chokes on that. In general, the idea that bundler makes a different dependency graph depending on the OS of the dev that happens to run "bundler install" is completely beyond my understanding.

I assume that it's a bit better if all your devs (and your servers) are on the exact same OS, but compared to say NodeJS or Java, the Ruby experience is pretty messy. With Node or Java, I just say "these are my dependencies", some package manager downloads them all into a subdirectory, and then I run the app. Done. The only global dependency is the VM itself, which is typically backward compatible so no problem there, just use the latest one. C# has it just as awesome except that some libraries were written by boneheaded Windows-only people (hardcoded backslashes in pathnames, Windows Registry dependencies, stuff like that), so in practice it isn't as cool.

Seems like you have never used bundler with .bundle/config file containing BUNDLE_PATH: vendor/bundle (or other subdirectory). That's it.

Practically speaking, Maven (the executable tool) and pom.xml (the language) create a hot dependency mess when projects get past the toy stage. Maven makes most simple things impossible and a few challenges trivial. "Hamstrung" is the word that comes to mind. Coupling a declarative, own-the-world, XML programming language with innumerable slap-dash/poorly maintained plugins is the pits.

All that said, maven repositories and the groupId:artifactId:version:type:classifier organization of modules are really genius. As you say, this genius shines in IDE's where, as just one example, you get nice binary/source .jar distributions automatically downloaded to your local repo. Having an IDE's debugger be able to seamlessly and accurately step through 1st party java sources and 3rd party source .jar sources is always so impressive to me.

Gradle (and Ivy before it) help paper over the pain and expose the good parts of the maven ecosystem. I have some gripes with Gradle, but after seeing it in action (and now that IDE's can grok its project configs) I'm so happy to be ditching maven/plugins and hand-written pom's for the most part.

Java needs a holistic dependency management/dependency injection/artifact repository/publishing workflow rethink where well-defined concepts like semver are enforced in the version numbering scheme, making it easier to express "safe" legal version ranges in transitive dependencies.

Because Maven (the tool) offers XML as the language, we can put any other language atop it, e.g. with https://github.com/takari/polyglot-maven we can use Clojure, Groovy, Ruby, or Scala as the language. Gradle offers only Groovy, nothing else, so we don't have any choice over how to interact with the build API.

I admit that maven on large projects can lead to some dependency problems, while not the final solution I found that Maven Enforcer Plugin[1] helps to reduce this mess, makes it harder to keep the dependencies because sometimes you have to exclude them everywhere and add your own by hand, but I don't see any better alternative, APIs change.

[1] http://maven.apache.org/enforcer/maven-enforcer-plugin/

After I first read your last paragraph, I wondered "how might you want to see semantic versioning get enforced?" ... but I think you are instead referring to the lack of version number ranges in Maven?

Yet another feature that Android devs won't get.

By the time Java 10 gets out, they can see how libraries that make use of modules, reified generics, value types, GPGPU integration, JIT plugins are out of reach.

Worse even, having library devs writing two versions of their libraries.

Google's fork of Java is leading to a Python 2 - 3 scenario.

Is there an actual JSR for reified generics? I was under the impression that would generally break VM compatibility.

I have a hard time believing Google will stop supporting Java on Android, especially considering how widely they support it for other various platforms. I do know Go support for Android was recently released though.

> Is there an actual JSR for reified generics? I was under the impression that would generally break VM compatibility.

Not yet. But it might happen, it depends how value types get implemented.

Check "New Bytecodes, New Objects", "Adventures on the Road to Valhalla":

http://www.oracle.com/technetwork/java/javase/community/jlss...

> I have a hard time believing Google will stop supporting Java on Android

Of course they won't stop supporting it. After all they stated at Google IO 2014, that only Java matters and they could not see why anyone would want to use something else.

But they also don't show any interest to reduce the gap with the official Java.

Even the actual version lacks many of the Java 7 libraries.

> I do know Go support for Android was recently released though.

It doesn't have any official support from the Android team.

Interesting, thanks for the links. Honestly, I tend to avoid writing Java in favor of native languages at this point largely because of latency requirements. I still wonder where code sharing outweighs the type cast overhead for most JVMs though.

> It doesn't have any official support from the Android team.

Go on Android definitely seems to be in its infancy.

Just as extra info in case you aren't aware, Java is also compiled directly native code on Android as of 5.0.

https://source.android.com/devices/tech/dalvik/index.html#AO...

And apparently Oracle is finally joining the third party JVMs by adding AOT support to the reference JDK, but it might only come after Java 10.

This together with the upcoming value types and JNI replacement, might eventually close the gap with the alternatives.

Not being a Java zealot, as a polyglot developer I use a bit of everything, just spreading the word.

I think Google probably is looking at Kotlin for Java replacement on Android.

That is completely orthogonal. Kotlin works on top of the Java ecosystem, and therefore enjoys the same capabilities and libraries enjoyed by Java -- which Android users won't get even if they use Kotlin. Kotlin code that uses Java libraries that use features that are not available on Andorid won't work on Android.

Or, you know, you can use OSGi (http://www.osgi.org/Main/HomePage)

One motivation for the module system is faster JVM boot time. OSGi doesn't help with that at all.

Except they don't mention at all how a module system should lead to faster boot times. Sure you could come up with faster lookup for classes by picking the right JAR but OSGi and JBoss Modules already do this today.

They have a protoype that does AOT compilation [1] but that's only a prototype and the gains are meager.

[1] https://www.youtube.com/watch?v=d4B8sc7ltZk&list=PLX8CzqL3Ar...

>Except they don't mention at all how a module system should lead to faster boot times.

They will switch from the basic jar (zip) format to a more optimized one. Additionally, with a modularized jre, it will load less modules (who use corba?). I'm quite sure there are additional reasons, but it's the 2 that I remember.

The JVM already only loads classes on demand so if you don't use Corba it doesn't have any impact on you (maybe minuscule extra seek time reading the JAR).

Clear, but the single rt.jar will be replaced with a more convenient representation, so, as you said, the seek time will be reduced.

The jar files used to be memory mapped, and as far as I'm aware they still are. So there should be a noticeable improvement in seek time by eliminating never-used files.

You could also perhaps be looking at a situation in the future where the JRE can selectively exclude things at install time, which would be nice.

> They will switch from the basic jar (zip) format to a more optimized one.

Straight from the article:

> A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.

> A modular JAR file can be used as a module, in which case its module-info.class file is taken to contain the module’s declaration. It can, alternatively, be placed on the ordinary class path, in which case its module-info.class file is ignored

So no.

That's why it's called a modular jar. There is also this "jmod" file described as:

"For the purpose of modularizing the Java SE Platform’s reference implementation, the JDK, we will introduce a new artifact format that goes beyond JAR files to accommodate native code, configuration files, and other kinds of data that do not fit naturally, if at all, into JAR files. This format leverages another advantage of expressing module declarations in source files and compiling them into class files, namely that class files are independent of any particular artifact format. Whether this new format, provisionally named “JMOD,” should be standardized is an open question."

Edit: from http://mreinhold.org/blog/jigsaw-modular-images

"The internal files rt.jar, tools.jar, and dt.jar have been removed. The content of these files is now stored in a more efficient format in implementation-private files in the lib directory."

> For the purpose of modularizing the Java SE Platform’s reference implementation, .... Whether this new format, provisionally named “JMOD,” should be standardized is an open question.

That just talks about the JDK. There is no promise that normal applications can use this format.

Use complex to achieve simple, that's Java Style, :P

Some realities are impossible to accept fully.

OSGi isn't that complex, it's more that modularisation is inherently complex and people try to wave that away. When thinking modularisation through you quickly end up wanting depencency managament, versioning, modules and dynamic updates (meaning no restart of the VM when updating a module), and combining all of this is inherently difficult to design, and some of the stuff the JVM wasn't really designed for from the beginning. OSGi solves all that in a quite nice way, but for political reasons Oracle doesn't want to use OSGi ('not invented here syndrome').

That's a naive solution. OSGi has a very specific target of embedded applications. Thanks to its classloader, its a headache to port existing libraries which weren't written with its classloader in mind. This leads to duplicate osgi safe versions in custom artifact repos.

In the end, not a solution, but a very nice environment and development choice if you are greenfield and in the target audience.

Doesn't solve the versioning problem. That's a little disappointing.

According to the article: "A module’s declaration does not include a version string, nor constraints upon the version strings of the modules upon which it depends. This is intentional: It is not a goal of the module system to solve the version-selection problem, which is best left to build tools and container applications."

I'm surprised by this as well. I was somewhat expecting Java 9 to tackle the dependency version problem.

I'm glad they didn't. Its not essential to solve the problems the module system is designed to solve, making adoption easier. A mechanism to resolve, download and cache artifacts is still needed anyway, at least I didn't see anything in that regard.

> I'm glad they didn't. Its not essential to solve the problems the module system is designed to solve, making adoption easier. A mechanism to resolve, download and cache artifacts is still needed anyway, at least I didn't see anything in that regard.

But, had they included versioning support, they could have also built upon that later to support module distribution to finally not have to worry about maven, ivy, etc.

The main difficulty about versioning though are snapshot (development) versions that don't have a defined version number. In maven, typically these are just "*-SNAPSHOT" which may not be unique enough if two projects wanted to depend on two different snapshots of the same module.

My reading is it's hard enough as is because of backwards compatibility. They seem to be under massive pressure to deliver something for JDK 9 and can't push the deadline much longer.

If memory serves me correctly I remember Mark Reinhold saying to do it "right" they would have to run a linear equation solver like Eclipse/P2.

It would seem that one of the JSR 376's key goals is to provide a saner way to manage dependencies than the Java classpath.

They say that but I don't know anybody who actually uses classpath (java -cp). Most people use a custom class loader, container and module system. In addition Oracle cops out of versioning. So I'm not sure who they are targeting.

The folks you work with seem to be more diligent than a lot of enterprise java folks I know. Explicit classpath is useful in deeply silo'd enterprise environment to get around multiple dependencies on incompatible versions of the same 3rd party jars. Often introduced while under the guns and then kept because maintenance dollars go elsewhere.

> Most people use a custom class loader, container and module system.

Huh? That sounds like a lot of work reinventing the wheel. Could you give me some details on how that's arranged?

The most extreme example is probably WildFly that has just the JBoss Modules JAR in the class path. That sets up class loaders for the modular service container that in term sets up class loaders for all the deployments.

Every Java EE server (even Tomcat) sets up its own class loaders. So does Jenkins/Hudson, Maven, all RCP platforms, even applets. It's really rare to find applications where all the code is in the class path.

Given that it was initially promised for Java 7, I'm sure you're right.

It is indeed a bit underwhelming, especially if you consider for how long it has been in the pipeline.

If you're talking about the inability to have multiple versions of the class, it is addressed, using layers: http://openjdk.java.net/projects/jigsaw/spec/sotms/#layers