The road to Java 9: Modular Java finally gets OK'd
infoworld.com
infoworld.com
Martijn Verburg here - London Java Community (LJC) representative on the Java Community Process (JCP) Executive Committee (EC) (dear Cthulu, the acronyms) otherwise known as the Java standards body.
Here's our reasoning for voting "Yes" the 2nd time around:
https://londonjavacommunity.wordpress.com/2017/06/28/our-yes...
This is a great step forwards for Java as a platform as it lays the foundation for a smaller, more secure and lightweight footprint. There is still more to tackle on the modularity story (expect to see further updates in Java 9.1 and beyond).
It's very important to note that you can still run your normal Java classpath based application with Java 9. The Classpath and Classloader mechanism is still there, although you'll see a few more warnings pop up if your application or its dependencies are relying on a deprecated API or dangerous internal lib.
If you choose to port an existing project to the new module system and module path way of loading then you can gain some of the benefits but of course there will be a learning curve and I'd argue that unless you're really needing that modular help then wait for Java 9.1 or even 10 until some of the edges have been ironed out.
Any other Q's throw my way!
What gives you this confidence? What kind of Java applications do you expect to run on Oracle could because they seem to target none of the existing ones? I don't see how Jigsaw can help existing Java applications with Oracle could:
They seem to be assuming nobody will use JAXB or activation either directly or indirectly because that will force you to deploy the full java.desktop module with all Swing PLAFs and everything including sound. They seem to be completely fine with this and do not seem to want to spend any time on splitting up java.desktop, having at least a headless version java.desktop or improving JAXB so that it doesn't depend on java.desktop anymore.
They seem to not care about existing Java EE APIs at all and have posted no replacements.
They seem oblivious to any needs of any dependency injection container (Spring, Guice, …).
Jigsaw only helps with disk footprint. It does not help with runtime footprint (which is usually several times larger than disk footprint) or startup time, if anything it makes both of them worse.
First and most important is module naming. Because Jigsaw dependencies work on modules and not packages (OSGi found out this was a mistake long ago) and because you can't make a module stand in for an other one the module name once decided can not be changed without breaking everyone depending on it. If you want to use any Java EE API in a Jigsaw application, e.g. Servlet API, you need to have a module name, even if you're using in Java SE. Right now if you're a Java SE application that uses any Java EE API at all you can't move to Jigsaw because none of the module names are set. Some Java EE specs use Java SE module guidelines because they say they are the only ones around (they ignore the ones from jodastephen) and then point the finger at the EE spec lead saying that since there are no guidelines doing what they do is fine. They also wrongly argue that they could change the module name later should naming guidelines be formalised.
Secondly a lot of the objections Red Hat made were because of the difficulties of making existing Java EE APIs run in Jigsaw, they were shut down with "you only want to promote JBoss modules".
And finally the current GlassFish 5 promoted build do not even start on Java 9 in classpath mode with all the internal API access checks disabled.
> Creating a module is relatively simple however. A module is typically just a jar file that has a module-info.class file at the root - known as a modular jar file. And that file is created from a module-info.java file in your sourcebase (see below for more details).
> Using a modular jar file involves adding the jar file to the modulepath instead of the classpath. If a modular jar file is on the classpath, it will not act as a module at all, and the module-info.class will be ignored. As such, while a modular jar file on the modulepath will have hidden packages enforced by the JVM, a modular jar file on the classpath will not have hidden packages at all.
[1] http://blog.joda.org/2017/04/java-9-modules-jpms-basics.html
I've read a few articles and they've left me very confused.
Is this about parts of the Java standard library (like the mentions of SQL and logging imply) and meaning it's only relevant for people targeting small devices?
Does it replace JARs, like the mentions of 'JAR hell' imply?
Does it add a maven-style package manager that can resolve dependencies by downloading code from the internet, like the mentions of npm imply?
Or are the changes / benefits something else entirely?
I think the breakage will be significant. I also think it'll be worth it.
> I also think it'll be worth it.
Yeah, it's fun to stick to abstract design principles, until an important dependency in your project refuses to play ball, in a way that cannot be trivially fixed by simply subclassing something and calling it a day. In a language like Java you essentially have three options - submit an issue and wait months (or years) 'till it gets resolved, fork the project now (if open source) and deal with costs of supporting your own version, or just poke at an appropriate private static variable somewhere and have a working version deployed in an hour. I know which one I prefer.
Because there are upsides -- a major one being proper implementation hiding and (not sure if this made it in:) "classpath" isolation.
> I'm not dumb, I understand the risk of depending on private APIs.
Well, you knew the risks, so I'm not sure why you're so upset that Java 9 precludes you using what were ultimately private APIs.
It's not possible to have it both ways.
Does this mean that things like jmockit's deencapsulation stops working?
In an ideal world, it makes sense. This is defensive programming, they're protecting themselves against unplanned usages of their libraries. That's always good when you're a maintainer. You know how annoying it is when you're trying to change and improve things and you realize you can't because someone was using your stuff in a completely dysfunctional way and now you'd break them if you proceeded.
But in a practical world, the world of business especially. You should also know that part of the popularity of your software comes from all those use cases users have managed to somehow satisfy without you explicitly planning and building them. Take those away, and your software also loses a lot of value, until you reimplement all these use cases in a more sane way. Tjis will take time, and your customers might move on to something else. We'll see how this impacts Java.
I think this is going too far. You have to jump through enough hoops already to mess with private state of library code; if some developers are dumb enough to do that irresponsibly and get hurt, let them. Don't take away a last-ditch tool those of us who know what we're doing sometimes really need.
I know the risks, but in this case, the private methods' implementation haven't changed, their signature hasn't, it's a completely artificial restriction.
That's where the frustration comes from. The code is there, you were allowed to use it, and now you are not, even though the code is still there.
If they had refactored, or oprimized, I would understand my code being broken. But this seems more ideological then practical.
It's no longer a breaking change for the library authors to break you if you're messing with private state, so you'll need to do some vetting of all changes to the library.
What do you mean by that? Maven JARs are just normal JARs downloaded with Maven. They are therefore valid in both the classpath and the module path (as automatic modules). Jigsaw modules are just normal JARs with a module-info.java, there is nothing preventing you form managing them with Maven. In fact in already have several of them in Maven Central, they went through the SonaType quality check without issues. In fact the compiler plugin will detect if a dependency in a modular JAR and if so put in the module path rather than the classpath (the javadoc plugin does not yet).
ETA: That being said, I think those who campaigned the hardest against Jigsaw with technical arguments were wrongheaded (e.g. "Just do it like OSGi, it works fine.")
The module system is taking the place of whatever you were previously using to order your dependencies on the classpath.
Now, instead, the module files list the dependencies of a module and, Java does a resolution of all of them to determine the dependency graph. So the order of modules on the module path doesn't matter, unlike the classpath.
The only advantage I know of is that the standard lib has been split in over 92 modules. So if you don't use everything, your resulting Jar will be smaller. Basically, making Java more modular in the sense that packages are now grouped into smaller Jars.
Now Java 9 has some other features, one of them is to prevent reflection access of non public methods. This was made optional for Java 9. But might be made default in 10 and mandatory in 11. This is the biggest issue people have with Java 9 modules, because it will not be backwards compatible, and certain code can't even be ported, since no public APIs exist for certain things.
Finally, most programmer at the cast, like me and you, we're hoping that modules would be version aware, and allow multiple versions of the same module to coexist, but they didn't do that unfortunately.
I'll admit that I'm not super well-versed in Java packaging, but I kinda assume that typical jars do not include any bits from standard library, so modularizing standard library wouldn't have any impact on jar size. I'd imagine that the effect would be instead visible for self-contained distributions, like e.g. docker containers.
That is correct.
> I'd imagine that the effect would be instead visible for self-contained distributions, like e.g. docker containers.
In theory. In practice you need to build a JDK contains only the bits you need. Sounds great in theory, you can get rid of the CORBA and GUI modules, that should save you a lot of disk space. In practice the CORBA module is only 2.7 MB and the desktop module is 14 MB and almost every framework indirectly requires the desktop module because Java Beans are in there.
Basically, if you the developer weren't using OSGI before, and planning to switch to java modules, then likely the only effects you will see are negative ones.
This is really for the benefit of the jdk itself, and tiny-platform installations.
The threat of java 10, and onward -- breaking stuff is the real thing to watch out for.
Most of these are proprietary modules.
> $ls $JAVA_HOME/jmods/ | grep java | grep -v javafx | wc -l
Return 30 on my machine (build 175). Most of the 94 modules listed in my installation are internal JDK modules.
Really? Well, that sucks.
In a large and complex project I'm currently working on, we've embedded a clojure REPL and we use reflection to poke at private data/methods of internal objects. Its been extremely useful for debugging and monitoring the state of the system. If we lose the ability of poking at private stuff, then we won't be able to do this anymore (unless we make everything public or add accessors for everything).
I'm not sure if it will impact the Clojure REPL. I know Clojure is aware of the Java 9 changes, and I think they have some ideas how to keep things working somehow.
There's also going to be a Java Repl now, called jshell. ButI don't it will be given special permissions such as that.
For example, we have a function 'inspect' that takes an object and series of Clojure keywords - it converts the object into a clojure map where the keys are keywordized member variable names and the values are its value. It then acts recursively consuming the series of keywords to look of the fields. This makes it trivial to look into deeply nested object graphs. (It also does some other things like pretty printing the results, accessing containers by index etc)
BUT this is only useful because 'inspect' sees all fields: public and private. It does this by using the reflection API to access private fields.
(inspect rootobject :someField 0 :privateField :foo)
rootobject.someField[0].privateField.foo
or (depending on the type of someField):
rootobject.someField.get(0).privateField.foo
or if, for example, privateField is a map:
rootobject.someField.get(0).privateField.get("foo")
...etc
You could visit these links to know more from a web-dev backend point of view.
* https://www.oreilly.com/ideas/modules-vs-microservices
* https://www.youtube.com/watch?v=fDoiydWS3D4
* https://www.youtube.com/watch?v=O77777Zy_HE
Ps... I created a library in Nodejs that implements concepts of modularity in JS codebases. Its known as archiejs (on github). I think modularity is a very useful feature that lot of people are going to want to use in Java (and also other programming languages).
How about the whole module versioning thing? I get the impression that the real discussions and negotiations happen in private and even the somewhat technical messages in the mailing list are only the tip of the iceberg.
I liked this comment about #MultipleModuleVersions:
"...Addressing this issue may entail reconsidering the multiple versions non-requirement. [Mike Hearn]
"Resolution Defer a specific solution of this problem to a future release since effective, though crude, solutions already exist and the lack of an immediate solution should not block developers who want to modularize existing applications. [Proposal]"
Module isolation should actually mean that you can have multiple versions of a set of classes (a module) available in the system (JVM process) and each gets their own classloader. Dependency expressions permit the deterministic wiring of these classloaders to each other... Simple.
Why wasn't the OSGi spec imported (at least in part) again?
The stated justification for the decision goes like this. With the current design of Java/the JVM, the only way to load two incompatible versions of the same module is to use different classloaders. You can in fact do this with Jigsaw. There is an API that can be used by an app loader or container of some kind to put every module into its own classloader.
This leads to an obvious question: if doing that lets you eliminate version conflicts, why not do it by default? The reason is compatibility. It turns out that lots of software makes assumptions about the classloader structure and which classloaders load it. They tried doing it that way early in Jigsaw's life and backed off because too much stuff broke.
It may seem odd to worry about breaking apps when Java 9 spent most of its development period unable to run basically any real Java software due to the widespread reliance on internal APIs, but:
• They backed off on that, one of the changes that let Jigsaw get voted through this time is that Java 9 no longer breaks apps that use internal APIs by default.
• Encapsulating internal APIs is at least a change that was always signposted as being possible - developers were warned not to use them in Java docs from decades ago. Whilst the classloader structure isn't something guaranteed by the spec (as far as I know), changes to it were at least not widely telegraphed in advance.
Ultimately, backwards compatibility concerns have won the day. So Java will not allow conflicting modules to be loaded by default. If you find yourself in a situation where you need that, there's an easy fix: change your app's startup code to use the Layer API to put every module in its own classpath and fix any modules you use that make now-invalid classloader assumptions.
Do you have a source for this? I have trouble accepting this argument for two reasons. First WilFly 7 and later (modular service container) completely changed the classloader layout to a non-traditional, non-hierarchical one. The amount of breakage introduced was negligible. In my experience the biggest source of breakage was read-only JNDI which is completely unrelated. Second they (Oracle people working on Jigsaw) were completely unaware of the amount of breakage encapsulating internal APIs would introduce and had to back out of it at the last minute (i.e. this year). This suggest to me they did at best minimal testing, they certainly never even started a hello world Spring or Guice application.
My recollection from one of many Jigsaw talks that Mark Reinhold gave at was they would have to implement a SAT solver (like OSGi) and didn't want to do that because reasons.
I think the Oracle guys were aware of how much would break when they blocked access to internal APIs, but felt the costs were worth it and/or that people would adjust their code during the Java 9 development cycle so the breakage would be less by the end.
Another cited reason was simply getting to the point of shipping something. Java 9 is already quite late. I can see why they'd punt features to future releases to try and ship what they have.
Jigsaw has always been controversial, and for good reasons.