Improved Docker Container Integration with Java 10 | Hacker News Reader

Improved Docker Container Integration with Java 10

blog.docker.com

74 pointsel_duderino59 comments

I can't seem to upgrade to Java 9 still. Its the module system. Waiting for open source to catch up in some situations. See one of the problems is that the way Java is started needs to change to enable JAXB. But many open source products have lots of code that start Java processes, and never use that command line. Did this change in Java 10, did they bring back JAXB so things will start again?

Nope. In fact the module containing JAXB (and 5 other EE technologies) have been deprecated and will be removed in Java 11. So indeed, you can use the `--add-modules java.xml.bind` workaround, but it will break with Java 11. The real solution is to add a JAXB implementation of your choice as dependency to the application.

(disclaimer: I'm author of the O'Reilly book Java 9 Modularity (see https://javamodularity.com) which discusses this and other migration issues in great detail)

If you are using maven you can try following dependencies for JAXB from glassfish:

        <dependency>                                                            
            <groupId>org.glassfish.jaxb</groupId>                                                                                                                            
            <artifactId>jaxb-runtime</artifactId>                               
            <version>2.3.0</version>                                            
        </dependency>                                                           
        <dependency>                                                            
            <groupId>org.glassfish.jaxb</groupId>                               
            <artifactId>jaxb-core</artifactId>                                  
            <version>2.3.0</version>                                            
        </dependency>                                                           
        <dependency>                                                            
            <groupId>javax.xml.bind</groupId>                                   
            <artifactId>jaxb-api</artifactId>                                   
            <version>2.3.0</version>                                            
        </dependency>

As other commenter said, JAXB will disappear form Java 11 (and all other Java EE stubs that exist in Java SE) so it is better to upgrade sooner than later.

And BTW, I would skip Java 9 and go straight to Java 10 now.

Could anyone point me to the advantages of running Java inside a Docker container? It seems to be VM in VM. Why do we need the extra redirection?

It’s not a VM, it’s faster and far lower overhead per container. Obviously it also looses some of the isolation guarantees.

I see your point but Docker is not a VM, there is not a "redirection" cost or anything.

So you can see it as a convenient way to distribute the JVM. Say that the JVM makes your application portable wrt the OS, Docker makes the JVM itself more portable.

A container is not a VM (at least in the traditional sense). The advantages of running java inside of a container is that you can ship all of your things as one artifact, and run it on a bunch of different infrastructures, with all it's dependencies packaged together. So rather than needing to use something to ensure your properties are configured correctly, and all your passwords are where they need to be, and all your other bits and such are in place, you can pack it all in a container and run it anywhere (Well... "anywhere" being a bit of a stretch, but you get the idea haha)

So Java, which was supposed to be "write once, run anywhere", didn't meet that goal enough, so Docker wraps it in something that "actually for real" is supposed to be "write once, run anywhere", but even then doesn't quite meet that goal?

And so, the cycle continues

Java is perfectly capable of running everywhere, so containers are used not to solve this problem. With a container you ship the fixed version of the execution environment: JRE, system libraries, startup scripts, etc.

No. Java's "write once, run anywhere" promise is that you would not have to re-compile your source code to target different architectures like you do with C or C++. Docker's promise is that you don't have to rely on external things like databases, filesystem details, and other things _outside your program_ being in place across environments. Same idea, different levels of abstraction.

I think the OP's point is, that's what a jar file was supposed to do. You can ship a bundled-up jar file to anyone and they can run it so long as they have the right JVM installed. Now, you replace 'the right JVM install' with, 'the right Docker install' and what have we really gained? It seems an added layer of complexity for little gain.

I like docker and use it, but it's not for everything everywhere and is overly-hyped IMO.

> they can run it so long as they have the right JVM installed

This is the key. The right JVM, the right classpaths, the right configuration, the right permissions, the right native libraries.

The setup process for every Java app we've used (thankfully just ActiveMQ and Kafka lately) have been incredibly complicated. JAR files in paths, long, convoluted shell scripts to set up all possible variables for every possible JVM, wrappers that wrap launchers, etc.

And then all of those steps are prone to breakage and are difficult to debug.

Shipping a Docker container lets you say "Here is a working environment that needs no configuration and won't suddenly fail until reconfigured when another app you have needs Java 9 and not Java 8".

This. Going from java 5 to java 5+n with everything have to use the same JVM, shared on a computer was a nightmare in my large organization.

Yes, when I tell people to start looking at Docker as an "complete application packaging" solution (vs a VM), the light seems to go on for most.

Wait until you get to Docker 5 -> Docker 6. I bet it will turn into Java basically.

"Right Docker install" - I agree, there will be that problem at some point. But containers are not just some extra layer: they do not solve the same problem as fat jar. JARs are used for the distribution of the code and related assets. With containers you can distribute and isolate the deployment configuration: you can run multiple applications on the same physical server and not worry about conflicts due to different versions of JRE or system libraries. Vulnerability in one app will not affect another one. Security patches and system updates may be tested in isolation in a single container, before propagating the change (in the form of a common base image) to others. And, of course, it's just very convenient to run the whole Linux-based production cluster or test load balancing on your work laptop with Windows. :)

It's the same advantages as running anything else in a container, cgroups and namespaces.

so it can be scheduled and managed by kubernetes/swarm .

As others have pointed out, it's not a VM. In fact, running the same Java app outside the container will have the exact same performance as running it inside the container (if you don't add memory/cpu limits).

A VM is also super heavy: all those libraries, the package system, init/systemd and you need some Ansible/Chef/Puppet scripts to build all that. A container still has a lot of those libraries (most are build on a base distro image: ubuntu, debian, alpine, etc), but no init/systemd. Your process runs as 1 with cgroups used for isolation.

You also have this nice contained artifact. No packaging it as an RPM/DEB and having to install it. It comes with its JVM and dependencies in there. It will always run the same way where ever you put it. If you have five containers that all package the same way, when you deploy, you don't get 5x the space usage. Docker uses overlay filesystems so the base can be shared.

Also once it's in a container, you can use something fancy like Marathon, K8s, nomad or swarm to schedule it to run somewhere out on a cluster. These ecosystems have tools for monitoring the number of services you have running in parallel and the ability to scale them up and down.

You don't get the isolation/security you get in a VM (if the kernel has an exploit, it can be exploited from within the container. It shares the host's kernel), but they are as light as a process with some security guarantees via cgroups and with all your artifacts and dependencies packed into an image you can easily save off, transfer and deploy.

Improved deployments, repeatable builds, easier monitoring, makes it platform-agnostic, easier networking, resource limits. There are many reasons.

And, as other people have pointed out, a docker container isn't a VM. It's a chroot process with a bunch of tooling around provisioning, networking, security, etc around it.

One of the big wins of docker is that it allows you to ship file-systems around and run them as if they were binaries.

I had the same question as you 2 years ago. Here is the long answer http://blog.codepipes.com/containers/docker-for-java-big-pic...

To truly set up a Java application in a production-ready way (not just running something quickly), you need to be sure of all these things:

1. A compatible JVM is installed somewhere.

2. When you run your application, it uses that JVM and not some other JVM.

3. You have the right flags passed to the JVM. For example, max heap size matching your application's actual usage, garbage collector tuning, thread stack size, and any properties that are set via -D.

4. Your jar file is there.

5. There is a place to write log files, and your application's logging system knows to write them there.

6. If you are using any native libraries through JNI, they need to be installed, and the JVM must be able to find them and use them.

7. If you listen on port, an available port number is found, your application knows what it is, and the rest of the world knows where to find you.

I believe Docker addresses all these points.

Is it me or does it seem odd that docker community edition is left out? And isn't adding an additional "silent c is for container" in front of cgroups a bit... Much?

I mean, if java 10 recognises cgroup limits, that should work under things like systemd and all container runtimes?

I can't see how it's EE only. It reads the appropriate bits to work out the memory available, can't see how CE would hide that.

The problem is wider than Java anyways. Containers see the system free memory, not what's available to that container regardless of the cgroup constraints. We've hit this many times and sometimes workarounds are available (mongodb) but sometimes they're not and in our k8s environment this means some pods will restart constantly when they try to cache over and above what their cgroup allows.

The real fix is to tell a container not the system free memory but that which it's been allowed to use - afaik this is really difficult to implement, which is why the kernel hasn't had this done yet.

EDIT: obvs the container could read the cgroup limits (which I assume is what Java 10 does) but all the userland tools would have to be changed as well as some kernel functions.

Hm, I wonder if this might possibly be related to an issue I've seen in production with docker - of a container being killed by the oom killer... But only on server boot.

Maybe. We've had serious problems with this, mostly with mongodb but other stuff as well.Java 10 is making an effort to fix this but I hate Java, and really it's a kernel/userland linux issue as you can't expect software not to request cache memory that it doesn't know exists.

Genuine question: how many startups are using Java now? is this dealing with older sites that are stuck with Java on the server, or are there genuinely many new deployments of Java-based systems out there?

JVM is pretty much running half of the internet.

My company is not a startup, but we use Java heavily (also a good bit of python). Spring Framework is quite nice these days and being able to use Clojure pain free has been a nice bonus.

Recently we have started a migration of our pipeline to Kubernetes, so these improvements are a real benefit to our use case.

Plenty. You'll find it in more enterprise-sy and older software shops, not so much startups.

edit: sorry I mean other than startups you'll find plenty.

We're currently using Groovy and Grails deployed within Docker containers to ECS.

Really depends on your definition of "startup". Either way, I'd say it's generally still popular for new projects - especially those where the path to product market fit is pretty clear.

Most startups in the Midwest are using Java in my experience. Since most of our job market is banks, insurance, etc there's just a lot of Java expertise.

It depends what you've done and what you're going on to do. Spent your time so far as a Java developer, spotted a gap in the market for a tool for Java developers to do [x], then gone on to build a startup around it? I'm sure there's a fair few of those. And if there aren't it's probably an underserved market, there are plenty of very wealthy companies using Java.

Almost every one that has an Android app :/

We're also moving ahead with our plan to replace node with the jvm for our backend.

Why are you replacing node?

1. Static typing without having to deal with TS

2 (main reason). Our codebase has turned into a disaster, and we realized that none of our node developers are senior despite what their resume says; our interview process was more geared towards hiring anyone than hiring well. We tried finding a more senior node engineer, but we couldn't given our time constraints, so I'm taking some of our actual senior engineers from our mobile team, and the consensus was that we wanted to work with Kotlin.

We use a bunch of different tech stacks, but there's some Java in there.

In fact, I recently had to create a cross-platform desktop app, and I'm still not aware of a better option then the JVM. I would have preferred to write it in Scala, but opted for Java to make it easier for the rest of the team to maintain.

Java isn't an awesome language, but it's fine. There's a lot of awful legacy Java code out in the wild, and I wouldn't want to work on it. But Java as a language is boring, inoffensive, and it works. As a result, it's probably going to be around for a long time.

> I recently had to create a cross-platform desktop app, and I'm still not aware of a better option then the JVM.

What GUI library(ies) did you use? Just Swing? Java-FX as well?

I'm curious about this - I've developed the macOS part of a desktop app, where the Windows part used WPF, and we shared C# code (.net core, so it would run in both platforms).

The advantages were that each app looked 'native' to its OS. But in my opinion, the disadvantages outweighed them: a) No linux support. b) Double the work on the UI parts.

The Java alternative would reverse this: no native-looking UI elements, but they would be done once and run everywhere, including Linux. Another alternative would be Electron, but I feel (though I'm not sure) Java would be more performant.

Can anyone chime in on this? I know there's no definitive winner on this space...

I just used JavaFX. It looks native everywhere, the tooling was good, and it works on Linux. I recommend taking a look.

I've written almost no Java before this project, and I had never used JavaFX before. It took about 2 weeks to go from beginning to end on the project, which I was happy with.

Oracle wants to drop JavaFX. I'm not sure about its future. It might be bright if someone wealthy would pick it up or dark if it'll end up in some apache catacombs. I would use Swing, it's very old, but it works and it's not going anywhere.

Ah, Oracle - where good frameworks and open source products go to die.

I am at an early stage startup which is using Java (and Kotlin) on the server-side. There are various reasons for this, most of them pragmatic, but also some technical ones. Pragmatic reasons:

- Many engineers know Java to at least an intermediate level, so they can be productive quicker (and hiring is easier)

- We had some existing libraries written in Java, so integrating them with the web platform was much easier if we went down the JVM route

- Java is the language most familiar to members of the startup (including me, although I also have experience in RoR, Kotlin, ObjectiveC and JavaScript)

Technical:

- Java is generally very performant and running on the JVM means we don't generally have to worry about platform specific issues (there are well known exceptions of course e.g. file path issues)

- The language itself is at a reasonable level of abstraction. It is strongly typed and IMO, writing a non-trivial backend in a dynamically typed language is a sub-optimal choice. It has a huge ecosystem of robust, battle tested libraries that are indispensable to us (e.g. JGraphT). There is a huge community - you are unlikely to run into 'uncharted' territory

- It is popular to hate on Java but it really isn't that bad a language, especially if you aim to follow guidelines like those in Effective Java (probably my favourite programming book) e.g. avoiding mutable state and side-effects. It will never be as good as a modern language like Kotlin (data classes are my favourite feature there), but it is good enough

- The server frameworks available are seriously robust and 'just work'

- There is always the option of using alternative JVM languages (we use Kotlin for a large part of the backend)

In summary, we are 6 months into developing the web platform, progress is rapid, and we have no regrets (yet) about the technology choices. IMO, sometimes boring is best.

I think people must think I hate Java. I don’t. I think it’s great and I love Spring, though I have been mostly programming in C++ so haven’t touched Java in a long while personally.

I’m more concerned with the copyright status of Java APIs. Can you really be sure that you will be safe from Oracle’s clutches?

I had assumed you weren't a fan, apologies for jumping to conclusions. The copyright concerns are valid - I've only followed the whole Oracle vs. Google litigation from a distance. I'm hoping the little guys like us won't be affected.

> I’m more concerned with the copyright status of Java APIs

Building something using OpenJDK using Java APIs shouldn't get you in any trouble.

What got Google in trouble with Oracle was that they copied and re-implemented all of Java's APIs. Are you planning on doing something similar?

> What got Google in trouble with Oracle

... was a big wallet.

Funny, the WINE project does this.

So does ReactOS.

Only both projects are pretty pointless to sue since they aren't making any money.

If Oracle would win this lawsuit, entire industry will be in a serious trouble. Don't forget to pay IBM for every SQL statement you wrote, they invented SQL as an API for relational database after all.

I like java. I don't love it, but it's a trusty workhorse and a nice tool with a great ecosystem.

Those are good reasons. What determines when you use Kotlin over Java?

I work mostly solo on the web platform which is in Kotlin. It uses our 3D printing algorithms which are written in Java (which I also work on). Some weeks I'm writing mostly Java, others Kotlin.

We're looking for engineers in London incidentally, if anyone is interested. See my post in this month's who's hiring.

> It will never be as good as a modern language like Kotlin (data classes are my favourite feature there), but it is good enough

Data classes may come to Java.

http://cr.openjdk.java.net/~briangoetz/amber/datum.html

http://mail.openjdk.java.net/pipermail/amber-spec-experts/20...

Fibers, continuations, string literals, pattern matching and value types are in the works.

http://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.htm...

http://openjdk.java.net/jeps/326

https://www.youtube.com/watch?v=n3_8YcYKScw

http://openjdk.java.net/projects/valhalla/

I think if you count the JVM languages as one (Scala, Java, Kotlin, Groovy) it is the largest even for startups. (This article deals with a property of the JVM not really Java the language.)

> are there genuinely many new deployments of Java-based systems out there?

Java is absolutely huge in the corporate world, where they build new projects with it all the time.

> Genuine question: how many startups are using Java now? is this dealing with older sites that are stuck with Java on the server, or are there genuinely many new deployments of Java-based systems out there?

Quite a few. It's a solid choice for a startup for a variety of reasons:

- The Java (and more generally, the JVM) ecosystem is huge and includes stable, performant libraries and frameworks for just about every use case you can have. No need to re-invent the wheel.

- Java as a language is mature and performant, but is still actively being improved.

- No vendor lock-in. There are multiple implementations of everything including the JVM itself (unlike C# for example).

- There are very good tools for developing, debugging and profiling JVM applications.

- Some of the most popular big-data related tools are all based on the JVM. Just to name a few - Kafka (Scala), Hadoop (Java), Cassandra (Java), Spark (Scala), Ignite (Java), Elasticsearch (Java), Storm (Clojure), Neo4j (Java) and many more. So even if you're not using Java, you're probably using Java.

- Android apps are JVM-based. So every startup with a native android app is writing Java (or Kotlin).

- Hiring Java developers is easy(er). Everyone knows Java. Want to hire a junior developer? No problem. Want a veteran with 20 years of Jave experience? You can find those too.

- Java is well supported - due to its popularity, vendors usually offer first class SDKs and support for Java developers.

- There are many JVM-based languages. Don't like Java? No problem - you can use Clojure, Kotlin, Groovy, Scala, JRuby or even Jython. Some of them are even directly compatible with Java (and/or each other) so you can mix and match in some cases.

FWIW, Java on the desktop is alive and well, including games and IDEs. For example, Spine is best in class 2D animator.

http://esotericsoftware.com

Reply on news.ycombinator.com