If after this change, Java is no longer able to compete in "The One Billion Row Challenge", it is a loss for the ecosystem.
This is a clear win for low-level coding in Java because today there's no way to tell if a Java program is unsafe as some transitive dependency may be using Unsafe even if it doesn't benefit the program at all. Offering full safety for (nearly) all while still offering maximum performance in special cases is an advantage for Java over other languages.
In case you guys conducted an ecosystem survey, where are we at now? Lots are still running on JDK 1.8, some important deployments are on 1.6 or 1.7, even more are on JDK 11. The transition to JDK 17 has started only recently.
How much more hesitant will this make projects to upgrade the JDK? There’s quite some tearing already due to 1.8, 11, and now 17, particularly with Pivotal EOL support for Spring 2.7 release train. I wish we could wait with it till the transition to 17 is complete.
Now that internals are encapsulated, upgrades are easier than they've ever been at any point in Java's history. Unfortunately, some large applications that depend on non-portable libraries — some of which were since abandoned — are stuck, but support for JDK 8 continues at lest until 2030.
Beyond JDK though, the community has been trying to move forward in ways inconsistent with the backward compatibility for projects with dependencies such as those that you describe. This is what's causing most of the hesitation, exacerbated by the modality that there's little clarity on how to replace or upgrade such shenanigans.
From your comments, it turns out I’m quite unaware of the details of the transition to the encapsulation of the internals. I recall you mentioning that some time ago too. I’d appreciate a pointer to some overview, I feel like I’m going to need one before making a dive into the rabbit hole.
Two months ago I did a fresh deploy of Java 8 in production for a SaaS solution, exactly because that is what the product and support team care about.
Also, there isn't much new code being written for 8, and there's a whole lot more of Java code yet to be written than the non-portable code still on 8. In fact, even most existing projects are no longer on 8. Sure, if you only look at the pain, there has certainly been some. But if you look at the pain compared to the benefit, the lesson is that it's been a great success, especially compared to other languages (or even frameworks) doing similar significant changes. I think the vast majority of those who've upgraded consider it worthwhile (again, comparing cost/benefit).
Virtual threads, FFM, and the upcoming Valhalla and Leyden are all things that could not have been done without modularisation, and certainly not without significant internal JDK changes, which would have kept the same people on 8 even if there was some way (doubtful and definitely much more expensive) to achieve that without modularisation. We would have been in a far worse place without it: fewer new features and no end to the problem of migration.
It's an obsession over "pure" code that has no basis in reality. I'm sorry: Applets don't exist anymore, this obsession is outdated and futile. There is no rational justification behind breaking compatibility with almost every large Java project in existence to the extent that https://java.com still downloads Java 8.
No. JDK 9 and 11 kept all default accessibility the same as in JDK 8 (all you got was a warning) until it was changed in JDK 16: https://openjdk.org/jeps/396
The adoption issues were almost entirely due to non-portable libraries depending on internal implementation details that had changed in 9. Some were due to changes like that to the version string that dropped the "1." prefix for the first time.
> It's an obsession over "pure" code that has no basis in reality. I'm sorry: Applets don't exist anymore, this obsession is outdated and futile.
I don't understand the relationship to Applets, but the emphasis on integrity (a generalisation of safety) is all around the software world. It's the main reason for Rust's existence.
> to the extent that https://java.com still downloads Java 8.
That's because java.com is a website for end users downloading the JRE, which was discontinued after 8 (really in 11).
This is simply not true. This sounds like gaslighting in an attempt to rewrite history, which IMHO shows poor credibility from the JDK developers.
> I don't understand the relationship to Applets, but the emphasis on integrity (a generalisation of safety) is all around the software world. It's the main reason for Rust's existence.
Yes, and Rust has "unsafe". Java still has JNI. If programmers want to write unsafe code, they will write it, to the point of shipping a modified JVM. It's their computers. It's their code. It is a misguided and fruitless effort to push ideology on them.
> That's because java.com is a website for end users downloading the JRE, which was discontinued after 8 (really in 11).
I don't buy this.
I'm sorry, but default access was not changed until JDK 16. Here's the relevant section from JDK 9's JEP 261 (https://openjdk.org/jeps/261#Relaxed-strong-encapsulation):
In this release the strong encapsulation of some of the JDK's packages is relaxed by default... --illegal-access=permit opens each package in each module in the run-time image to code in all unnamed modules, i.e., to code on the class path, if that package existed in JDK 8. This enables both static access, i.e., by compiled bytecode, and deep reflective access, via the platform's various reflection APIs... This mode is the default in JDK 9.
> Yes, and Rust has "unsafe". Java still has JNI. If programmers want to write unsafe code, they will write it, to the point of shipping a modified JVM. It's their computers. It's their code. It is a misguided and fruitless effort to push ideology on them.
And Java will still gives you access to unsafe operations (even after this change) albeit with a flag that serves a similar purpose to Rust's unsafe, only done better. The goal is not to impose an ideology on programmers. If you read the JEP, the goal is for them, and the runtime, to know which invariants can be trusted and which can't (e.g. the JIT might want to optimise things based on the assumption that strings are final, but using unsafe code or native code makes that untrue). That's why JNI will also require a flag. As it is today, neither the runtime nor the application developer is able to know whether there's some code that could mess about with their assumptions.
> I don't buy this.
It literally says that on the front page of java.com: "Get Java for desktop applications." OTOH, the developer website, https://dev.java/, lets developers download JDKs only, as JREs no longer exist.
We don't give ETAs as people may make business decisions based on them. As a matter of policy, we only announce a targeted JDK within the 6 months prior to its release.
> it’s getting quite tedious to persuade everyone to keep it up and not jump ship, after getting burned by the previous JDK (and Spring) upgrades.
If nothing else, they should be placated by the fact that upgrades are significantly worse in every other ecosystem.
A lot of ecosystems care about their users and care about compatibility. Java is a toxic relationship at this point.
Yep, that's why we introduced the LTS service. People may not have noticed before we did away with major releases and changed the version naming scheme, but in the past you had no choice but to upgrade to semi-annual releases with major VM changes, like 8u20 or 7u4 (probably because the name was too similar to patch releases, like 8u30 or 7u5). Now, with LTS, we try to only backport security patches and major bug fixes to make sure that such legacy users get the stability they need.
In fact, why limit ourselves to technology? The same goes for deciding to make any change in anything. Every intentional change has only ever happened because someone had bet, without absolute certainty, that it will make things better. So you could say that "hope" is the only thing that has ever made anything better (or worse, I guess, if you're a glass-half-empty kinda guy).
Java developers didn't just start using Unsafe for fun. They use Unsafe because they benchmarked, measured, and found a real performance gain.
There is no reason to make this change besides misguided ideology. There is zero benefit.
We know that a small subset of Unsafe-using code will have to continue using internals to keep performance and won't be able to use the supported APIs. There is absolutely no indication about this having any impact of significance on the performance in the ecosystem at large.
> There is no reason to make this change besides misguided ideology. There is zero benefit.
Elimination of undefined behaviour by default (and so fewer crashes) and better security come to mind. Also, better compiler optimisations. This is explained in: https://openjdk.org/jeps/8305968
The fact that you think this is of new benefit to you doesn't make this empty ideology. We have a couple of decades of vulnerability reports, customer support tickets, and challenges of introducing compiler optimisations that have sufficiently convinced us of the benefit. Not every Java user has to be convinced by every change to the JDK. In fact, we rarely make any JDK change that everyone is convinced we should do.
You seem to be really averse to evidence. In my opinion, the JEP should have benchmarks. It does not have any. I would say that raises concern for lack of credibility and shows dishonesty.
> Elimination of undefined behaviour by default (and so fewer crashes) and better security come to mind.
> Also, better compiler optimisations. This is explained in: https://openjdk.org/jeps/8305968
There is nothing stopping a compiler from omitting conflicting optimizations in functions using "unsafe". Undefined behavior due to optimizations is also already commonly understood in C/C++. If you don't like codebases using unsafe, don't use them!
> In fact, we rarely make any JDK change that everyone is convinced we should do.
That's a very round-about way of saying "we only do what we want and who cares about the users". Funny.
You have evidence that a significant portion of the ecosystem will suffer some performance degradation? If so, it would be immensely helpful if you could share it on the mailing list. We take each and every report seriously.
> In my opinion, the JEP should have benchmarks.
The Panama project has been publishing and discussing benchmarks for years. Here's one example: https://mail.openjdk.org/pipermail/panama-dev/2020-November/...
I don't understand, however, what kind of benchmark could show that most libraries can migrate from Unsafe to FFM without adversely affecting most of their client applications. I'm sure you can find some discussions about performance on panama-dev, where JDK developers have been working with library authors and other Java users on the project for the past years.
> There is nothing stopping a compiler from omitting conflicting optimizations in functions using "unsafe". Undefined behavior due to optimizations is also already commonly understood in C/C++.
Yeah, the world is moving away from undefined behaviour; Java is certainly not going to start moving toward it. Avoiding unspecified behaviour has sort of been one of Java's main selling points from day one. Because we've been delivering significant performance improvements while improving safety at the same time for years (JDK 21 improves the performance of a lot of applications by a significant amount compared to JDK 8, all without changing a line of code), I can't see a reason why we should change our strategy now. We don't need to let undefined behaviour more easily in to give our users the performance they want.
So while we allow undefined behaviour with a flag, allowing it by default would mean it could be everywhere, and most of our users prefer safety to be the default, also in line with the global trend.
> If you don't like codebases using unsafe, don't use them!
The problem is that a Java application has no easy way of knowing whether some transitive dependency may be using Unsafe. The integrity JEP goes into that.
> That's a very round-about way of saying "we only do what we want and who cares about the users". Funny.
It means we have 10M opinionated users, we work very hard to make sure we satisfy most of them with most of what we do (after all, our job depends on Java's continued thriving, which depends on our users' happiness), but even if 1% is unhappy with something we do, and 1% of those express their opinion on social media, that's 1000 angry people saying they're not being listened to. The fact is that different programmers want different things, often those things are contradictory, and it's quite rare to find something that millions of programmers agree on.
I don't know if it's funny or not, but part of doing this job is knowing that you work all day, every day, to make most of your users happy, and no matter what you do or don't do, there will be people yelling at you on social media, because programmers just can't all agree on pretty much anything. But I'm not complaining. The overall satisfaction with our recent features and releases has been palpable.
The matter is one of these tools is being taken away.
We've got Rust now, offering both low-level control and safety. No need to take the SUV on the racing track, it is not at home there.
Amusingly, though somewhat tangential, I observed a rather simple Go implementation (nothing fancy, ordinary channel bound message passing) outperform a fairly sophisticated Rust implementation for the same logic when we were toying with a prototype. But that was at an IO boundary. There are many other variables of course, but an amusing anecdote to share.
https://medium.com/@jadsarmo/why-we-chose-java-for-our-high-...
LMAX Disruptor customers
https://lmax-exchange.github.io/disruptor/
Among many other examples.
The newer, better thing from the same author is aeron.
I'll easily believe that it can take significant work to outperform Golang (or Java) in Rust where application complexity is high (not just a simple cmd app), they do have an excellent baseline out of the box.
I personally am often quite astonished by the amazing performance Go apps squeeze out of the box, with many efficiency tweaks and tricks we way too often take for granted, which you need to be aware of to come by in C++, Rust, or even Java. But I do also often get disappointed by the sudden inconsistency of that performance in Go after a while, the variance is a bit high, with C++, Rust, and Java runtime behavior being very consistent in general.
This is subjective. In my opinion, even 10%~ is massive. Performance matters and this would be a step backwards.
Dropbox's experience https://dropbox.tech/infrastructure/lossless-compression-wit... shows a >10% performance difference caused by bounds checks in a similar context for a real world application. And before you say it: the Java JIT is not capable of optimizing bounds check elimination more than LLVM at this point in time. The JIT is not magical.
Edit: Unless you FFI to Rust / Ada / formally verified C, in which case perhaps we can claw back the performance and safety at the cost of a lot more dev work.
Also, FFI to Rust will buy you nothing. Safe Rust performs the same bound checks as Java while unsafe Rust would be just as unsafe as using JDK internals. If you want to prove the safety of unsafe code in some other unsafe language, you might as well do the same for unsafe Java code using JDK internals (it would probably even be easier).
Are you sure you should be using Java if your use-case is performance sensitive?
Sorry, but this is just a bad advice.
But if it turns out to be a real problem, it might still be possible that they will add a way to disable checks with the foreign memory API. It would actually allow for a better user experience, you could do “debug” builds with extensive checking for correctness, which could be disabled via some flag.
We are no longer in the Sun days, with Java being the answer for everything.