But even if you weren’t using the new features, there was basically no downside to upgrading the run time. The newer version is usually ran much better thanks to improvements in threading or garbage collection. And you knew you’d get security patches, even if you ended up getting stuck on that version for a while.
As you said, libraries and frameworks can be a big issue to upgrade.
But the backwards compatibility promises of Java have been so incredibly strong it hasn’t been an issue.
The one time breakage in Java 9 is basically the first I’ve experienced in my career.