And even for Java 8 you need a latest Oreo device, because not everything can be desugared into Java 6.
There is also the issue that going forward Java 9+ libraries won't be possible in Android.
So the only way forward appears to be indeed Kotlin and Kotlin specific libraries.
Apparently they thought they could get away with it.
And there are still lots of jars on Maven central that are unusable on Android Oreo.
Because even after switching to OpenJDK, they cherry pick APIs from there and ART does not support all JVM bytecode features.
It seems to be more that they don't prioritise keeping up with the Java ecosystem very much, they'd rather work on their own stuff. It's more a cultural issue than anything else IMO. Google has always had a big NIH problem.
Switching languages only helps the syntax. If the Android Runtime can't do something it that means it can't do it in Java or Kotlin.