I find that the pain I experience with Gradle isn't usually about how to do something clever or customized etc, but instead it's when I haven't thought about Gradle syntax in the last 3 months since everything has been silently working, but now I need to figure out some small thing, and that means I need to go re-learn basic Gradle stuff - whether it's groovy, Kotlin, or some aspect of the build DSL - since my mind has unloaded everything about Gradle in the meantime.
Simplifying the semantic complexity of a general purpose build system will always help, but the most useful thing for me would be if the configuration for a Java build were to natively use the Java language directly.