1. Bizarre and inscrutable syntax. They say things like "build scripts are just Groovy code", but some things, like how you define sourcesets or configurations, don't look like anything that should parse, much less compile, in any C-like language. They're doing some fancy meta-programming stuff behind the scenes to keep things light and simple, but it really isolates (and possibly alienates) developers from exactly what's happening under the covers. The Kotlin DSL seems to help make it more clear what's going on, but I haven't used it much yet.
Having used Gradle for years, it is strangely off-putting how unsure I often am about whether to put a colon in a particular place or a comma, or if something should be in parenthesis.
2. If you want to do anything beyond what's documented you can get into a bit of trouble, and if you find answers more than a couple of years old there's a good chance they often won't work. Making sense of the DSL documentation is sometimes a little tricky.
Beyond that, I would dare say it's the least awful option for building for the JVM. Their documentation is very comprehensive, it's fast, they are very fair about providing deprecation warnings before features are removed. It's rare I find a case where I'd like to use some external library that has a Maven plugin but no Gradle support.