You can speed it up somewhat by tweaking ~/.gradle/gradle.properties and adding org.gradle.daemon=true and org.gradle.parallel=true. That way at least it doesn't have to re-read all the build files each time and will run some parts in parallel.

The benefit wrt the old Ant-based build is the way it lets you specify 3rd party dependencies. Being able to just add in say Timber, ButterKnife and Guava with a few lines of config, rather than downloading the jars, faffing with paths, making sure the pre-processing bits are in place, etc, is really a big benefit. Maybe Maven would have been a more sensible choice, since Gradle/Groovy seems pretty niche and not really used that much outside Android.