Your org needs better distributed build caching if you can't get that number lower. All builds should be "incremental" with appropriate caching.