Hacker News Reader

Comment by hayd | Hacker News Reader

Is any of the build cached?

In order to limit costs, everything is pulled from source on each build and there is nothing cached. This strategy takes advantage of the fact that AWS doesn't charge for ingress traffic and unfortunately puts additional load on Google's servers. I've attempted a few different strategies on caching AOSP and Chromium source trees, but since you have to incur the storage costs on an ongoing basis, it's just not very economical.

You could certainly do it. No point having multiple EBS volumes lying around - just create a snapshot of the volume with the git checkout / build cache after each new build is done.

When you want to build again, create the instance and then recreate the EBS volume from the snapshot and attach it to the new instance. Pull the latest set of changes from the git repo and build with the old cache!

Obviously there are cache purging considerations (e.g. starting from scratch once per week/month) you could optimise as well.

I investigated EBS snapshot as an option, but there were two problems. 1) cost as i mentioned initially - for just AOSP source tree alone you are looking at > 250GB and at a cost of $0.05 per GB you are already at > $10/month and 2) EBS snapshots lazy load from S3 which gives TERRIBLE performance which means you end up with far far slower builds. AWS released a feature "EBS Fast Snapshot Restore" to workaround this issue, but it's extremely expensive.

Are you doing partial clone?

https://source.android.com/setup/build/downloading

Although, an answer at https://stackoverflow.com/questions/33053615/how-to-download... from October 2020 says that even a partial clone is still 73 GB in size!? That’s insane! How the heck come it’s so big?

That's likely the size of the entire checked out tree, which would include all of the files.

I just ran the command here and my AOSP 11.0 checkout is 54GB, minus any git history, since I clone from a local mirror and use '--reference' to avoid having to copy objects.

A lot of the size here is from the various prebuilts, AOSP build is quite self-contained (jdk, clang, etc) and barely uses anything from the host.

Reply on news.ycombinator.com