Make builds in docker by mounting volumes and have your sources, intermediate files, caches, etc. in these volume mounts. Building a bunch of intermediate or incremental data IN the container every time you execute a new partial compile is insanity.
It's very satisfying just compile an application with a super esoteric tool chain in docker vs the nightmares of setting it up locally (and keeping it working over time).