Disclaimer: I'm one of the engineers on the Quay team
Quay doesn't (yet) support the idea of multi-step builds, but it is definitely on our roadmap.
One recommended approach would be to make use of our squashed image support [1]. You can build the artifacts using the in-image compiler toolchain and then delete it (in the same step or a followup step). If you then download the image as a squashed image, all deleted and overwritten files are automatically removed from the layer when it is created. Not a perfect solution, but definitely better than downloading all the unneeded binaries.