In the end I created a hacky build process where I use a single container to build both the x64 and ARM versions serially, and then create multi-arch containers in a separate step. It was very painful to get the right native libraries installed, and it's not terribly easy to build these two platforms in parallel.
In short, having access to real ARM builders would be great, and persistent disks would probably boost my build performance quite a bit.
The dockerfile that I had to use: https://github.com/mmastrac/progscrape/blob/master/Dockerfil...
Example build run (~20 mins): https://github.com/mmastrac/progscrape/actions/runs/42285298...