I use gitlab multi-runners. It's a privileged mode docker container that spins up other containers on the host machine to run its builds. I have them sitting in an autoscaling group where if it hits 40% cpu usage another server running gitlab multi-runner starts and automatically registers to do more builds.

You can do this with any ci system though, not specific to gitlab.

I use gcloud compute instances create ... in a shell script.