Shameless plug, but another option for running your experiments on a GPU is Spell (https://spell.run). Very simple to use and will allow you to use as many GPUs as you want at once and only pay for the compute time. You can also run GPU-enabled kernels for Jupiter.

Fast.ai instructions here: https://learn.spell.run/fast_ai