I use axolotl for training, I didn't check your notebook but axolotl likely comes with more optimized defaults for speed and vram than what you're doing.

Yeah, GCP GPU prices are terrible. $150 for a short time on a P100 is highway robbery.

TPUs are better, but still kinda pricey.