On demand vs. Reserved instances for LLM fine-tuning
I am looking at different options to get access to GPUs to train an LLM on an enterprise use case for a customer. They are on a specific Cloud provider with associated credits, so I have no ability to go outside, like Runpod.
We are exchanging on whether we should go for On-demand or Reserved instances. This client is a Fortune 500, so they could reserve it but it might not be the best choice.
It seems to me that with LoRA / QLoRA, and so on, we might be able to fine-tune a Llama 2 with one or two GPUs on-demand, but I am unsure yet.
So our main evaluation criteria are:
- Price
- Availability of GPUs, aka we don't want to waste too much time to get started
In your experience, have you had difficulties to get access to GPUs at a good price? How many of you had to go with reserved instances, and if so what made you choose this option?
I would love to have your feedback!