With that aforementioned A6000 ($5k retail) you'd have to use it for at least six thousand hours to break even on the cloud cost.
Something people forget too is that if you have no Nvidia GPUs at all locally, you'll need to spend an significant amount of time installing a new node, copying data, and debugging in your cloud instance, each time you want to do something, while being charged for it. It's a pretty big boost in terms of my time to develop locally and then scale to the cloud once something smaller scale is working.
But most people who toy with LLMs will probably never make money out of them. Even those who do will often spend a lot of time getting their bearings during which the GPU sits idle. Then you begin to ramp up your use but by the time, there's a new generation of GPUs out.
That's why my recommendation is to start with something lightweight.
It's also much less frustrating to start working for a few hours on a rented A100 rather than running into OOMs all the time while fine-tuning batch sizes and waiting for the nth highly quantized model to download.
I have 2 4090s personally, which is perfect for pretty serious 7B fine-tuning and inference, and doing development work on smaller stuff before scaling to larger runs in the cloud.
At work anything less than 8 GPUS per run is small time stuff - we sometimes scale up to 128 or 256 GPUs for some runs.