Last time I did the math, by the time I spent 4090 money on compute, I could have been done with a PHD. Now, if all you need is a 3060? Yeah you could probably make that work, but you wouldn't be doing LLM inference even quantized
"The break-even point for a desktop vs a cloud instance at 15% utilization (you use the cloud instance 15% of time during the day), would be about 300 days ($2,311 vs $2,270):"
And that was written before the current GPU crunch and assumes availability in the cloud, which currently is not a given at all.
[1] https://timdettmers.com/2023/01/30/which-gpu-for-deep-learni...