Hacker News Reader

What amount of ram and GPU are people considering adequate for playing around with all the local inference possibilities over the next year?

As much as your budget allows for.

Agreed. But your choices are now the 4060 to 16gb for $500 or a way more expensive 3090/4090 with 24gb and the power draw to go with it.

I started out with a 1080ti that I had lying around. Within days upgraded to an RTX3060. Within weeks of that upgraded to an RTX 4090.

In retrospect I wish I had of known that "buy once, cry once" might have been the way to go here.

That's just me.

3090s go for under $800 used, and you can set the power limit to whatever you want.

I found that 290W keeps the fans from spinning up audibly under heavy CUDA workloads with ~2% lower performance.