If I were Phind, I'd be looking at Deepseek 33B instead. While obviously dumber for anything else, it feels much better at coding. Its just begging for a continued pretrain like that, and it will be significantly faster on 80GB cards.
As I mentioned, being such an extensive continuation train can (sometimes) totally change the capabilities of a model.
What's best that can run fast on 4090 laptop?
- Hybrid offloading with llama.cpp, but with slow inference.
- Squeezing it in with extreme quantization (exllamav2 ~2.6bpw, or llama.cpp IQ3XS), but reduced quality and a relatively short context.
30B-34B is more of a sweetspot for 24GB of VRAM.
If you do opt for the high quantization, make sure your laptop dGPU is totally empty, and that its completely filled by the weights. And I'd recommend doing your own code focused exl2/imatrix quantization, so it doesn't waste a megabyte of your vram.