Ask HN: What is the best Llama model that you can deploy on a single A10G?

It's hard to choose between a 13B model with 8 bit quantization, or the 33B model with 4 bit quantization.

For some context, the idea is to build a text to SQL interface. The interface allows you to select certain tables from the data warehouse and injects their definitions in the prompt, so the 4096 context limit is useful here.