7B 8bit GGML running on a single 4090 with llama.cpp. It's hard to overstate the massive jump in capability between llama 1 and 2.
Tensordock. RTX4090 instances are ~$0.50/hr and can handle 3/4 concurrent users each.
Reply on news.ycombinator.com