Plug and Play AI DevBox

I'm building a pre-built AI setup to test various LLMs and have successfully integrated it into my business. I want to share it with everyone here.

This already integrates Qwen2.5-Coder 32B, Llama 3.1 70B 405B, FLUX.1 [dev] and Hermes-3-Llama-3.1-8B. It's highly customizable and can be used as a cloud AI, similar to Runpod when it's not in use.

Tech specs:

- AMD Ryzen Threadripper PRO 5955 WX

- 2 x RTX 4090

- 128GB RAM

- SSD NVMe 1TB