Comment by theYipster | Hacker News Reader

Comment by theYipster | Hacker News Reader

Great article. I'm about to embark on a similar journey.... Doing a ton of AI development right now. Don't need a server, but a very, very high end workstation is super appealing to me right now. Looking at $50-$80k. 1TB RAM. 2x RTX Pro 6000s. 64 core Threadripper Pro. As many 4tb or 8tb nvme drives as I can stuff.

I envision NixOS at the core... then everything I need virtualized on top with KVM/QEMU. Maybe a dual boot setup with Windows for gaming and Flight Simulator (but I could virtualize that too with easy GPU passthrough.)

Lingering questions I'm working to figure out:

- Will 2 RTX Pro 6000s run on a 1600 watt PSU? Not sure how much higher I can go without calling an electrician. (standard US home.)

- Assuming I plop this into my home office, should I expect the PC to run significantly hotter than my current rig? (3960x threadripper, 128GB RAM, 1600watt psu, overclocked and watercooled 4090.) My water temp, measured at radiator, is about 60c at peak load. (This is the only number I care about, as this is what I have to consider to be comfortable sitting next to it.)

What do you want to do with the workstation? I have a similar setup:

- 512 GB

- Epyc 9684x

- 2x RTX 6000 Pro

- 1400 W PSU x 2 but in redundant mode

Mine is in a colo where it stays nice and cool. In my case, I went with less RAM and more GPUs (bought 4). Secondarily, the Max-Q blower version of an RTX 6000 Pro Blackwell is easier to keep cool and also only needs 300 W at the cost of very little performance. The non-max-q also only really use 300 W during inference, but the good thing about a lower power use is you can put more GPUs in very safely.

I assume you want the Threadripper Pro to maximize single-core performance? So you're spending a lot of time on CPU? Interesting stuff.

I gained a lot putting the machine somewhere else. TTFT on a thing like this is between 100-800 ms depending on batching and model size and so on, and your nearest datacenter is likely <10 ms. It sits on nice dual redundant power in a place where it's blown icy cool.

Good luck with your setup. If you get around to it, and end up writing about your setup on a blog, do share. Email in profile.

Very nice. Primary use case is application development, where the applications leverage a mixture of cloud based and local models. Modelling complex architectures. My work is primarily in the aerospace and defense arena, so hybrid and on-prem are important, as are ITAR and CMMC compliance. The idea is to have the local rig to build and validate architectural deployments that can sit on prem on customer hardware, in cloud, in gov cloud, or in a mix.

Not really looking at colocation, as this machine would double as a heavy duty gaming and flight sim rig. That means at least one regular RTX 6000 Pro. Not sure if I can mix and match with the Max-Q version, or if I even want blower fans in a desktop case (last time I did that was about 16-18 years ago with an ATI card... wasn't a fan--pun intended.)

Haha seems dope. It’ll be cool to see what you build. Do post.

Reply on news.ycombinator.com