Building A16Z's Personal AI Workstation | Hacker News Reader

Building A16Z's Personal AI Workstation

a16z.com

52 pointsProofHouse73 comments

A rough price out using new egg/microcenter pricing, substituting a few items where they didn't specify a specific item/brand. I didn't bother trying to figure out what case they used.

Grand total: ~ $41,000

Motherboard https://www.newegg.com/gigabyte-mh53-g40-amd-ryzen-threadrip... $895

CPU https://www.microcenter.com/product/674313/amd-ryzen-threadr... - $3500

Cooler https://www.newegg.com/p/3C6-013W-002G6 $585

RAM https://www.newegg.com/a-tech-256gb/p/1X5-006W-00702 $1600

SSDs https://www.newegg.com/crucial-2tb-t700-nvme/p/N82E168201563... $223 x 4 = $892

GPUs https://www.newegg.com/p/N82E16888892012 - $8295 x 4 = $33,180

Case https://www.newegg.com/fractal-design-atx-full-tower-north-s... $195

Power Supply https://www.newegg.com/thermaltake-toughpower-gf3-series-ps-... - $314

As soon as you drive it off the lot, it is worth half the value.

It'll lose another half if you spray paint it gold like in the OP photos.

So we are talking 10k ebay price.

This could have been much more expensive. They built it from off-the-shelf parts anyone could get (except for the cost) and listed them which I appreciate. Microcenter/Newegg could put together a bundle as a joke and they'd likely get a few orders.

Of course I don't personally have any use for this but it's good to have an idea what it takes to run the best openweight models in a secure/controlled environment. To get started a single 96GB GPU system is only $16,115. For perspective I spent about $10k (today dollars) for a Toshiba Portege 320CT laptop with as much memory and accessories as I could get in 1998.

The case is an InWin Dubili with a modified, and color mismatched, front panel: https://estore.in-win.com/products/dubili?variant=4711876034...

In less than a year when A16z is finished with the few, pointless experiments they want to run on this and it is relegated to a closet forgotten, some poor founder is going to see it appear as part of their term sheet. "Oh, yes, $75,000 of our $500,000 commit is compensated through this state-of-the-art AI Workstation"

If you actually want a good multi-GPU system, just get an NVIDIA-designed system. There's no good reason to try to design and build one yourself when you will be relying almost entirely on NVIDIA cards and their multi-GPU communication.

They aren't selling one yet for the Blackwell generation.

I think the DGS Station only has a Single Blackwell GPU. Are there any other variants that has Multi-GPU that is not a Cluster or a Rack?

Super cool specs but I can’t stop laughing at that goofy A16Z logo on the case. Zero taste

If some rich guys want to role play as engineers - sure go ahead. But making this article should be a humiliation.

HN at its worse. Whining about anything and everything. Role playing as engineers? Don't you think there are many engineers working for AZ16?

> But making this article should be a humiliation.

Article? It's their f*king website. Did you say the same when someone post their complaint about LLM on their blog?

A VC fund selling “self-hosted” AI compute rigs was never on my bingo card.

I’m glad they did. It’s weird and different.

Other than the fact it's cool to run locally, what's the advantage of having one of these vs renting h100 clusters from runpod/lambda?

If you can’t already just buy a Lenovo or Dell workstation with this configuration, I’m sure you can just buy 4x GPUs and plug them into a base system that will support them.

Who is buying hardware this expensive from a business that probably doesn’t really know how to do (or isn’t setup to do) proper manufacturing tests?

> access to raw compute is still one of the biggest bottlenecks

> We are planning to test and make a limited number of these

So this does approximately nothing to solve the original problem of supply and cost. Even if you sold it at a loss, that GPU is still going to be expensive.

Just be honest and say you thought it would be cool and you're not Y Combinator so you gotta do whatever you can to make your firm seem like a special smart kids club.

> personal AI Workstation delivers complete control over your environment, latency reduction, custom configurations and setups, and the privacy of running all workloads locally.

What's the recommended operating system with support for this hardware and local compute without cloud telemetry/identity?

linux?

This is the not-so-distant-future that a lot of people don’t see. We’re in the AI mainframe era and it’s coming to the pc era soon. I hope that’s what Apple is waiting on. Perhaps we will buy LLMs and install them locally one day too like a video game.

Can you guys make one more that I could stop by and pick up from your office (and pay for, if you care for that sort of thing)? I checked with Puget Systems, but they are only doing 3 cards max.

My only question is: why not Zen 5? No suitable motherboards?

How much heat does it generate and how loud is it at full tilt? Try keep comparing it to a modern under-desk computer, but I’m not sure you’d want to have that thing in the same room while you’re using it?

According to the specifications the max power draw is 1650W. All of that is converted into heat so it's like a small space heater.

It’s more than a space heater. Any heater is usually 1500w.

And that is to keep it safe. Continuously pulling 1650 W out of a US power socket is not safe. I've seen even 1500 W continuous heat up loose connections inside the socket to a level that charred plastic. Leaving this machine running a heavy compute load for hours without a 20A socket is a fire hazard:

> Surprising efficiency: Despite its scale, the workstation pulls 1650W at peak, low enough to run on a standard 15-amp / 120V household circuit.

Ok but what are you doing with them? Much more interested in that.

just about enough power to run Microsoft Defender and maybe Teams too

It’s Teams or Defender, they haven’t built a supercomputer.

I imagine it runs civ 2 pretty well

Dude imagine mounting a virtual drive on that GPU and loading the game straight from that. You won’t even notice the loading screens!

Price: "Call for pricing. Never mind, if you have to call and ask, you can't afford it."

Building a16z’s Personal AI Workstation with four NVIDIA RTX 6000 Pro Blackwell Max-Q GPUs

Cringe.

For this money a tenstoreent silent machine is much better

It seems like this post has been removed from the front page of HN. I think it was in the #1 spot 15 minutes ago.

Happy april 1st!

If AI bubble pops can I mine crypto on it

Are they Trump? Oy vey

A16Z is consistently the most embarrassing VC firm at any given point in time. I guess optimistically they might be doing “outrage marketing” but it feels more like one of those places where the CEO is just an idiot and tells his employees to jump on every trend.

The funny part is that they still make money. It seems like once you’ve got the connections, being a VC is a very easy job these days.

VC is a marketing game. You want to be attractive to founders, so that the best founders/companies come to you and want to choose you.

But is gassing up founders something they want? Idk, maybe. But just remember these guys crypto play and it feels like they'll just yes man you off a cliff if you're a founder...

Yes people like that even if they think it doesn't work on them. Just like people who say advertising doesn't work on them when it really does work on us all.

It's been such a mind-boggling decline in intellect, combined with really odd and intense conspiratorial behavior around crypto, that I went into a bit a few months ago.

My weak, uncited, understanding from then they're poorly positioned, i.e in our set they're still the guys who write you a big check for software, but in the VC set they're a joke: i.e. they misunderstood carpet bombing investment as something that scales, and went all in on way too many crypto firm. Now, they have embarrassed themselves with a ton of assets that need to get marked down, it's clearly behind the other bigs, but there's no forcing function to do markdowns.

So we get primal screams about politics and LLM-generated articles about how a $9K video card is the perfect blend between price and performance.

There's other comments effusively praising them on their unique technical expertise. I maintain a llama.cpp client on every platform you can think of. Nothing in this article makes any sense. If you're training, you wouldn't do it on only 4 $9K GPUs that you own. If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.

> If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.

I was with you up till here. Come on! CPU inferencing is not it, even macs struggle with bigger models, longer contexts (esp. visible when agentic stuff gets > 32k tokens).

The PRO6000 is the first gpu that actually makes sense to own from their "workstation" series.

Er, CPU inferencing? :) I didn't think I mentioned that!

The Framework Desktop thing is that has unified memory with the GPU, so much like an M-series, you can inference disproportionately large models.

If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.

Well, you're getting the ability to maintain a context bigger than 8K or so, for one thing.

Well, no, at least, we're off by a factor of about 64x at the very least: 64 GB GPU M2 Max/M4 max top out at about 512K context for 20B params, and the Framework desktop I am referencing has 128 GB unified memory.

What's the TTFT like on a GPU-poor rig, though, once you actually take advantage of large contexts?

I guess I'd say, why is the framework perceived as GPU poor? I don't have one but I also don't know why TTFT would be significantly lower than M-series (it's a good GPU!)

Compared to 4x RTX 6000 Blackwell boards, it's GPU poor. There has to be a reason they want to load up a tower chassis with $35K worth of GPUs, right? I'd have to assume it has strong advantages for inference as well as training, given that the GPU has more influence on TTFT with longer contexts than the CPU does.

Right - I'd suggest the idea that 128 GB of GPU RAM gives you an 8K context shows us it may be worth revising priors such as "it has strong advantages for inference as well as training"

As Mr. Hildebrand used to say, when you assume, you make...

(also note the article specifically frames this speccing out as about training :) not just me suggesting it)

Sequoia is also increasingly embarrassing. A shame because it wasn't but 10 years ago that these firms seemed like they were leading the charge of world-changing innovation, etc...

Increasingly? This is the Sequoia who wrote thousands of words on Sam Bankman-Fried that uncritically said little more than “he’s so quirky and smart! ^_^” https://web.archive.org/web/20221027181005/https://www.sequo...

A great example. This wasn't but just 3 years ago, so definitely part of their increasing slide into embarrassment...

YC seems to be hopping on every trend more than what A16Z does. The latter still bet on momentum and not just heat in the game

> consistently the most embarrassing VC firm at any given point in time

Based on what? your feelings?

> being a VC is a very easy job these days.

There you go. Why hasn't everyone who have connections became VC.

Are they going to gift it to the POTUS for some favours or what now? Apple's gold bar type s...

It's gaudy as fuck but the charitable explanation is that they're mimicing the nVidia DGX aesthetic rather than Trump, which would at least make sense

Ugly. Just like their new logo and their president.

This article is a great way to showcase A16Z standinging head and shoulders above other VCs with REAL technical expertise in the partnership. Love reading this kind of stuff, but the article really needs a price to put this in perspective. It would be the VC that would ignore price, lol, but roughing this out it looks like it costs 45k to build this thing. Seems, at first glance, that this is a cost efficient way to dodge buying a Kia Carnival and get a tier-1 GPU workstation..

> This article is a great way to showcase A16Z standinging head and shoulders above other VCs with REAL technical expertise in the partnership.

what? a 12 year old with a titanic budget could put this PC together

I don't want to be insulting here, but have you sat down with a partner at a VC before? You may be surprised to discover their skill is rarely deeply technical...

yes, I worked for a VC funded micro startup for 5 years

I was not impressed by any of the partners

(but hey they were better than the PE partners I worked for immediately after)

There's almost 0 skill required to plug in components meant to be plugged in to each other...

The technical expertise of... buying four of the fanciest Nvidia GPU and plugging them into an off-the-shelf motherboard in an off-the-shelf chassis? A serious attempt at this kind of build would use the server variant of the RTX6000 and custom air ducts to cool them efficiently, but they packed four blower coolers like sardines so it no doubt sounds like a screaming jet engine under load.

Exactly, 4 blower GPUs!

https://www.youtube.com/watch?v=mvBeCSaaDxA

What did you say?? I can't hear you over these blowers!!

It’s a… low bar.

In all fairness, not even Lambda Labs offers workstations with RTX6000 server variants.

There are no RTX6000 "server" variants. These GPUs are designed for workstation use. I mean it is even in the name.

https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackw...

The workstation versions are fine if you're running one or maybe two cards with an airflow gap between them, but if you pack four of them right next to each other then you're going to have a bad time when the fans get going.

Ou, I was wrong. Unexpected. Are there (comparatively) a lot of uses for the that card in a server?

Maybe they’re a vibe coder and to them this looks like engineering

Reply on news.ycombinator.com