Hacker News Reader

Comment by USB5 | Hacker News Reader

I just built a workstation with a 32 core Threadripper Pro, 128 GB ECC RAM, Thunderbolt 4, and an RTX 4090 for $6,500.

Likely better on any workloads that are GPU heavy and fit in 24GB of vram.

Two data points from Apple:

  a) M2 Ultra 24 cores, 128GB ram, 2TB storage = $5,200
  b) M2 ultra 24 cores, 192GB ram, 4GB storage = $6,600

Likely 1/4th the size, 1/4th the power consumption, and 4x the ram bandwidth. Have you by chance played with any LLMs? Just saw a post that someone managed 5 tokens/sec with the llama 65B model.

I don't care about playing with LLMs, so this is simply meaningless to me.

But for many people training or running inference on large models is important and the apple approach actually makes that feasible in a desktop solution which is pretty awesome.

Reply on news.ycombinator.com