Hacker News Reader

I read that as it runs in data centers (H100 GPUs) or high-end desktops/laptops (Strix Halo?).

I'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.

Excellent! I have a Framework Desktop with 128GB on preorder—really looking forward to getting it.

Well if nVidia wasn't late, it would be runnable on nVidia project Digits.

Yes, they are late to the party. Maybe they do not want to eat into the RTX Pro 6000 sales. In the meantime, there is the AMD Ryzen™ Al Max+ 395.

I actually tried to ask the Model about that, then I asked ChatGPT, both times, they just said that it was marketing speak.

I was like no. It is false advertising.

Don’t forget about mac studio