Hacker News Reader

Most discourse around GPU prices is nonsense right now. Some people using Spot prices for providers who won't have Spot capacity, some people using hourly rates for instances that are never in stock, some people ignoring commitment discounts.

Not to mention no one serious is serving this on 8xB200 instead of multiple nodes: the vast majority of Moonshot's inference work is focused on PD-disaggregation

View on HN

> Not to mention no one serious is serving this on 8xB200 instead of multiple nodes: the vast majority of Moonshot's inference work is focused on PD-disaggregation

The GPU price discourse is absurd, but many are serving models on single node setups when the model fits

You can't beat current API pricing running Kimi K3 a single node, so as I said anyone serious is not doing that for Kimi K3.

Not sure how you're getting "no one serves models on a single node" out of that.

I meant serious players often do serve on a single node. They can beat API pricing as well. Multi-node can add gain, but also adds a lot of deployment complexity so its just not always possible or optimal.

Reply on news.ycombinator.com