Nvidia DGX Spark | Hacker News Reader

Nvidia DGX Spark

nvidia.com

15 pointstanelpoder10 comments

Open articleView on HN

Heard coming from the underground parking area at One Apple Park Way earlier today https://youtu.be/BMm4FFrSxMk

Heard coming from the Taiwan-infused PC clone manufacturer industry https://youtu.be/wIE5wwvicew

Meanwhile, at whatever planet OpenAI lives on https://youtu.be/615lNmMiOSQ

"ConnectX-7 Smart NIC"

https://resources.nvidia.com/en-us-accelerated-networking-re...

So those two QSFP ports can be configured for 400GbE IB and/or RoCE?

Also is it fair to say that this can run a desktop considering it has an HDMI port?

Yes, the whole point is that it is a compact desktop for AI applications, so, yes, it can run a desktop.

Nvidia DGX and HGX servers packed NVME, 8 H100 GPUs and 8 Infiniband cards don't run a desktop. They don't even have an HDMI port.

Am I missing something here or is inference going to be painful given the "low" memory bandwidth compared to, say, HBM2E?

273GB/sec with good FP4 performance should be fine for developers playing with inference. This isn't the kind of thing that you'd use for inference workloads supporting millions of users.

I'd like to see a inference benchmark vs the strix halo, which has better memory bandwidth and costs 2/3rds as much.

I guess since these devices aren't meant for production throughput, but rather about having enough RAM for local experimentation with large enough models, it's an ok tradeoff at this price point...

Have you seen anything with 128GB of HBM2E at anywhere near the DGX Spark's $3,000 price point?

The AMD Radeon VII has 16GB HBM2 and sold for $700 in 2019. I don't know how that would translate to today's HBM2E's pricing like if its price change follows that of GDDR's.

I honestly can't say I have however, that doesn't mean it couldn't physically exist/happen? Perhaps a "little" more cost but I'd be willing to bet people would gladly pay the premium for such a device. I'm also very curious to know what the BOM for an A100 actually is as well as HBM2E per GB.

Reply on news.ycombinator.com