Hacker News Reader

What is the memory bandwidth? Otherwise, it’s not clear what are we buying.

Memory Bandwidth 273 GB/s

Not comparable with an H series gpu. I am not sure what kind of applications make sense but I am sure that if it sells enough, developers will find a way to squeeze good stuff out of this.

Edge inference most likely. Its FP4 performance is about 1/3 of 5090, power 170W for the whole thing. It can run big model or several small. Shifting balance to memory favors MoE. Would be nice to see FP32 numbers, they are used in training. My guess about 20 TFLOP, may be more, but 5090 is still times better.

Is this saying that it is focussed on inference and would be less cost-effective for trainimg as compared to alternatives?

Reply on news.ycombinator.com