Hacker News Reader

Comment by atty | Hacker News Reader

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their recent supercomputer deals include 100 million for software development for ROCm/HIP.

However, considering how robust CUDA is compared to ROCm, I feel like all Nvidia would need to do to take the HPC market back completely would be to get close to AMD’s current FP64 performance. I don’t think anyone would buy AMD if prices were comparable and FP64 performance was anywhere in the ballpark. It’ll be very interesting to see Nvidia’s new cards, hopefully next year.

View on HN

> I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning.

I develop scientific simulation software, and I can't tell you how happy I'm about it. Because while doing high precision work, GPUs fall flat fast.

I also work at a HPC center, and there's mountains of FP64 dependent applications running on CPUs. Moving them to GPUs will bring a lot of improvements in a lot of disciplines. It's not uncommon to let things run for a week on multiple nodes for meaningful results.

If I could in any way afford it I'd buy one of these just for doing (hobby) n-body simulations and fractal rendering. I bought a Radeon VII specifically for this and it died a month or so after I got it, with no replacement possible because they were EOL :( So badly want a replacement, but with GPU prices as they are now, there's just no way to justify buying a 2nd hand one.

Right now I think the best you can do for FP64/$ is AVX-512 CPUs and Radeon GPUs.

Not sure I really trust AMD not to trip over their own feet when it comes to software unfortunately.

Hats off to them if they manage to really land a blow on NV here.

Also potentially keep an eye on Intel a few years down the line? Since they can actually do software in my experience (and, get this AMD, document the software!)

Supercomputer market is still there. Intel Sapphire rapids owes its existence to the next US nuke design (to be made on Aurora supercomputer.)

I don't think so. The new ISA that is in the SPR is mostly about deep learning: it supports int8 and bfloat16 (https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-...). You can emulate higher precision using bfloat16 (https://arxiv.org/abs/1904.06376), but I have not seen this used in the wild.

You know about surrogate models being run in simulations (physics informed neural network and all that stuff) the US is investing a lot in this. Maybe then it makes sense why bfloat16 exist in SPR.

Nvidia can’t improve FP64 performance to near parity without sacrificing a lot in the process, which would make them more vulnerable on the deep learning side of things.

?? Build two separate product lines.

With which engineers, managers, etc? Intel has long faced the same issues, they want their best R&D on x86 CPU’s so they had major issues expanding into other product lines.

Reply on news.ycombinator.com