Hacker News Reader

Comment by mrb | Hacker News Reader

Note that in terms of pure compute performance the new 16nm Nvidia GTX 1080 (2560 shaders at up to 1733 MHz = 8873 SP GFLOPS) barely equals the performance of the previous-generation 28nm AMD Fury X (4096 shaders at up to 1050 MHz = 8602 SP GFLOPS). Of course the 16nm chip does so at a significantly lower TDP (180 Watt) than the 28nm chip (275 Watt), so it will be interesting to see what Nvidia can achieve at a higher thermal/power envelope with a more high-end card... I am waiting impatiently to see how AMD's upcoming 14nm/16nm Polaris chips will fare, but from the looks of it it seems like Polaris will beat Nvidia in terms of GFLOPS per Watt.

View on HN

The most interesting thing about AMD is their ability to create GPUs with superior synthetic specs than their NVIDIA competitor and yet they lose across the board in anything but embarrassingly parallel applications. Not only do they lose, but they lose big.

Here are two examples. First, the engine behind Folding@Home: https://simtk.org/plugins/moinmoin/openmm/BenchmarkOpenMMDHF...

See also AlexNet (Deep Learning's most prominent benchmark): https://github.com/amd/OpenCL-caffe versus GTX TitanX: https://github.com/soumith/convnet-benchmarks

TLDR: AMD losing by a factor of 2 or more...

So unless this changes dramatically, NVIDIA IMO will continue to dominate.

But hey, they both crush Xeon Phi and FPGAs so it doesn't suck to be #2 when #3 is Intel.

Which FPGAs in which workloads do they crush?

Arria 10: http://www.nextplatform.com/2015/08/27/microsoft-extends-fpg...

600 images/s inference (highest ever publically reported), projected to be as high as 900 images/s. ~$5000 per FPGA.

Compare and contrast with TitanX/GTX 980 TI now topping 5000 images/s. GTX 1080 will only be faster than this and $600.

https://github.com/soumith/convnet-benchmarks

And so far, no FPGA training numbers. But here are some distributed training numbers for $10Kish Xeon servers:

https://software.intel.com/en-us/articles/caffe-training-on-...

64 of them train Alexnet in ~5 hours. A DGX-1 does it in 2. Don't have/Can't get a $129K DGX-1? Fine, buy a Big Sur from one of several vendors and throw in 8 GTX 1080s as soon as they're out, implement "One Weird Trick", and you'll do it in <3. That ought to run you about $30-$35K versus >$600K for those 64 Xeon servers.

FPGAs OTOH are fantastic at low memory bandwidth embarrassingly parallel computation like bitcoin mining. Deep Learning is not such a domain.

I don't see anywhere on that page any benchmark about 5000 images/sec, maybe you should update your link to a page showing that info or update your claim to match what you link to?

Even your previous post had links to two page that were not measuring the same thing... and thus could not support your claims?

I think you need to clarify.

Based on the text you apparently couldn't be bothered to read, for AlexNet, a forward pass on 128 images (Input 128x3x224x224) takes ~25 milliseconds (ignoring O(n) dropout and softmax where n is 128). I'll let you do the math for the rest of this...

Yeah, it will be interesting to see how good AMD's Polaris chips will be, especially when it comes to energy efficiency. Fury was a step in the right direction (I'm wondering how much of this can be attributed to HBM's lower voltage etc.). But I'm even more interested in AMD Vega - the high end competitor to Nvidias GP 100 that is equipped with HBM2. Especially if AMD has an advantage over Nvidia when it comes to implementing HBM, since they developed it together with SK Hynx and have previous experience with Fury, or if this doesn't make a difference.

Vega is also exiting because it should be released when AMD's Zen CPU's will be released, and there were rumors about a HPC Zen APU with Vega and HBM.

Both Vega and Zen are supposed to be due sometime next year, correct?

Yes, Zen for the desktop (Summit Ridge) is even listed for end of 2016 [1], followed by first Zen APU's (Raven Ridge) sometime 2017 and Vega early 2017. [2]

[1] http://imgur.com/rRjKZDk [2] http://i.imgur.com/lkjgayO.jpg?1

If we're discussing future GPUs, does anyone know what exactly AMD means by "next-gen memory" when it talks about Navi?

At first I thought it means that HBM2 would be available across their whole lineup, but I think if that was the case they would've just said so.

I've also read a bit about how Nvidia wanted to go with HMC (Hyper Memory Cube) instead of HBM, but it seems it's still at least twice as expensive/GB, and they needed to go with high-end GPUs that have at least 16GB. HMC is not even at 8GB yet. Intel also seems to have adopted HMC for some servers.

So is the "next-gen" memory HMC, or something else? AMD is supposed to come out with it in 2018, hopefully at 10nm.

Was the difference between HMC and HBM not basically HMC had good latency comparable to normal main RAM like DDR3 plus good bandwith (up to 400 GB/s), while HBM had worse latency than DDR3 (like GDDR5) with a large bus and really good bandwith (1 TB/s)? Will indeed be interesting to see what AMD means with next gen memory.

No. HMC is fully buffered and provides a packet-based interface to the CPU it's connected into. This means that the CPU doesn't need to know anything about the memory it uses, and saves die space on the CPU side, but it adds latency. HBM has lower latency than HMC.

Also, GDDR5 does not have higher latency than DDR3. The memory arrays themselves are the same and exactly as fast, and the bus doesn't add any extra latency. GDDR5 doesn't trade latency for bandwidth, it trades density for bandwidth.

Things built to use GDDDR5 often have much higher latencies than things built to use DDR3, but this has nothing to do with the memory and instead about how the memory controllers in GPUs delay accesses to merge ones close to each other to optimize for bandwidth.

Thank you for the detailed explanation. Always thought the added latency came from GDDR5. Good to know.

AMD chips have almost always had a lead over Nvidia from a per-flops perspective, and AMD's architecture is definitely ahead of Nvidia as far as supporting integer operations and bitwise ops efficiently(see their dominance of cryptocurrency mining). However, Nvidia's software tooling has always been superb, and they basically own the GPGPU-for-HPC space because of it. On top of that, one of the biggest growth areas of consumers of GPGPU in recent years has been "deep learning", for which Nvidia has embraced and has great tooling and software for their hardware. AMD has potential to compete with Nvidia in this area, but they don't seem to have the resources to develop optimized Neural Network software to run on their GPUs like Nvidia does, so they are way behind. Basically, GFLOPS per Watt is nice and all, but if nothing you care about runs on the hardware, you won't buy it. AMD seems to have been stuck in this position for HPC, and it doesn't seem to be changing sadly.

AMD pulls off better compute in pure GFLOPS but you'll have to extract that compute power in OpenCL. AMD still doesn't have a good answer to CUDA.

OpenCL is however a fairly nice target for compilers, which opens up different ways of using it.

Witness also PyOpenCL versus PyCUDA - both are essentially equally convenient to use, while raw OpenCL C is a nightmare compared to CUDA C.

I think the near future in compilers is going to be dominated by accelerator capable LLVM variants (such as NVVM). Ideally, all Nvidia, AMD and Intel would and should need to do is to provide LLLVM based backends. Frontends just need a sane way of specifying SIMT. And no, loop + directive based stuff like OpenMP and OpenACC is IMO just a stopgap measure in order to serve higher ups the lie that portation is going to be swift and easy. There's basically one sane way to deal with data parallel problems, and that's to treat what's going on in one core as a scalar program with precomputed indices (i.e. the CUDA / OpenCL way).

Python has some good support for this already. I haven't done a project like this yet, but if setting up something fresh I'd immediately jump on NumPy interacting with my own kernels and some Python glue for the node level parallelism.

> AMD still doesn't have a good answer to CUDA.

I don't understand why AMD doesn't just jump on the CUDA bandwagon and support the API?

1. They have no say in the API so NV would just steer the API in ways to make sure it would be suboptimal on AMD hardware.

2. It's not just the API. It's the entire SDK that has NV specific hooks that AMD can't tap into

Alternatively they could make their own API, independent from OpenCL. I have written a small amount in each languages (a rainbow table generator), and found the CUDA version much more pleasant to use. CUDA made it really easy to get a good idea of the underlying hardware, and make architectural changes in order to make the code faster.

OpenCL (using an ATI card) was much harder to program, since the abstraction level was much higher. Writing two separate kernels and have them each be faster than a generic version that ends up compromising for compatibility.

The OpenCL one ended up being faster, but I suspect that's due to ATI hardware being superior at the time.

They do. They have a retargeting compiler. It's performance sucks.

They are working on something like that - http://www.extremetech.com/extreme/218092-amds-boltzmann-ini...

The current versions only work with a couple of latest AMD chips though. And CUDA is just one part of it, they have nothing equivalent to CuDNN

AMD are supporting open standards like OpenCL, which is effectively their answer to CUDA. The problem is that Nvidia refuse to keep their drivers up to date with the latest OpenCL capabilities. The latest version nVidia supports is OpenCL 1.1, but its now up to OpenCL 2.2. nVidia are years behind on OpenCL. This is pretty much the primary reason why CUDA seems so far ahead.

nVidia released new features and capabilities on CUDA that later versions of OpenCL also supported, but were not available on nVidia cards due to lack of OpenCL driver support. Hence, lots of people developed libraries and tools for CUDA, with great encouragement and support from nVidia (which attracted developers). These tools were useful and powerful, encouraging others to also use nVidia hardware to take advantage of those tools. CUDA has now taken root.

If nVidia were to start keeping their drivers updated with the latest version of OpenCL, i suspect CUDA's dominance would start to wane. Though the CUDA roots have gone deep, so it may take several years to reverse its dominance. However, its highly unlikely that nVidia will update their drives any time soon. The only thing that would probably force them is AMD gaining market dominance for a few years.

AMD are "supporting" OpenCL as in it works. If you compare the resources NVidia are putting into their deep learning support vs AMD it is miles ahead.

If you read between the lines of how dismissive AMD were of NVidia's work on self driving cars you can see they don't care about compute.

They are chasing the VR market, and that is fine.

AMD is also developing a tool to convert CUDA into HIP but I haven't looked into that

http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/...

https://github.com/GPUOpen-ProfessionalCompute-Tools/HIP

The Fury X is AMD's current flagship, not last-gen.

I meant previous-gen chip fab process (28nm) vs. new-gen (14/16nm).

Reply on news.ycombinator.com