Hacker News Reader

Interesting. What benchmark are you running? Getting it within 10% on such drastically different hardware looks wrong to me. And sounds like GPU is not a bottleneck in that particular case. Could it be that the system is CPU bound (with some horrid thread contention in TensorFlow or something)?

LambdaLabs' benchmark: https://github.com/lambdal/lambda-tensorflow-benchmark

It's certainly around 10-15% and it's not CPU bound, as the benchmark uses synthetic images without the need for data loading and preprocessing.

I cannot rerun them now for precise numbers because my ROCm packages are broken and I don't have the spare hours to fix things. Surely it can be fixed but needs fiddling.

You can still be CPU bound, even if the benchmark is using synthetic images. A quick check could be to look if your 2080 Ti is 100% utilized (with nvidia-smi). And calculate percentage of theoretical peak FP32 throughput that you are getting for both cards.

It is just that "more normal" result that you would expect to get, would be a drastic differences in performance. Note, Cvikli comment on "May 12" on that GitHub page:

* The result with RNN networks on 1 Radeon VII and 1080ti was close to the same

* Comparing convolutional performance the 4AMD and 4Nvidia, difference got really huge because of cuDNN for Nvidia cards. We can get more than 10x performance from the 1080Ti than the Radeon VII card. We find this difference in speed a little too big at image recognition cuDNN, I can't believe that this should happen and the hardware shouldn't be able to achieve the same.

Note also in that comment, it looks like 4xGPU system just didn't work.

It is hard to fully utilize GPU resources. Getting same result normally means that the same bottleneck is being hit. Algorithmic differences could result in drastic performance differences (10x). And as a final point of small things making a difference, note it is 1080Ti in that comment.

I fixed the installation and ran a benchmark again: https://news.ycombinator.com/item?id=21666411

The benchmark is not CPU bound, as evidenced by looking at rocm-smi, nvidia-smi (GPU usage at 100%) and htop (low CPU usage).

Nice. Your results could be more reproducible, if you'd include CuDNN version. Turing architecture support (and optimizations specific to DNN training) are still relatively recent and there are differences in performance between the versions [1]. Multiplying matrices is kinda tricky ;) [2]

[1] https://developer.nvidia.com/cudnn . [2] https://scholar.google.com/scholar?as_ylo=2019&q=nvidia+gemm

Cannot edit that comment any more, but it's driver version 430, CUDA 10.1, CuDNN 7.5.

We (lambda) are thinking about offering amd virtual workstation instances. One concern we had was that AMDs drivers have in the past had poor stability. Did you do burn in testing as well?

> Did you do burn in testing as well?

Not sure what that means. I did this: https://news.ycombinator.com/item?id=21666411

I run the fan at 100% and even after several thousand iterations it keeps its speed at 274-275 im/sec and 68 C temperature with the stock fan and a standard desktop chassis kept open.

Cool! Hey if you want to get together and chat about amd GPUs, I'm i@<lambda url>, happy to get you a coffee or beer or something.

What's drastically different between Radeon VII and 2080 Ti? They have basically the same peak fp32 tflops.

> it does work at about 90% the speed of an NVIDIA RTX 2080 Ti, but is much cheaper, which can be a good trade-off.

Is this 90% of fp32 performance, of 90% of fp16 performance?

FP32 on both.

What card/brand are you using?

I'd have to look up the brand, someone else ordered it.

Anyways, my complaints may be related partially to me using the xenial (Ubuntu 16.04) repo on bionic (18.04), because the there is no support for 18.04 LTS, even though it came out more than one and a half years ago. I also had to downgrade the kernel from 5.0 to 4.15 as 5.0 is not supported. I also had to force the installation of several 32-bit libraries on this 64-bit system, which took quite some googling and trial and error, otherwise installation didn't succeed. Little things like that.

float16 or bfloat16?

I wasn't aware of the difference, but now I looked. I've always used float16, not bfloat16 (tf.float16 is the usual float16), and it causes a large speed boost on Turing NVIDIA cards with tensor cores as opposed to float32.

But certainly bfloat16 sounds attractive with its larger dynamic range, as float16 requires workarounds like loss scaling to avoid rounding down to zero when the exponent gets too small.

Seems like bfloat16 is only available for TPUs in TensorFlow[0-1].

[0] https://github.com/tensorflow/tensorflow/issues/21612

[1] https://github.com/tensorflow/tensorflow/issues/21317

>it does work at about 90% the speed of an NVIDIA RTX 2080 Ti

Minor nitpick, but it looks like the Radeon vii is in the same tier as the 1080ti, and actually performs worse [1] in benchmarks.

1. https://gpu.userbenchmark.com/Compare/Nvidia-GTX-1080-Ti-vs-...

The two compare differently depending on the benchmark that is used.

Do you have any benchmarks which show comparable performance? Two quick searches now but I can only find video game based benchmarks but in every case the 1080TI outperforms the VII, which is kind of surprising and IMO isn't a good sign for ML performance.

https://www.phoronix.com/scan.php?page=article&item=radeon-v...

Here it performs VERY well for machine learning. Look at PlaidML.

That's using OpenCL on the NVidia card ("Once hitting PlaidML for various machine learning benchmarks with OpenCL").

No one does for machine learning that because the CUDA path is much better.

Video games are never a great comparison for raw compute performance. AMD's GCN based architectures have all been very good at compute relative to their graphics performance (recall how it was nearly impossible to find an MSRP AMD GPU during the mining boom). The Radeon VII is pretty ridiculous for compute, aided by the 1TB/s bandwidth due to it's HBM2 stacks.

I just ran LambdaLabs' ResNet50 training benchmark (I could do more models, but this is it for now) on a few machines I have access to:

TensorFlow 1.14.4

    git clone https://github.com/lambdal/lambda-tensorflow-benchmark.git --recursive
    python lambda-tensorflow-benchmark/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --optimizer=sgd --model=resnet50 --num_gpus=1 --batch_size=64 --variable_update=replicated --distortions=false --num_batches=10000 --data_name=imagenet

  FP32
  - NVIDIA GTX 1080 Ti: ~215 images/sec
  - NVIDIA RTX 2080 Ti: ~300 images/sec
  - NVIDIA TITAN RTX: ~320 images/sec
  - NVIDIA Tesla V100: ~383 images/sec
  - AMD Radeon VII: ~275 images/sec

  FP16
  - NVIDIA GTX 1080 Ti: ~277 images/sec
  - NVIDIA RTX 2080 Ti: ~495 images/sec
  - NVIDIA TITAN RTX: ~518 images/sec
  - NVIDIA Tesla V100: ~725 images/sec
  - AMD Radeon VII: ~373 images/sec

So in FP32 for this ResNet50 benchmark the Radeon VII is about 9-10% slower than the 2080 Ti and about 25-30% faster than the 1080 Ti.

In FP16 it is 25% slower than the 2080 Ti and 35% faster than the 1080 Ti.

Sure, it's just one benchmark, but ResNet is quite representative of deep learning architectures in vision.